On 2011-12-23 03:24:27 +0000, Jonathan M Davis <jmdavisProg@gmx.com> said:

> On Thursday, December 22, 2011 10:29:59 Michel Fortin wrote:
>> I'd tend to say that for general purpose time representation not
>> involving local time, SysTime is suboptimal because it forces you to
>> carry around a pointer to a time zone. Imagine an array of SysTime all
>> in UTC and the space wasted with all those pointers referencing the UTC
>> time zone object.
>> 
>> It should be very easy to make a separate type, let's say UTCTime, and
>> allow SysTime to be constructed from it and to be implicitly converted
>> to it (with alias this). Then put UTCTime in a different module from
>> SysTime and you can deal with time in UTC without having to ever import
>> the module with SysTime the time zone class it wants.
>> 
>> Then redefine all APIs not dealing with local time so they work with
>> UTCTime instead of SysTime.
> 
> That could certainly be done, but it complicates things that much more. The
> idea, at least, of SysTime was that it would deal with all of the time zone
> stuff correctly without you having to worry about it unless you wanted to deal
> with the time zone stuff, in which case it would give you those capabilities.
> That requires having it to carry the time zone around.
> 
> Something like UTCTime would allow you to carry the time around without the
> time zone, but then anyone who wants to be able to do stuff like convert it to
> a string or get its year or anything like that is almost certainly going to
> want it in a particular time zone (probably local time) rather than UTC, so it
> increases the burden on the programmer to deal with something like UTCTime. If
> you're dealing with anything beyond comparing times or adding durations to
> them, you're going to need the time zone, and in most cases, UTC is not the
> one that people are going to want.
> 
> So, it harms usability IMHO to using something like UTCTime instead of
> SysTime, and just to save yourself the cost of the reference for the time
> zone? If you're _that_ worried about the space, you can always use a SysTime's
> stdTime property or toUnixTime and get an integral value to store. Granted,
> that's not as safe as something like UTCTime, since it's a naked number, but I
> really don't think that the cost of that reference is generally an issue.

Well, what I'm getting at is that most of the time you don't care which time zone the time was recorded in, so you don't need to attach a time zone to it, you only need to take the time zone into consideration when formatting as a string, and then you mostly always use local time.

The real issue remains that you can't use SysTime without including all the code for all the time zones. Think about this: if you don't need to carry around the time zones but instead only ask for a time zone when formatting as a string, you much less need time zones to be polymorphic. The time zone could be a template argument to the formatting functions for instance.

On the other hand, if you need to carry the associated time zone along with the time, then things gets more complicated and a polymorphic time zone type tend to solve that problem well. But how many of us need to carry a time zone with a time value?

So in my opinion associating a time zone with SysTime was a mistake. Not just because it forces you to carry around an extra pointer, but mostly because it forces time zones to be polymorphic which brings all the drawbacks of a class: less inlining and worse performance due to virtual functions, and all the virtual functions need to be included in the binary even if you don't use them. It does benefit the use case where you need to tag a time with a specific time zone, but that sounds rather specialized to me.

-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/

On 2011-12-23 02:10, Michel Fortin wrote:
> On 2011-12-22 17:22:59 +0000, Jacob Carlborg <doob@me.com> said:
>
>> On 2011-12-22 16:56, Michel Fortin wrote:
>>> The benefit of referencing classes within module info: you can
>>> instantiate them using Object.factory, if they have a default
>>> constructor. We pay a heavy price compared to what we get with this very
>>> limited runtime reflection.
>>
>> It's a really nice feature to have when implementing serialization.
>
> True, for serialization and other things. I'm not arguing against the
> feature.
>
> What I am observing is that the the capabilities of D's runtime
> reflection are quite small compared to the footprint it has. We often
> hear on this list about how adding more information about functions and
> fields to typeinfo would bloat executables, yet I am under the
> impression the biggest part of that bloat (the code of all the virtual
> functions and whatever they call, even if you don't use those functions
> or even the class) is already part of each and every D executable, only
> we don't realize this because it is not exposed in the API, it just sits
> there in a mostly non-interpretable form (the vtable).
>

If that's the case I think we should take advantage of the already available data in the executables to implement better runtime reflection.

-- 
/Jacob Carlborg

On 2011-12-23 03:21, Walter Bright wrote:
> On 12/22/2011 11:25 AM, Piotr Szturmaj wrote:
>> I wish D could support partial modules - partial as analogy to C#'s
>> partial
>> classes.
>>
>> module std.datetime-unit1;
>> import std.datetime-unit2;
>> // dash allowed only in submodules with the same module name
>> ...
>>
>> module std.datetime-unit2;
>> import std.datetime-unit1;
>> ...
>>
>> // then
>>
>> module whatever;
>> import std.datetime; // as usual
>
>
> I have no idea why anyone would want this. (Is it because the file is
> too big to fit on a floppy disk? <g>)
>

std.datetime is kind of hard on IDE's and text editors. I know TextMate has problems with it and Descent has problems with a lot less code.

-- 
/Jacob Carlborg

On 2011-12-23 03:38, Jonathan M Davis wrote:
> On Thursday, December 22, 2011 13:38:51 Andrei Alexandrescu wrote:
>> On 12/22/11 1:32 PM, Jonathan M Davis wrote:
>> [snip]
>>
>> Now that we got to talk about std.datetime, here are three things that I
>> think we could do to make it more manageable.
>>
>> 1. Put files in data. I find it a tad awkward that we have time zone
>> information in hardcoded strings inside the code. That means any such
>> change would have us redistributed Phobos. I'm thinking a small data
>> file would be more appropriate. Better yet, hook into OSs timezone
>> information and let the OS worry about keeping that timely.
>
> The only reason there are any hard-coded time zone names is that they're
> required to convert between the names used by Posix and those used by Windows.
> So, you _can't_ hook into the OS information and get them. Now, conceivably,
> you could move that information into a file and then parse the file when it's
> needed. That would obviously be less efficient, but creating a WindowsTimeZone
> or PosixTimeZone (which is what they'd most frequently be needed for) isn't
> exactly terribly efficient to begin with, since you have to read in the time
> zone information from from the disk or from the registry (which is probably on
> disk). So, that's not unreasonable.
>
>> 2. datetime == time + date. We could reduce std.datetime to "public
>> import std.time, std.date;" and define:
>>
>> (a) std.time ->  everything having to do with sheer time information, no
>> date-related oddities. That means the largest formalized interval would
>> be the week.
>>
>> (b) std.date ->  all of the bizarre calendar stuff, dealing with months
>> and more. Naturally std.date would use std.time.
>
> Well, the only time point type which only deals with time and not dates is
> TimeOfDay. Date, DateTime, and SysTime all deal with dates. You can't really
> get away from dealing with dates once your type holds more than 24 hours worth
> of time unless it's a duration as opposed to a time point. So, I really don't
> think that trying to split std.datetime into std.date and std.time makes much
> sense.
>
> A better division would be to put SysTime in a module and TimeOfDay, Date, and
> DateTime in another. SysTime deals with the system time, has a time zone, and
> is intended for use with stuff which isn't calendar-based (timestamps and file
> times being good examples - anything where you need the absolute time when it
> occured), whereas the others don't have a time zone and therefore _are_
> calendar-based. However, they all share common code, so they'd either need to
> duplicate that code or any modules that they're split up into need to be in
> the same package. They _could_ both be sitting in std directly, but that would
> give package access to completely unrelated functions. It's also possible that
> we'll have more date and/or time related modules in the future (for instance,
> having one for handling date-recurrence patterns would be nice), and if that
> occurs, it makes that much more sense to use a sub-package rather than std.
>
> If we're splitting it up, there's also the question of how far we want to
> split it up. In the extreme case, we could put every struct and class in its
> own module, though that's going too far IMHO. But we're probably going to want
> to go farther than just splitting it in two. In addition to the benchmarking
> functionality, I'd like to see the time interval and range functionality in a
> separate module, and there's the question of whether the time zone stuff should
> be in its own module - though there's not  much point to the time zones
> without SysTime, so I'm not sure whether that's really valuable.
>
> In any case, if we keep std.datetime and have it publicly import the other
> modules, we can split it up more or less however we like, but having a sub-
> package would make the most sense IMHO. My original std.datetime proposal was
> that way, but it was split badly, and we didn't really have any sub-package
> stuff in Phobos beyond the C stuff and Windows stuff at the time, but that has
> been slowly changing.
>
> - Jonathan M Davis

I think a sub-package would be nice for all time and date related code.

-- 
/Jacob Carlborg

On 2011-12-23 04:15, Walter Bright wrote:
> On 12/22/2011 2:00 PM, Jacob Carlborg wrote:
>> I like the 1:1 mapping between file and modules, and folders and
>> packages as well.
>
> That feature is something that took some getting used to for a lot of
> people, but when they got used to it they preferred it.
>

I had no problems with it. Although I came from a Java background and hadn't used C++ before I learned D.

-- 
/Jacob Carlborg

On 2011-12-23 11:14, Joshua Reusch wrote:
> Am 22.12.2011 22:57, schrieb Jacob Carlborg:
>> On 2011-12-22 19:32, Walter Bright wrote:
>>> On 12/22/2011 9:22 AM, Jacob Carlborg wrote:
>>>> On 2011-12-22 16:56, Michel Fortin wrote:
>>>>> The benefit of referencing classes within module info: you can
>>>>> instantiate them using Object.factory, if they have a default
>>>>> constructor. We pay a heavy price compared to what we get with this
>>>>> very
>>>>> limited runtime reflection.
>>>>
>>>> It's a really nice feature to have when implementing serialization.
>>>
>>> Sure, but we need to be aware of class overhead, and not use classes
>>> unless necessary. I.e. a class shouldn't be used to merely create a
>>> namespace. Classes also should not be used if it is not intended to be a
>>> polymorphic type.
>>
>> Exactly. But I'm referring to deserializing classes, I don't care what
>> they're used for.
>>
>
> IMHO, the user should know the type of the object he wants to
> deserialize, so it can be done only with compile-time reflection.

That might work, I haven't thought about it.

-- 
/Jacob Carlborg

On Friday, December 23, 2011 12:02:26 Stewart Gordon wrote:
> What is the use case for including a full date in some externally defined format within a longer formatted date string?  ISTM the way to do this is to define a function that just generates this format straight off.  In my library, toShortFormatString and toLongFormatString are already examples of this.
>
> > It also means that you can't reuse letters.
> 
> At the moment only 12 letters are used.  I can't see the whole alphabet being used up any time in the foreseeable future.

It's not that I want to put %ctime and %mpeg7 in the same string. It's that I don't want to have to go define a function for every single one of them. It's a much smaller hit to the API to have flags for them.

Another alternative is to supply an enum of format strings for a variety of formats, though that also expands the API a bit (not as badly much though). That may a be better approach for known formats. That doesn't work as well though in cases where the format strings to toCustomString and fromCustomString aren't identical (which happens in the case of mpeg-7, since it uses %cond). So, I don't know. Regardless, I think that the abliity to have more or less arbitrary strings used for flags is valuable.

> I think I could expand my scheme to include ISO signed-year notation easily enough.  In the mpeg-7 standard, does the denominator of the fractional second have to be the smallest possible power of 10, or is F20/1000 or F0/1000 allowed just as well?

If it's 0, there is no fraction, just like there's no decimal if it's 0 for ISO. I don't _think_ that it has to be the smallest possible power of 10, but I don't remember at the moment. I'm going to have to track down the info on the spec again. I'd figured it out for work previously, but unfortunately I lost whatever bookmarks I had with the info.

I don't want to shell out money for the spec if I don't have to. Even if I were willing to buy it, it's sold in 12 parts, and I'd probably end up buying several parts before I found the one with the definition of its time format, wasting that much more money. So, I'll have to go digging online again.

- Jonathan M Davis

On Friday, December 23, 2011 09:36:14 Michel Fortin wrote:
> Well, what I'm getting at is that most of the time you don't care which time zone the time was recorded in, so you don't need to attach a time zone to it, you only need to take the time zone into consideration when formatting as a string, and then you mostly always use local time.
> 
> The real issue remains that you can't use SysTime without including all the code for all the time zones. Think about this: if you don't need to carry around the time zones but instead only ask for a time zone when formatting as a string, you much less need time zones to be polymorphic. The time zone could be a template argument to the formatting functions for instance.
> 
> On the other hand, if you need to carry the associated time zone along with the time, then things gets more complicated and a polymorphic time zone type tend to solve that problem well. But how many of us need to carry a time zone with a time value?
> 
> So in my opinion associating a time zone with SysTime was a mistake. Not just because it forces you to carry around an extra pointer, but mostly because it forces time zones to be polymorphic which brings all the drawbacks of a class: less inlining and worse performance due to virtual functions, and all the virtual functions need to be included in the binary even if you don't use them. It does benefit the use case where you need to tag a time with a specific time zone, but that sounds rather specialized to me.

The core issue with not carrying around a time zone is that you get conversion problems. People do dumb stuff like convert time_t values from UTC to local time and back again, which causes all kinds of bugs. For most time zones, you cannot convert them back to UTC correctly in the general case. The hours when DST switches occur screw that up. So, in order to handle time correctly, you need to always keep it in UTC until you need to present it (e.g. converting it to a string) or need to know something about its date (such as what year or month it's in). If you're actually storing a converted time value, you're doing it wrong and will almost inevitably have bugs. I did std.datetime in the first place, because I was sick of having to fix time-related bugs at work and wanted D to get it right. That requires a type which holds the time in UTC and knows its time zone.

Now, "knowing its time zone" can be dealt with in several ways. One way is to integrate it into the type. Your proposed UTCTime would do that. It's always in UTC. You could also have another type which was called something like LocalTime which held the time in UTC internally but any functions on it which needed time zone conversions used the appropriate calculations for the local time zone instead of UTC. There are three problems with that approach though.

1. It doesn't scale.
2. You have to care about the time zone of the time object when you pass it to
functions.
3. Code duplication.

#1 isn't really a big deal if all you care about is UTC and local time. Dealing with other time zones can be very useful, but is definitely more of a niche issue. It's not all that hard to end up in situations where it matters though when you're getting data from another computer. There are lots of applications out there which have to deal with that, but I don't know how common it ulimately is. Certainly, where I work, we have to do it all the time. And I know that Adam Ruppe has to deal with similar issues with his web software based on questions he asked about std.datetime previously.

#2 can be dealt with with templates to some extent, but functions still still have to return a specific time type. So, unless you do something like make it so that you pass the type you want returned as a template argument, you have to deal with a specific time type and potentially convert it to whichever one you actually want. This isn't necessarily a big deal, but it does complicate things. With a solution like SysTime, you don't generally have to care what the time zone of a particular SysTime actually is.

#3 can be dealt with in several ways - including using a bunch of free functions instead of having the functions on the types themselves, but that's no OO at all, and highly ugly IMHO. Mixins would be a better approach, but regardless, you have to deal with the fact that you're having to effectively define a bunch of functions multiple times.

So, all around, a solution where the time zone is treated as a property of the type which can be changed seems like a much cleaner solution to me. It's also much more flexible and powerful, because you _can_ then define more time zones than just two. The current solution handles time correctly without generally requiring the programmer to care about what time zone the time is in and yet still allows for more advanced time zone usage if the programmer wants it.

The only real downside that I see here is that for those trying to keep their executables as absolutely small as possible and don't care about anything beyond comparing file times end up with a larger executable due to features that they don't care about. My greatest fear here is that we would be pushed into an inferior solution which would increase the likelihood of bugs just because some folks want a slightly smaller executable.

- Jonathan M Davis

On 12/23/2011 7:19 AM, Jacob Carlborg wrote:
> On 2011-12-23 03:21, Walter Bright wrote:
>> On 12/22/2011 11:25 AM, Piotr Szturmaj wrote:
>>> I wish D could support partial modules - partial as analogy to C#'s
>>> partial
>>> classes.
>>>
>>> module std.datetime-unit1;
>>> import std.datetime-unit2;
>>> // dash allowed only in submodules with the same module name
>>> ...
>>>
>>> module std.datetime-unit2;
>>> import std.datetime-unit1;
>>> ...
>>>
>>> // then
>>>
>>> module whatever;
>>> import std.datetime; // as usual
>>
>>
>> I have no idea why anyone would want this. (Is it because the file is
>> too big to fit on a floppy disk? <g>)
>>
>
> std.datetime is kind of hard on IDE's and text editors. I know TextMate has
> problems with it and Descent has problems with a lot less code.

I'm not going to defend programs that can't handle large text files, but in D one could always:

module std.datetime;

mixin(import("datetime-unit1.d"));
mixin(import("datetime-unit2.d"));

On 23/12/2011 21:24, Jonathan M Davis wrote:
<snip>
> It's not that I want to put %ctime and %mpeg7 in the same string. It's that I
> don't want to have to go define a function for every single one of them. It's a
> much smaller hit to the API to have flags for them.

So this is the principle of API design you go by - aim for one function that does everything? :)

OK, so if there's room for it in a given format string scheme, it doesn't really do any harm to have it.  But there's no point convoluting a scheme just to make room for named full-timestamp formats.

> Another alternative is to supply an enum of format strings for a variety of
> formats, though that also expands the API a bit (not as badly much though).

This seems to me a good idea.  It avoids implementation bloat, and these symbolic constants can be used just as they are or concatenated into the format string if you want to put something else there as well.

<snip>
> I don't want to shell out money for the spec if I don't have to. Even if I
> were willing to buy it, it's sold in 12 parts, and I'd probably end up buying
> several parts before I found the one with the definition of its time format,
> wasting that much more money. So, I'll have to go digging online again.

I've just discovered it on the ISO website.  It does seem an extortionate price for a spec.  Any idea where the money goes?

Stewart.