>> If the compiler and/or linker don't strip unused symbols, then how on earth is
>> importing the module _not_ going to pull in everything in it save for
>> uninitialized templates?
>
> There are data that needs to be there, but is never symbolically referenced, for example, the exception handler tables. The compiler does the best it can, like emitting one object file per function, but it always must behave conservatively.

AFAIK, dmd used to remove unused modules but then bearophile complained that unittests are not run for imported but otherwise unused modules, so it looks like frontend's responsibility, not backend's.

On 2011-12-22 15:44, Piotr Szturmaj wrote:
> Walter Bright wrote:
>> My first thought is that std.datetime is already very large. Few will
>> need a custom date formatter, so it should be in a separate module to:
>>
>> 1. reduce cognitive load on the programmer
>>
>> 2. reduce the overhead pulled in for every program that may want to use
>> an std.datetime function, but not need custom formatting
>
> Why not just extract unittest code to separate module?

Exactly, that's what I've said several times.

See my reply to Jonathan: http://dfeed.kimsufi.thecybershadow.net/discussion/post/jcv92h$1kus$1@digitalmars.com

-- 
/Jacob Carlborg

On 2011-12-22 11:03:13 +0000, Jonathan M Davis <jmdavisProg@gmx.com> said:

> On Thursday, December 22, 2011 02:12:31 Walter Bright wrote:
>> Timezone information is not necessary to measure elapsed time or relative
>> time, for example.
> 
> The type requires it though. No, comparison doesn't require the time zone, but
> many (most?) of the other operations do. And the type can't be separated from
> the time zone. That's part of the whole point of how SysTime is designed. It
> holds its time internally in UTC and then uses the time zone to adjust the
> time whenever a property or other function is used which requires the time in
> that time zone. That way, you avoid all of the issues and bugs that result
> from converting the time. The cost of that is that you can't not have a time
> zone and use SysTime. So, if someone cares about saving that little bit of
> extra size in their executable by not using the time zone, they're going to
> have to use the C functions or design their own time code.

I'd tend to say that for general purpose time representation not involving local time, SysTime is suboptimal because it forces you to carry around a pointer to a time zone. Imagine an array of SysTime all in UTC and the space wasted with all those pointers referencing the UTC time zone object.

It should be very easy to make a separate type, let's say UTCTime, and allow SysTime to be constructed from it and to be implicitly converted to it (with alias this). Then put UTCTime in a different module from SysTime and you can deal with time in UTC without having to ever import the module with SysTime the time zone class it wants.

Then redefine all APIs not dealing with local time so they work with UTCTime instead of SysTime.

-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/

On Thu, 22 Dec 2011 12:03:13 +0100, Jonathan M Davis <jmdavisProg@gmx.com> wrote:

> On Thursday, December 22, 2011 02:12:31 Walter Bright wrote:
>> Timezone information is not necessary to measure elapsed time or relative
>> time, for example.
>
> The type requires it though. No, comparison doesn't require the time zone, but
> many (most?) of the other operations do. And the type can't be separated from
> the time zone. That's part of the whole point of how SysTime is designed. It
> holds its time internally in UTC and then uses the time zone to adjust the
> time whenever a property or other function is used which requires the time in
> that time zone. That way, you avoid all of the issues and bugs that result
> from converting the time. The cost of that is that you can't not have a time
> zone and use SysTime. So, if someone cares about saving that little bit of
> extra size in their executable by not using the time zone, they're going to
> have to use the C functions or design their own time code.
>
>> Can it be added with PIMPL? PIMPL is good for more than just information
>> hiding, it can also be a fine way to avoid pulling in things unless they
>> are actually dynamically used (as opposed to pulling them in if they are
>> statically referenced).
>
> I don't follow you. You mean use PIMPL for the time zone? I haven't a clue how
> you're going to do PIMPL without .di files, and Phobos doesn't use .di files
> (and arguably _can't_, because it would destroy inlining and CTFE). Not to
> mention, PIMPL would make SysTime less efficient, because in order to avoid
> needing to know what the functions are on a TimeZone, you'd have to hide the
> bodies of a number of SysTime's functions, which would disallow inlining and
> CTFE (not that CTFE is terribly likely to be used with SysTime, but inlining
> could be very important). You'd be losing efficiency of execution just to save a
> few KB in the executable.
>
> Rearranging stuff to save some size in the executable without costing efficiency
> is one thing, but if it's going to cost efficiency, then I'm generally going to
> be against it.
>
I recently pimpled the core.thread module.
https://github.com/dawgfoto/druntime/commit/4626c20b66a647a497b9086b104e10ea89cfef02
The compiler is not really your friend here, e.g. it requires to compile the
implementation separately. But if your type is already used by reference then
this is a good option to reduce coupling and improve ABI stability.
Are you sure that inlining is a performance issue?

>> Please check and see if additional symbols are pulled in or not. I've seen a
>> lot of template code where the author of that code never checked and was
>> surprised at the instantiations that were happening that he overlooked.
>
> Nothing would pull in toCustomString or fromCustomString in unless the user
> decided to call them, because they're templated and no other functions in
> Phobos are going to use them at this point (if ever). What exactly will be
> pulled in when they _are_ called, I don't know, because they're not completed
> yet. It probably wouldn't be much though, since toCustomString is basically a
> fancy toString. But there's a good chance that it's stuff that's already being
> pulled in for the standard string functions (e.g. toISOExtString).
>
> - Jonathan M Davis

On 2011-12-22 08:14:21 +0000, Jonathan M Davis <jmdavisProg@gmx.com> said:

> On Thursday, December 22, 2011 00:05:58 Walter Bright wrote:
>> We must deal with our imperfect tools as they are. For example, I am not
>> going to rewrite the Linux ld linker, and the OSX linker, and the FreeBSD
>> linker, etc.
> 
> If the compiler and/or linker don't strip unused symbols, then how on earth is
> importing the module _not_ going to pull in everything in it save for
> uninitialized templates?

I'm pretty sure, even though it's been a long time I've looked at those details, that the problem is caused by classes. Basically, the typeinfo for all classes in a module are referenced by the generated module info structure. The class's typeinfo points to many things including the default constructor and vtable, the vtable points to all virtual functions, which finally points to everything used by those functions. You can thus easily have all the code in the module referenced indirectly by the module info, so the linker can't strip it. That might be why std.datetime has a heavy footprint, even when you use only a small part of it. But still, you should look at the generated code to test this hypothesis.

The benefit of referencing classes within module info: you can instantiate them using Object.factory, if they have a default constructor. We pay a heavy price compared to what we get with this very limited runtime reflection.

Things you can do if this hypothesis proves to be correct: make it so that any virtual function in your classes are short and do not depend on heavy functions, as all the code referenced by those functions will be dragged along. Don't forget that final functions are still virtual if they override a function in a base class or defined in an interface. Also, don't use a class if you don't need one, or if you need one put it in a separate module that you only import when you need it.

-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/

On Thursday, December 22, 2011 15:44:48 Piotr Szturmaj wrote:
> Walter Bright wrote:
> > My first thought is that std.datetime is already very large. Few will need a custom date formatter, so it should be in a separate module to:
> > 
> > 1. reduce cognitive load on the programmer
> > 
> > 2. reduce the overhead pulled in for every program that may want to use an std.datetime function, but not need custom formatting
> 
> Why not just extract unittest code to separate module?

Because it's harder to maintain that way. The interval types are templated, so their unit test blocks can't be next to the functions that they're testing, and it's a royal pain to manage those tests in comparison to the others. It's _really_ nice to have the tests right next to the functions that they're testing.

- Jonathan M Davis

On 12/22/11 4:40 AM, Jacob Carlborg wrote:
> It would break code, but it's better to break it sooner than later.
> Seems more and more like D should support importing all modules in a
> package, that Java does:
>
> import foo.*;

I think this style is currently discouraged in Java.

Andrei

On 12/22/11 6:50 AM, Jacob Carlborg wrote:
> You should treat your testing code just as you treat your "regular"
> code. Just as well designed, just as modularized, just as effective,
> just as clean. The testing code is in fact just as much part of the
> "regular" code as the rest of the code.

This. YES. A liability of the current std.datetime is that it assumes that unittest code is exempt from the rules that apply to regular code. I am increasingly worried about that module. It has been argued that its sheer size is not a problem, but somehow the task of accounting for that has taken a life of its own - e.g. we can't test std.datetime like everything else in Phobos, it needs its own version.

Andrei

On 2011-12-22 16:56, Michel Fortin wrote:
> The benefit of referencing classes within module info: you can
> instantiate them using Object.factory, if they have a default
> constructor. We pay a heavy price compared to what we get with this very
> limited runtime reflection.

It's a really nice feature to have when implementing serialization.

-- 
/Jacob Carlborg

On 12/22/11 4:20 AM, Stewart Gordon wrote:
> Looks complicated compared to mine at first sight.

++

I strongly prefer the look of Stewart's formatting strings; at a glance it is much easier for me to understand "yyyymmdd`THHiisszzz" than "%4yplus%04+Y%02M%02D{T}%02H%02m%02s%f%tz".