Thread overview
February 29

D's support for shared libraries has historically presented challenges for users, especially when dealing with complex use cases. This proposal aims to improve the user experience by establishing a user story that will provide a consistent and user-friendly approach to usage and compilation of binaries without a thorough understanding of linkers.

With the recent resolution of dmd's exportation support on Windows (druntime support is still under development), we are now ready to focus on making D's shared library capabilities more accessible and usable for common use cases.

Links: current latest

Some features of this proposal:

  • The export attribute is no longer a visibility modifier
  • A new switch is added -extI, the external import path switch
  • Templates are not exported by default (prevents template emission bugs like 23255 by requiring all templates in modules out of binary to instantiate)
  • The D interface generator will be able to produce .di files that are suitable for usage with both static libraries and shared libraries.
  • Gives a name to using export versus the visibility override switch as positive and negative notation
  • Officialize the Have_ prefix used in dub and introduce InBinary_ and Compiling_ to enable very fined grained control over symbol mode as part of export
  • Generated symbols will be exported for a given encapsulation unit that has exportation

There has already been quite a bit of back and forth between me and Walter with the aid of Adam Wilson (Thanks!), for this reason this proposal is skipping the idea step.

In a previous version of this proposal a focus upon anything that affects shared libraries was taken. In this version of the proposal a focus upon things required for common use cases is taken, and when these are not taken direct failure to implement the use case in D occurs.

March 08

On Thursday, 29 February 2024 at 11:15:07 UTC, Richard (Rikki) Andrew Cattermole wrote:

>

D's support for shared libraries has historically presented challenges for users, especially when dealing with complex use cases. This proposal aims to improve the user experience by establishing a user story that will provide a consistent and user-friendly approach to usage and compilation of binaries without a thorough understanding of linkers.

With the recent resolution of dmd's exportation support on Windows (druntime support is still under development), we are now ready to focus on making D's shared library capabilities more accessible and usable for common use cases.

Links: current latest

Can a module, which is compiled as part of the application, still contain prototypes for symbols from a DLL? Here is an example:

extern(C++) void foo(const char *s, size_t length);
void foo(string s)
{
    foo(s.ptr, s.length);
}

Function foo is implemented in a C++ library, which could be compiled as a DLL. The module contains a prototype for the function, but also an overload with a more convenient interface for D. The module needs to be compiled, because it contains the overload. Can the prototype still be imported from a DLL and be part of the same module?

March 09
On 09/03/2024 5:49 AM, Tim wrote:
> On Thursday, 29 February 2024 at 11:15:07 UTC, Richard (Rikki) Andrew Cattermole wrote:
>> D's support for shared libraries has historically presented challenges for users, especially when dealing with complex use cases. This proposal aims to improve the user experience by establishing a user story that will provide a consistent and user-friendly approach to usage and compilation of binaries without a thorough understanding of linkers.
>>
>> With the recent resolution of dmd's exportation support on Windows (druntime support is still under development), we are now ready to focus on making D's shared library capabilities more accessible and usable for common use cases.
>>
>> Links: [current](https://gist.githubusercontent.com/rikkimax/1d7cfdb8ed74e9e4efc9ba0208f42e7e/11e36e8a332324c5323792da2835337d0306a6c9) [latest](https://gist.github.com/rikkimax/1d7cfdb8ed74e9e4efc9ba0208f42e7e)
> 
> Can a module, which is compiled as part of the application, still contain prototypes for symbols from a DLL? Here is an example:
> ```D
> extern(C++) void foo(const char *s, size_t length);
> void foo(string s)
> {
>      foo(s.ptr, s.length);
> }
> ```
> 
> Function `foo` is implemented in a C++ library, which could be compiled as a DLL. The module contains a prototype for the function, but also an overload with a more convenient interface for D. The module needs to be compiled, because it contains the overload. Can the prototype still be imported from a DLL and be part of the same module?

Yes.

Mangling and ABI is left untouched.

About the only thing you need to deal with is setting a symbol into DllImport mode, which is already something you are meant to be doing.

With this you can set it via ``export extern``, ``export(none)`` or when using an import path use ``-extI`` and have it ``export``.

You can get away with using internal symbol mode because Windows will generate a wrapper function that is in internal mode as part of the import library.

There is no difference in internal vs DllImport symbol modes on non-Windows.
March 29

On Thursday, 29 February 2024 at 11:15:07 UTC, Richard (Rikki) Andrew Cattermole wrote:

>

D's support for shared libraries has historically presented challenges for users, especially when dealing with complex use cases. This proposal aims to improve the user experience by establishing a user story that will provide a consistent and user-friendly approach to usage and compilation of binaries without a thorough understanding of linkers.

With the recent resolution of dmd's exportation support on Windows (druntime support is still under development), we are now ready to focus on making D's shared library capabilities more accessible and usable for common use cases.

Links: current latest

Quick remark:
export ( Identifier )
will have all the problems associated with version identifiers, in particular that you cannot combine version identifiers (A && B) or generate them briefly from boolean expressions. Is there a reason why you do not allow boolean expression here?

-Johan

March 29

On Thursday, 29 February 2024 at 11:15:07 UTC, Richard (Rikki) Andrew Cattermole wrote:

>

D's support for shared libraries has historically presented challenges for users, especially when dealing with complex use cases. This proposal aims to improve the user experience by establishing a user story that will provide a consistent and user-friendly approach to usage and compilation of binaries without a thorough understanding of linkers.

With the recent resolution of dmd's exportation support on Windows (druntime support is still under development), we are now ready to focus on making D's shared library capabilities more accessible and usable for common use cases.

Links: current latest

"The compiler must not inline a function from an out of binary module if it refers to non-exported symbols."

I think this should instead be detected at module compile time, giving a warning that an exported pragma(inline, true) function is calling a hidden function.
Let's keep pragma(inline, true) as a very strong inlining hint.

-Johan

March 30
On 30/03/2024 2:28 AM, Johan wrote:
> On Thursday, 29 February 2024 at 11:15:07 UTC, Richard (Rikki) Andrew Cattermole wrote:
>> D's support for shared libraries has historically presented challenges for users, especially when dealing with complex use cases. This proposal aims to improve the user experience by establishing a user story that will provide a consistent and user-friendly approach to usage and compilation of binaries without a thorough understanding of linkers.
>>
>> With the recent resolution of dmd's exportation support on Windows (druntime support is still under development), we are now ready to focus on making D's shared library capabilities more accessible and usable for common use cases.
>>
>> Links: [current](https://gist.githubusercontent.com/rikkimax/1d7cfdb8ed74e9e4efc9ba0208f42e7e/11e36e8a332324c5323792da2835337d0306a6c9) [latest](https://gist.github.com/rikkimax/1d7cfdb8ed74e9e4efc9ba0208f42e7e)
> 
> Quick remark:
> `export ( Identifier )`
> will have all the problems associated with version identifiers, in particular that you cannot combine version identifiers (A && B) or generate them briefly from boolean expressions. Is there a reason why you do not allow boolean expression here?
> 
> -Johan

At the current point in time, I don't see any reason to introduce that behavior.

The version that would be provided should be coming from the compiler or your build manager.

If you are not limiting yourself to ``InBinary_*``, you're probably going to have a lot of pain.

It is meant for advanced situations where the external import path switch might be giving the wrong results and you need to go on a per symbol basis.

March 30
On 30/03/2024 2:33 AM, Johan wrote:
> On Thursday, 29 February 2024 at 11:15:07 UTC, Richard (Rikki) Andrew Cattermole wrote:
>> D's support for shared libraries has historically presented challenges for users, especially when dealing with complex use cases. This proposal aims to improve the user experience by establishing a user story that will provide a consistent and user-friendly approach to usage and compilation of binaries without a thorough understanding of linkers.
>>
>> With the recent resolution of dmd's exportation support on Windows (druntime support is still under development), we are now ready to focus on making D's shared library capabilities more accessible and usable for common use cases.
>>
>> Links: [current](https://gist.githubusercontent.com/rikkimax/1d7cfdb8ed74e9e4efc9ba0208f42e7e/11e36e8a332324c5323792da2835337d0306a6c9) [latest](https://gist.github.com/rikkimax/1d7cfdb8ed74e9e4efc9ba0208f42e7e)
> 
> "The compiler must not inline a function from an out of binary module if it refers to non-exported symbols."
> 
> I think this should instead be detected at module compile time, giving a warning that an exported pragma(inline, true) function is calling a hidden function.
> Let's keep `pragma(inline, true)` as a very strong inlining hint.
> 
> -Johan

That statement isn't just for pragma inline, it also applies to preventing the backend from performing an inline too.

However in saying that currently I have enough reasons to believe that honoring it will only result in linker errors that to users is just a garbled mess. Even with the warning from the compiler.

I would need evidence that an inlined symbol accessing another binaries symbol that was not exported will work. Right now I have no evidence that this is possible. Therefore this change would allow non-linking code that had a zero percent chance to link, to pass compilation.
April 10

On Thursday, 29 February 2024 at 11:15:07 UTC, Richard (Rikki) Andrew Cattermole wrote:

>

D's support for shared libraries has historically presented challenges for users, especially when dealing with complex use cases. This proposal aims to improve the user experience by establishing a user story that will provide a consistent and user-friendly approach to usage and compilation of binaries without a thorough understanding of linkers.

With the recent resolution of dmd's exportation support on Windows (druntime support is still under development), we are now ready to focus on making D's shared library capabilities more accessible and usable for common use cases.

This proposal is now officially DIP 1045. I've worked with Rikki to revise it and it's with Walter and Atila for assessment. Unless they let me know they need more time, we should expect a decision before April 24th.

https://github.com/dlang/DIPs/blob/master/DIPs/DIP1045.md