You'd want C-compatible bit fields if/when you're building a hybrid D/C/C++ binary, like gdc and ldc are.

On 8/3/24 3:38, Walter Bright wrote:
> You'd want C-compatible bit fields if/when you're building a hybrid D/C/C++ binary, like gdc and ldc are.

In that case, why not have the best of both worlds and have `extern(D)` (the default, well-defined, perhaps `std.bitmanip`-compatible) and `extern(C)` bit fields, like with classes and C++?

On Thursday, March 7, 2024 7:37:02 PM MST Walter Bright via dip.development wrote:
> The inconsistency of C bitfield layout is far less important than it is made out to be. Other inconsistencies C has: char/short/int/long sizes, float/double/longdouble behaviors, 1s or 2s complement arithmetic, field alignment and endianness. Even the signed-ness of chars is implementation-defined. D matches that behavior of the associated C compiler. Living with that is normal.

Yeah, but with the D types, we've mostly tried to fix that (e.g. making char/short/int/long have fixed sizes), and D defines char, wchar, and dchar all as being unsigned. We have enough compatibility to define aliases for the C types and use those in bindings, but normal D code doesn't have to deal with the issue.

bviously, there are some cases where we're forced to do the same as C even when we might prefer not to, but we have managed to avoid some of C's mistakes in normal D code in a number of cases while still having extern(C) work.

> The gdc and ldc backends are written in C++. D bitfields would enable bitfields to be used in the D front end, and the backends can use them effortlessly (should be no problem for dtoh).
>
> An important feature of D is interoperability with C (and to some extent C++). Reducing the friction as much as we reasonably can will lower the barriers to D adoption. It means D code can co-exist peacefully in the same binary. When gdc and ldc are ported to a different platform, with different C bitfield layouts, it will still *effortlessly* work correctly.
>
> I like effortlessly.

Well, I guess that structs don't technically have extern(C) or extern(D) on
them, which makes this harder (structs and classes do with extern(C++), but
that doesn't help with C). Otherwise, the obvious thing would be to just use
the C behavior in structs / classes that are extern(C) or extern(C++) and
use well-defined, cross-platform behavior with extern(D) types, but it's the
functions that are extern(*), not the types.

However, the next thing that comes to mind would be to use a pragma or
attribute to control it. Then you could mark whether the bitfield member
would be C-compatible or not. If we really wanted to, we could even reuse
extern(C) / extern(C++) (which would then work well with files putting
extern(C): at the top like we often do in druntime), though I don't know how
desirable that is.

Still, if we have a way to tell the compiler whether we want bitfields to be C-compatible or not, then we can have the C behavior where we need it and better behavior for D everywhere else.

And it seems like it would be kind of bad to give the C-compatible version the nice syntax for bitfields while requiring that D code that wants consistent behavior use std.bitmanip's bitfelds instead, since that's going to encourage folks to use the C bitfields when they arguably shouldn't be. We need them for compatibility, but if at all reasonably possible, we should be doing the better thing with normal D code.

- Jonathan M Davis

On 8/3/24 10:10, Jonathan M Davis wrote:
> Well, I guess that structs don't technically have extern(C) or extern(D) on
> them, which makes this harder (structs and classes do with extern(C++), but
> that doesn't help with C). Otherwise, the obvious thing would be to just use
> the C behavior in structs / classes that are extern(C) or extern(C++) and
> use well-defined, cross-platform behavior with extern(D) types, but it's the
> functions that are extern(*), not the types.

Isn't that already a solved problem? What happens currently when an `extern(D)` struct contains an `extern(C++)` class as a member? Or both `extern(D)` and `extern(C++)` classes?

```d
struct S {
    extern(C++) class C {}
    extern(D) class D {}

    C c;
    D d;
}

void main() {
    S s;
}
```

I don't see why we can't have something similar with `extern(D)` and `extern(C)` / `extern(C++)` bit fields.

On 3/8/2024 1:02 AM, Arafel wrote:
> In that case, why not have the best of both worlds and have `extern(D)` (the default, well-defined, perhaps `std.bitmanip`-compatible) and `extern(C)` bit fields, like with classes and C++?

Not a bad idea, but it may be confusing which one is in play, as the extern(C) can be far removed from its effect.

On 09/03/2024 8:28 AM, Walter Bright wrote:
> On 3/8/2024 1:02 AM, Arafel wrote:
>> In that case, why not have the best of both worlds and have `extern(D)` (the default, well-defined, perhaps `std.bitmanip`-compatible) and `extern(C)` bit fields, like with classes and C++?
> 
> Not a bad idea, but it may be confusing which one is in play, as the extern(C) can be far removed from its effect.

Since fields do not currently have block behaviors for externs, doing this would be a new pattern in the language usage.

Otherwise per encapsulation/scope setting of extern may be required. Which is already used, but as it would be required this is new.

Either way as per my original comment it needs to be argued which ever way you go, the cost of implementing our own layout shouldn't be an issue considering how many others will be needed > 1.

I would love to see this replace the library solution.

To clarify a bit, C allows ints to be 18 bits (e.g. for a PDP-10). But D defines them as 32 bits. Technically, we're crazy to do that. But practically, we can rely on the people who implement C compilers to be sane, and CPU makers have long accepted the idea that ints are 32 bits. We are on very safe ground there.

Even chars being signed by default, although popular in the 80s, has disappeared from modern compilers.

We simply do not have to worry about the C bitfield layout being implementation defined. No sane person is going to invent another scheme for a C compiler if at all possible.

There are two dominant schemes today, VC and gcc. I'm not aware of any other scheme for a modern C compiler (Digital Mars C emulates the VC layout). Even so, VC and gcc lay things out the same way for `int` bit fields. This is never going to change.

Just stick with int bit fields and some common sense, and you're code will be fine.

Much of D's design (and many other languages' design) is predicted on C behavior not being implementation defined, even though it technically is.

There's no need to solve a non-existent problem.

(Yes, I know, there is a version of D that has 16 bit ints. But that's a special build of the compiler, and it is technically not a D compiler :-/ and the people who use it understand that and it isn't a big problem for them.)

The original design of D allowed the compiler to lay out struct fields as it saw fit, and extern(C) had the C layout. This turned out to be rather pointless, and I dropped it.

Sometimes, too many options is not helpful. This is one of those situations. The problem with implementation-defined layout is much more of a pedantic one than a pragmatic one, as I've explained elsewhere in this thread.

If one sticks with `int` bit fields, they behave the same across C compilers that I've seen. Just like ints are 32 bits.

We don't have an attribute to switch endianess. It's just not worth the added complexity.

If anyone is worried about exactly specifying the layout, using masking and shifting will guarantee it (absent endianess issues!).

We do not need to solve every detail, just go for the critical ones.

On Saturday, 9 March 2024 at 07:51:08 UTC, Walter Bright wrote:
> Sometimes, too many options is not helpful. This is one of those situations. The problem with implementation-defined layout is much more of a pedantic one than a pragmatic one, as I've explained elsewhere in this thread.
>
> If one sticks with `int` bit fields, they behave the same across C compilers that I've seen. Just like ints are 32 bits.
>
> We don't have an attribute to switch endianess. It's just not worth the added complexity.
>
> If anyone is worried about exactly specifying the layout, using masking and shifting will guarantee it (absent endianess issues!).
>
> We do not need to solve every detail, just go for the critical ones.

Fully agree. There's nothing wrong with implementation-defined.

I guess it might be possible to make an even stronger guarantee, specifying that the current behavior is how it must work on windows and unix in order to be compliant with the D language spec. The same way we decided to define the common practise for integer wrapping and two's complement.