Happy birthday, ImportC!

Posted 2022-05-09

ImportC was released one year ago today. Let's take a look at where it is now and how it compares to the alternatives.

Core D Development Statistics

In the community

Community announcements

See more at the announce forum.

ImportC

The ImportC announcement was made on May 9, 2021. The idea in that first announcement was that it should be pretty easy and doesn't need any thought about the preprocessor. It mentions dstep as al alternative in the changelog, but had zero analysis of it.

I'll argue that dstep did a better job back then, still does a better job than importC, and will continue to do a better job in the foreseeable future despite its potential.

ImportC's potential

First, the potential of importC is:

  • Integration of the build system

    There would be no need for running a separate translation step as part of the build if importC delivered. You might consider doing it anyway though for compile speed purposes, but there's a good chance it wouldn't be necessary.

    Of course, running dstep or gcc separately is not necessarily a big deal anyway (and if we had a capable build system it wouldn't matter anyway.) but when they need extra fixups, it can be tricky.

  • Integration of the system headers

    Similar to the previous point, but with a focus on specific versions and proprietary code concerns. If a system changes frequently, you want to match the specific version on the user computer instead of from the build. Or the files may have copyright restriction on distributing to third parties, so using the ones already there will allow compliance with these terms.

    Again, you might be able to run the step separately in situ for these situations, but the potential of importC is that it consistently just works, even with a plain dmd test.d build.

  • Integration of the languages

    This is what I wrote about before called mixinc - having access to import, inspect, generate, and compile C code from inside D's ctfe.

    This is the part that is most interesting since there is no (full) solution with outside build steps. Best you could so is have helper programs generate code, then compile time, and call it as extern(C), but you can't go all the way into inspecting macros as structured data and applying them selectively to some code embedded inside, or generated by, D code during the build.

    This also would allow full textual macros from C headers to be used without mangling D code, since it is run only selectively on certain areas - it is semantically aware of all three languages (C, D, and the C preprocessor language).

    There's some tension between this - which benefits from more custom implementation - and the above, which benefits from using existing system facilities. But there's significant synergies as well, so the work done so far might still be used to realize this potential.

ImportC at one year old

None of that potential has thus far been realized, though some of it is close. Since it doesn't have any preprocessing facilities, you still must use an outside build step anyway. And once a file has been preprocessed on the outside, you lose vital information, like #define'd constants. dstep attempts to translate these to D, and usually succeeds, making it a more useful outside build step in practice.

There is a work in progress to have dmd shell out to the system preprocessor, which will remove the build step, but still not allow access to defined constants!

It still cannot be used for system headers since there's configuration gaps and reliance upon specific compiler extensions dmd doesn't implement in many of those system headers. Basic code compiles though, and with some macros, you can translate many of those extensions over to more standard C... but if you're translating things anyway, again, you've just reached parity with what dstep has been able to do for years!

It cannot be used for C++, Objective-C, or COM, and has no plans to, meaning you are still likely to want outside steps for those vital pieces in common operating systems. (Or use the Calypso project, which I don't know enough about myself but have heard it works for C++).

The most interesting thing ImportC can already do that dstep can't is to take in a whole .c file and build it, so things like the stb single-file libraries might not need translation to D anymore. Of course, you could just compile it to a object file and generate a header previously, but that's a fairly complicated build for users; I've translated some of those myself to avoid that hassle, so I see some benefit here. ... just translating those files wasn't that hard and now it is done, so the benefit isn't especially large in my eyes. Still, it can be good.

ImportC's near term future

There is some work pending on preprocessor integration, so that will probably come soon. And some buzz internally about my mixinc concept, though it doesn't look like they are actually going that direction right now.

Once they figure out all the flag forwarding and such we might be close to parity with dstep. But I doubt it will realize the potential to exceed it for at least another year, probably more.

And what could have been done to D itself in that time?