Demo on custom sections

Posted 2023-02-06

Object.factory might be on its way out. What are some alternatives for its users?

Core D Development Statistics

In the community

Community announcements

See more at the announce forum.

Object.factory alternatives

Generally speaking, you want a way to register factories without triggering druntime's picky cycle detection with module constructors. A mixed in module constructor is, in many times, exactly the solution you want, but it has a cycle detection algorithm that you either have to live with or turn off globally.

to turn it off globally, add this line of code to your file with main:

extern(C) __gshared string[] rt_options = ["oncycle=ignore"];

Or pass --DRT-oncycle=ignore to your program when you run it.

If you're doing a library though, you probably don't want to turn it off globally as that can break things. You'd like a more local solution.

Alas, D's current module ctors don't offer a way to locally say "this is fine, trust me". We could implement that, by adding a new array to the module info constructor list, separating on if they are marked cycle safe or not. (Currently, the compiler combines all static ctors for a module into a single generated function that calls them all. This generated function is then referenced from ModuleInfo so it can be called by druntime upon initialization. We'd want two generated functions, one that gets the current behavior, and one that runs independently of any ordering.)

I think we should add this to the language.

But what can we do today?

Well, using ldc or gdc, you can put function references in a custom linker section... if you look under the hood at how ModuleInfo is implemented, that's how it works too. (On some platforms, there's a .minfo section in the executable it looks for, on others, is emits a crt ctor that calls a particular D function to do the registration. But how are crt ctors implemented? By linker sections with pointers!)

Indeed, using a crt ctor is a possibility. Emit one of these and it builds a list which you then call later from a regular D module constructor. You have to be careful with what you call from a crt ctor though - druntime is right out, and even other parts of the C runtime might not be fully initialized yet and shared library load locks may be in place. There's a priority number, but D doesn't let you set that. And even if it did, it doesn't solve everything. I believe the current impl does let you get away with calling realloc though, so that's how you could build a list fairly safely. Just don't do much else from it and I think you're ok.

But I wanna dig into the linker sections anyway. Let's do a code dump:

1 module sections.demo;
2 
3 import core.internal.elf.io;
4 
5 import sections.demo2;
6 
7 // you register your handler with this
8 mixin template Register(alias fn) {
9 	import ldc.attributes;
10 	@section(".mytest") __gshared void function() registered = &fn;
11 }
12 
13 // we must register a dummy one to trick the linker so we'll keep one here in this file
14 void specialCtor() {
15 	import std.stdio;
16 	writeln("hello special ctor");
17 }
18 mixin Register!specialCtor;
19 
20 
21 
22 // impl to fetch it
23 alias SectionDataHandler = void function(scope const(ubyte)[] data);
24 
25 int findSectionInfo(SectionDataHandler handler) {
26 	import core.internal.elf.dl;
27 	SharedObject exe = SharedObject.thisExecutable();
28 
29 	ElfFile file;
30 	if(!ElfFile.open(exe.name.ptr, file))
31 		return 1;
32 
33 	foreach(index, name, sectionHeader; file.namedSections) {
34 		if(name == ".mytest") {
35 			if(auto offset = sectionHeader.shdr.sh_addr) {
36 				auto beg = exe.baseAddress + offset;
37 				auto size = sectionHeader.shdr.sh_size;
38 
39 				auto data = beg[0 .. size];
40 
41 				// stupid useless nogc limiting me
42 				// if only we could mark a thing as nogc iff the body is nogc...
43 				alias BypassAttribute = void function(scope const(ubyte)[]) @nogc nothrow;
44 
45 				(cast(BypassAttribute) handler)(cast(ubyte[]) data);
46 			} else {
47 				// not mapped
48 			}
49 		}
50 	}
51 
52 	return 0;
53 }
54 
55 void main() {
56 	import std.stdio;
57 
58 	// and the thing to call the functions from the list
59 	static void handler(scope const(ubyte)[] data) {
60 		// we must reference one of the variables
61 		// in the magic section or else the default
62 		// linker --gc-sections arg will strip it out!
63 		auto reference = registered;
64 		reference();
65 		auto fns = cast(void function()[]) data;
66 		foreach(fn; fns) {
67 			// don't double call our root reference
68 			if(fn is reference)
69 				continue;
70 			fn();
71 		}
72 	}
73 
74 	writeln(findSectionInfo(&handler));
75 }

And a second file registering something too:

module sections.demo2;

import sections.demo;

void otherCtor() {
	import std.stdio;
	writeln("here too");
}

mixin Register!otherCtor;

You'll notice I used ldc.attributes, so this demo is LDC only, but gdc.attributes works identically for this case, so you can easily support both of those compilers. dmd is no help here right now though.

Compile and run it (Only tested on Linux! Should work from BSD too, but Mac and Windows and other things would need further adjustments):

$ ldc2 -i=core.internal.elf sections.d sections2.d  && ./sections
hello special ctor
here too
0

Notice that I used -i=core.internal.elf to bring in that ElfFile type. Otherwise, it would not be included in the build (the compiler assumes that, since it is part of druntime, it'd be in the precompiled library, but these specific things may not actually be there. adding it to the build makes it work easier).

Otherwise, just a basic compile and run and we got our special things run.

It is easy to imagine building up a little bit on top of this, like if you registered a variable it could auto-generate a function to append it to an array. Or you could make the section itself be of a particular type instead of a list of functions and just use it directly without running anything; a linker section is really just some static data concatenated ahead of time, so you can slice and cast to the other type and use it without further ado.

You could even do things like register your http routes this way, and a dedicated inspector knowing which section to look for in the executable could print them out without even running the program!

Lots of cool things you can do with static data, and the executable sections are your window into that world.