This Week in D

a little more webassembly

Blog
Posted 2019-05-27

I was just goofing around with webassembly a bit more this weekend.

Core D Development Statistics

In the community

Community announcements

See more at the announce forum.

What Adam is working on

Some of you might remember that back in 2011, I hacked up dmd to make a D to Javascript converter. However, I found it pretty useless in practice because it outputted fairly bloated code and was awkward to use.

With webassembly getting bigger and ldc supporting it, I have played around a little. The bloat isn't too bad, though you are currently stuck using a subset of D that interacts awkwardly with the browser environment.

I was thinking about looking at this once Mozilla figured out the awkward interop problem, but it doesn't seem to be a priority to them, so I started goofing around to see what we can do with the current interface. And you know, it isn't the worst thing ever in its early stages:

1 export extern(C) void magic() {
2         document.insertAdjacentHTML("beforeend", "cool!<span id=\"beans\">beans</span>");
3 
4         auto e = document.querySelector("span");
5         e.insertAdjacentHTML("beforeend", " man");
6 }

Look, I'm calling DOM functions! Through the magic of D's variadic opDispatch and a little index operator and .apply code on the Javascript side, I'm able to make those calls with fairly little support code. First, the D:

1 
2 //ldc2 -mtriple=wasm32-unknown-unknown-wasm -betterC -L--no-entry -L-allow-undefined test.d -fvisibility=hidden -L--export-dynamic -release
3 
4 extern(C) void retain(int);
5 extern(C) void release(int);
6 
7 extern(C) int internString(); // returns handle
8 extern(C) void appendToString(int handle, int ch);
9 
10 extern(C) int doNativeCall(int objectHandle, int nameHandle, int startArgs, int endArgs);
11 
12 struct NativeObject {
13 	int handle;
14 
15 	NativeObject opDispatch(string name, T...)(T args) {
16 		int[args.length] handles;
17 		foreach(idx, arg; args) {
18 			handles[idx] = internString();
19 			foreach(ch; arg)
20 				appendToString(handles[idx], ch);
21 		}
22 		int nameHandle = internString();
23 		foreach(ch; name)
24 			appendToString(nameHandle, ch);
25 		return NativeObject(doNativeCall(handle, nameHandle, handles[0], handles[$ - 1]));
26 	}
27 
28 	this(this) {
29 		retain(handle);
30 	}
31 	~this() {
32 		release(handle);
33 	}
34 }
35 
36 extern(C) int getDocument();
37 
38 @property NativeObject document() { return NativeObject(getDocument); }
39 
40 // and this is what I showed above
41 export extern(C) void magic() {
42 	document.insertAdjacentHTML("beforeend", "cool!<span id=\"beans\">beans</span>");
43 
44 	auto e = document.querySelector("span");
45 	e.insertAdjacentHTML("beforeend", " man");
46 }

The test.wasm file is 3.0 kilobytes out of ldc. Not too bad, at least there isn't a megabyte of library you have to download just to get started.

Now, on the HTML/JS side

1 <html>
2   <head>
3   	<meta charset="utf-8" />
4   </head>
5   <body>
6     Test page
7 
8     <div id="doc"></div>
9 
10     <script src="test.js"></script>
11   </body>
12 </html>
1 	// stores { object: o, refcount: n }
2 	var bridgeObjects = [{}]; // the first one is a null object; ignored
3 
4 	const request = new XMLHttpRequest();
5 	request.open('GET', 'test.wasm');
6 	request.responseType = 'arraybuffer';
7 	request.onload = () => {
8 	var importObject = {
9 		env: {
10 			test: function(arg) {
11 				console.log(arg);
12 			},
13 
14 			retain: function(handle) {
15 				bridgeObjects[handle].refcount++;
16 			},
17 
18 			internString: function() {
19 				bridgeObjects.push({
20 					"refcount" : 1,
21 					"obj" : ""
22 				});
23 				return bridgeObjects.length - 1;
24 			},
25 
26 			appendToString: function(handle, ch) {
27 				bridgeObjects[handle].obj += String.fromCharCode(ch);
28 			},
29 			// tbh I'm not sure why it referenced these but it did
30 			__assert: function(a, b) {
31 				console.log("assert: ", a, " + ", b );
32 			},
33 			_Unwind_Resume: function() {},
34 
35 			// this calls the given function on the DOM object.
36 
37 			doNativeCall: function(handle, name, start, end) {
38 				console.log(handle + "["+ name + "]" + "(" + start + "..."+end+")");
39 				var r = bridgeObjects[handle].obj[bridgeObjects[name].obj].apply(
40 					bridgeObjects[handle].obj,
41 					bridgeObjects.slice(start, end + 1).map(function(a) {
42 						return a.obj;
43 					})
44 				);
45 
46 				bridgeObjects.push({
47 					"refcount" : 1,
48 					"obj" : r
49 				});
50 
51 				return bridgeObjects.length - 1;
52 			},
53 
54 			getDocument: function() {
55 				var e = document.getElementById("doc");
56 				bridgeObjects.push({
57 					"refcount" : 1,
58 					"obj" : e
59 				});
60 				return bridgeObjects.length - 1;
61 			},
62 
63 			release: function(handle) {
64 				bridgeObjects[handle].refcount--;
65 				if(bridgeObjects[handle].refcount <= 0)
66 					bridgeObjects[handle] = null;
67 			},
68 		}
69 	};
70         WebAssembly.instantiate(request.response, importObject).then(result => {
71           const { exports } = result.instance;
72 	  // call the D function
73 	  exports.magic();
74         });
75       };
76       request.send();

Just about 2 KB of glue code here.

Now, as you can see, this code is terrible. It copies in strings byte-by-byte via a callback every time they are used. I figure the next step would be to make them ArrayViews into the webasm module's memory.

But, while it is terrible, it also works, at least for the very few functions I have tested. The concepts are:

  • Use a list of native objects on the JS side, reference counted from the D side. If D has any reference to it, keep it pinned in that JS list. Otherwise, we can null it out and let the javascript garbage collector handle it from there.
  • Generate code as it is used on the D side (opDispatch) and access all the javascript strings through integer handles, just like the objects, so we can refer to them across the divide.

Actually pretty simple, and I think that if I spend a little bit more time on it, I can make the code less awful on both sides. More efficient on the JS side, and more statically typed on the D side. (I would love to ultimately just access them through interfaces on the D side, and generate the same code to bridge them as opDispatch does. Should be fairly simple, just might need to work with TypeInfo to get all the way there.)

I'm not actually in a big push to work on this, I just wanted to give it a try and see what I could do in an hour. So no promises I will finish or even revisit this stuff, but we'll see where the winds take me.

Blog Articles
Page generated by adrdox