See more at the announce forum.
The version of Phobos available in gdc has docs available here: https://docarchives.dlang.io/v2.076.0/phobos/index.html
In fact, you can find several versions of phobos docs on that site.
I've been quite busy this last week on a variety of things...
I got an email reporting a bug in arsd.dom.Element.querySelector matching the this element when it should have only looked at children. I'm a bit surprised I haven't noticed this yet, but it seems to have been there for years on a particular edge case!
I fixed it (release pending additional regression testing, will be soon) then moved on to some other related items and things from my todo list.
At some point, I also want to basically do an audit of the code and formalize the @scriptable list and reorganize the dom.d documentation using new adrdox features. It'll probably take a while though.
It might be nice to finally finish the mutation api support in there too, but that's a daunting task. So many places to edit, so idk if it will actually happen in 2020!
One thing I wish I did in the first place though was returning ranges everywhere. Javascript actually returns live views in many functions too, but mine were all eager for easy implementations in the beginning (except the explicit Lazy ones, there's a few relating to selectors and trees that are in fact lazy ranges), and now I rely on it - looping over to modify them. I actually kinda like it. So maybe I'll never follow the standard on that.
I plan to add insertAdjacentHtml soon and also classList... but classList is one of those live objects... and I'm going to delay a bit until I have the time to actually do that right instead of hacking it in. But shouldn't be a long wait! Of course, since dom.d already has addClass, removeClass, and hasClass, and toggle isn't as useful in dom.d's static context, it is easy to ignore for now anyway.
I added a new module called arsd.archive that has public domain LZMA and LZMA2 code embedded, as well as a homemade tar reader. You can combine these to read tar.xz files, and soon I will also add tar.gz support. Phobos already has std.zip so you can access plain .zip files too.
I sort of want to add .7z support, but I'm not sure how complex it is. I also plan to eventually add another archive file that will work better for my doc generator. ketmar gave me code a loooong time ago I still need to actually use, so it might be that or I might do something custom. We'll see.
But for now the tar.xz support is pretty useful for things like downloading Android runtime binary packages!
My arsd.http2 module doesn't depend on curl, which is nice for self-contained deployments, but it does depend on openssl. I've been meaning to do some changes to this for a while: I want to make it dynamically load openssl's SO to deal with ABI changes more easily if you use it, and on Windows, actually use native APIs instead so it doesn't require anything outside the OS.
I am thinking about using WinHTTP as a backend instead of my custom implementation there. It should be simple enough that I'll probably do it next week.
The Android runtime installer can then be used more easily with plain dmd android-setup.d on any system.
But as you can see with me writing this on Wednesday instead of Monday, I am way behind schedule, so it will wait a little while!
arsd.jni has a Java .class and .jar file reader, and with it, it can create D bindings to the classes within. I'm not 100% happy with this yet, but nevertheless already committed it to the arsd repo and generated bindings to https://github.com/adamdruppe/d_android
The generator is a (currently undocumented) function in arsd.jni that can be driven like this:
1 import arsd.jni; 2 import std.algorithm; 3 import std.file; 4 5 void main(string[] args) { 6 auto f = args.length > 1 ? args[1] : ""; 7 auto dPackagePrefix = args.length > 2 ? args[2] : "translated"; 8 auto outputDirectory = args.length > 3 ? args[3] : "."; 9 10 JavaTranslationConfig jtc; 11 jtc.doImports = true; 12 13 if(f.endsWith(".class")) 14 rawClassBytesToD(cast(ubyte[]) std.file.read(f), dPackagePrefix, outputDirectory, jtc); 15 else if(f.endsWith(".jar")) 16 jarToD(f, dPackagePrefix, outputDirectory, jtc); 17 else 18 assert(0); 19 }
subject to change.
Running it on android.jar spat out about 8,800 files all of which can compile... if you have infinite RAM and CPU time. But that's why there's double the number of files as there are classes in the archive - it actually separates interface and implementation. This also is a change in arsd.jni from last week to allow this (though the old way still works since I think it looks so elegant).
The file you actually import will public import the interface function, then run mixin ImportExportImpl!Class; below. All the interfaces only import other interfaces.
As a result, you can build using the -i switch to dmd/ldc2 and only compile in the implementations you actually use. This takes the build time down from infinity to about five seconds on my computer, and the memory usage down from OutOfMemoryError (apparently my new 32 GB was insufficient!) down to ~700 MB.
It still isn't great! But it is at least usable. I'll write more about this on the android tutorial when I finish it, unless I can think of a better way between now and then. (I also had to skip default ctors because stupid Object.factory in druntime causes all link errors. Same with final class - virtual methods actually work in the guts of arsd.jni now, sort of, though I am iffy on if it is actually useful, but the generator with separate implementations make them all final anyway so the linker won't try to eagerly reference all those implementations to populate the vtable, defeating my attempts at lazy pay-as-you-go.)
Of course, if you want real pay-as-you-go minimalism, nothing beats just writing the few methods you need and letting the CTFE generator do the rest incrementally! You can get very quick and lean builds that way.
But you can also use this generator on an individual Java .class file to generate the interface you fill in later by hand. I expect that will also be useful.
Well, if you've read this whole page, you know a lot of things about this already! Among the other things I changed:
Last week, I had helper programs android-dub-build and android-ldc. I've decided to delete them in favor of a ldc2.conf setup process that user kinke suggested on the newsgroup.
Meanwhile, LDC's CI generates druntime binaries, so my setup program (not 100% finished btw) can detect them and download it, then modify the conf file for you.
After that's done, you can just build with ldc2 specifying the platform via the command line and it does the right thing and pulls the right druntime! You can also use dub's -a option to forward all this.
# 32 bit x86 build for the emulator
dub build --compiler=ldc2 -a i386-none-linux-android
# need to copy the file so Android Studio can see it
mv libdubtest.so BasicActivity/app/src/main/jniLibs/x86I am hoping to get dub to implement an option for me to set the output directory from its command line so that second step is unnecessary too, but for now it is.
But this ldc2.conf based setup should now work pretty easily with other build systems as well, including potentially full integration with Android.mk or gradle builds in the Android IDE (though I do NOT intend to implement that myself. I am personally happy with a separate D lib build then using the ide for the rest, with it seeing D as just a pre-built lib to include).
Let's talk about the d_android dub package a bit though. It includes a few pieces:
And here's the file:
1 { 2 "name": "d_android", 3 "targetType": "sourceLibrary", 4 "description": "Helper programs, instructions, and bindings for using D on Android", 5 "license": "Apache-2.0", 6 "dependencies": { 7 "d_android:ndk_bindings": "*", 8 "d_android:java_bindings": "*" 9 }, 10 "sourceFiles": ["source/hacks.d"], 11 12 "subPackages": [ 13 { 14 "name": "ndk_bindings", 15 "description": "Bindings for Android NDK", 16 "sourcePaths": ["translated_headers"], 17 "importPaths": ["translated_headers"], 18 "dependencies": { 19 "arsd-official:jni": "*" 20 }, 21 "targetType": "library" 22 }, 23 { 24 "name": "java_bindings", 25 "description": "Bindings for Android Java classes", 26 "importPaths": ["java_bindings"], 27 "dflags": ["-i"], 28 "dependencies": { 29 "arsd-official:jni": "*" 30 }, 31 "targetType": "sourceLibrary" 32 } 33 34 ] 35 }
It starts off with a sourceLibrary with one file, hacks.d.... that I forgot to commit last night. How did that ng user actually make this work for them?! maybe it is easier than even I think now :)
Anyway, that file has two lines:
1 // hack around missing TLS 2 void main() {} 3 4 // enable cycles in static ctors for the Java bindings 5 extern(C) __gshared string[] rt_options = ["oncycle=ignore"];
These are why it is a sourceLibrary - so these two thingies are included in the final build. The D compiler requires a main() to be present to output its tls sections, so failing to do this will lead to a runtime link problem. Don't want that, but also don't want to burden the user with this hack of an implementation detail. So it goes in here.
The second line - general tip here, that line is how you set default --DRT- options for your program, I'll probably write more about this again later - disable's druntime's cycle detection algorithm in static constructors.
I'm not happy with this hack, but it appears to be the only real option right now for my generated Java bindings, whose dependency graph looks like a plate of spaghetti.
What I'd prefer to do is to disable this check on individual modules, but alas the only option is to do it app-wide :( It is safe for me - my code just builds an array for future processing, so the order in which they are added is irrelevant. But it may be a problem for other code and this hack would affect it too...
But without this, the static ctors would just... not run. It wouldn't immediately crash the Android program (tbh idk why, must be exception in the C constructor just terminates it instead of a full abort), but without that setup, the jni glue wouldn't be registered, so calling the D programs from Java would fail. I need those to run!
So hack it is. The side effects are undefined behavior soooo technically it is your fault if you break it at least. yeah some consolation i know
"subPackages": [ { "name": "ndk_bindings",
Nothing fancy here, simple precompiled ndk binding library. I put it as a subpackage since it might be nice independently too, but it is also a dependency of the main package so it is available by default. Really no reason not to given how trivial it is.
General dub tip though: using subpackage libraries as dependencies, even if you don't really care for it from the outside, you can break up your build a little into sub-steps for better caching and perhaps reduced dmd memory usage. I am generally an advocate of dmd *.d, but there's times when you don't want that. Easy with makefiles, a little more complex with dub, but still doable with subpackages, especially if you break up interface and implementation in the D itself (with various techniques, traditional (interface vs class, PIMPL, etc) and non-traditional (like what I do with the java generator described above).
Which brings me to the end:
1 { 2 "name": "java_bindings", 3 "description": "Bindings for Android Java classes", 4 "importPaths": ["java_bindings"], 5 "dflags": ["-i"], 6 "dependencies": { 7 "arsd-official:jni": "*" 8 }, 9 "targetType": "sourceLibrary" 10 }
This is the craziest part of this file. It spills over into your program too a little thanks to the sourceLibrary stuff (and the top-level package being one too means this carries though)... but it is still cool.
Note the presence of importPaths and absence of sourcePaths - quite intentional. The idea here is the big Java class library is available if you want it... but not included if you don't.
The -i dflag tells the compiler that if you do import it, include it (and its recursive imports) in the formal build... but if you don't import it, none of this is eagerly added. Exactly what I want. And since it is a sourceLibrary, this is not attempted separately, but instead as part of your end-user build, so it is all there.
There's some quirks to this - if you import a Java class that returns another Java class, you must also import its file or you'll get linker errors (which on Android, come when the program is loaded, not in your build! So be sure to test in the simulator's debugger.). This is because the interface is all included, but the implementations are hidden behind those imports. Weird, but it keeps the build reasonable while still having complete bindings on demand.
I'll probably change this more as time goes on, but as of yesterday when I write this, this is my new state-of-the-art. If you know of better techniques, please tell me!
But the end result of all this is the cute little test/demo program in the d_android repo:
1 module test; 2 3 import arsd.jni; 4 5 // it is possible to import Java methods from Android APIs 6 import android.java.android.widget.TextView; 7 8 // snip 9 10 // as well as export your own native methods for JNI to call pretty easily: 11 // this shows the short-form implementation 12 final class MainActivity : JavaClass!("com.example.basicactivity", MainActivity) { 13 14 // the object will be passed in from Java to change the color 15 // via the Java method. Triggered on click in the sample program. 16 @Export static void DoSomethingInD(TextView tv) { 17 tv.setTextColor(0xffff00ff); 18 } 19 20 // can also return values from D to Java 21 @Export wstring saySomething() { 22 return "Hello from D!"w; 23 } 24 }
Works, proving the potential for even more. And that's pretty cool.
I might take on D/iOS next - I already know a good amount of what's needed there, but there's some real work needed on ldc to make it all work. Probably will be months until I get it all ready too even if I do take it on, so don't get too excited yet.
Lots of open source stuff this week by me!