This Week in D

dmd 2.086 beta, dstep 1.0 released, Adam works on memory usage

Blog
Posted 2019-04-22

A new dmd beta came out this week, along with DStep officially reaching version 1.0! Meanwhile, I have been fighting growing memory usage in my programs.

Core D Development Statistics

In the community

Community announcements

See more at the announce forum.

My thoughts about community announcements

dmd

The new dmd and dstep releases look quite interesting. Three features in particular got my attention:

  • The new copy construction stuff is interesting, though I'm slightly apprehensive, since I have found decent success with the existing postblit. I am guessing that if they deprecate postblit, we can reimplement it with a simple tupleof assignment in the copy constructor easily enough.
  • The -lowmem switch indicates they might actually be working on fixing up dmd to actually work with the garbage collector, a proper step toward undoing dmd's mistake of never freeing memory which makes it utterly unusable on some systems. Hopefully, they will continue with this and make it work well enough to use all the time.
  • __traits(getMember) now bypasses protection, allowing __traits(getProtection) to work on private things, simplifying reflection code more. I would defend the old way, it isn't as bad as some people say, but I think the new way is a step forward. And my old getProtection implementation is no longer flawed in design! :)

I have been kinda excited over several releases lately, and this one is looking good too. I personally feel D is moving in the right direction again after a lull, though I am still disappointed at the lack of attention to error messages...

dstep

The blog post about dstep makes it look fairly complete - by the looks of it, it has very good support for C now, hence its new 1.0 label. I have never personally used it... but I might soon, it looks ok.

It just also needs to convert some Objective-C base files and stick them in druntime now ;)

What Adam is working on

My RAM wasting

Over in my world, the week started with the realization that my computer was out of memory! After doing some hunting, I traced a plurality of the waste to my own programs: unlimited, inefficient scrollback in my terminal emulator, multiplied by about 80 instances, was wasting a full gigabyte, and the dpldocs.info search program was wasting two more.

Both were because of poor design in my application code. The terminal emulator used a whopping 24 bytes per character of output, and had few limits on how far it would go. I added limits (and a circular buffer to make it avoid reallocations once it saturates), and determined the biggest inefficiency of its storage was my 24 bit color support. Since I have never used that feature and don't seriously care about it anyway (I think it is silly: in a terminal, you are limited to two colors per terminal cell, so it isn't like you get high fidelity anyway. And then 256 colors is enough for almost any terminal task after that.), so I removed it. ... and personally, I rarely use more than 8 colors in terminals anyway.

Removing that detailed storage slashed the memory usage in half instantly. Now, I may have been able to store it more efficiently without slashing the feature, but why bother complicating the code for a useless feature?

It now caps at about 30 MB per instance, and the average is actually 10. It is still a bit fat, but much better than the 120 MB a few of them peaked to before! Yikes.

Similarly, the dpldocs.info search system kept all its database in memory, and it has now grown to 800 MB.. multiplied by two or three processes at peak times. Yikes. I just moved the database to PostgreSQL and now the server processes eat 60 MB. Again, still bloated, but a HUGE improvement.

It is kinda embarrassing to admit my poor programming, but eh, that's what happened.

simpledisplay changes

I also decided to finally use signalfd on Linux in simpledisplay's event loop to capture SIGINT and SIGHUP. This allows it to more cleanly exit upon receiving them, and opens up the possibility of user-defined handlers.

The downside is ctrl+c might not be as effective from the user's perspective. You shouldn't be doing much processing/waiting in your gui thread anyway... but simpledisplay doesn't really encourage or enforce this. I might change that somehow someday, but for now, remember you can ctrl+\ to forcibly kill a process too. And as a programmer, don't block the event loop longer than you have to!

I also used simpledisplay to make a little X11 clipboard helper. The goal: given a list of strings, let me paste them sequentially into another program. (I had to copy a list to a particularly clunky web ui, but I could ctrl+v in it.) The beauty of writing my own library with the underlying details was that this was a trivial little addition to the library; I had the hack program working in under 15 minutes.

1 import arsd.simpledisplay;
2 
3 import std.string;
4 import std.stdio;
5 
6 void main() {
7 	auto window = new SimpleWindow();
8 	window.hide();
9 
10 	string[] list;
11 	foreach(line; stdin.byLineCopy)
12 		list ~= line.strip;
13 	
14 	void next() {
15 		if(list.length == 0) {
16 			window.close();
17 			return;
18 		}
19 		setX11Selection!"CLIPBOARD"(window, list[0], &next);
20 		list = list[1 .. $];
21 	}
22 
23 	next();
24 	window.eventLoop(0);
25 }

I wrote that as a one-off thing, so it isn't amazing ui, but hey it worked for me, and maybe it will be usefull to you all too.

Web stuff

I am also progressing on a little web job in D, which gives me an excuse to use my new cgi.d stuff... but in practice, I still don't like it yet. But that's why I use these things for real projects: it tells me what is actually valuable and what doesn't work yet. I still have a lot of work to do! I did use the new WebPresenter and RestObject stuff, but I am so far preferring to just hand-write the code soooooo the code generator needs more work.

I also played with some new CSS and HTML features though, and I like it. The <main> tag is nice. CSS flexbox, grid, display: contents - all kinds of things to make writing clean html better. I have used some of these before, but I haven't had a chance to do a from-scratch project for a while, and some of it is actually new to me.

I already thought most websites were bloated trash, but seeing what the new browser features keep adding, it shows how REALLY BADLY bloated trash most websites are.

Blog Articles
Page generated by adrdox