Solving vs managing problems

Posted 2019-07-15

Not much time to work on D this week. Got some open source contributions, but mostly the free time was spent elsewhere. This next weekend, I'm visiting family so don't expect much then either.

But that said, I'll write a bit on solving vs managing problems and metaprogramming today.

Core D Development Statistics

In the community

Community announcements

See more at the announce forum.

Adam's thoughts - solving vs managing problems

I like to draw a distinction between solving problems and managing problems. When a problem is solved, you don't have to worry about it anymore. By contrast, when it is managed, you can move on... but the problem will require ongoing work to manage.

For example, in most dynamic languages, you manage the problem of variable name typos crashing a production system. With unit tests and the sort you can prevent this problem from stopping you, but these solutions must be consistently used going forward; tests need to be continually updated and run, or the problem creeps back in.

With a static language like D, however, the problem is solved (well, except in templates, but even there, it is impossible to ship a broken template to a production system) - thanks to the compiler being an intrinsic part of the system and it including these checks across the entire program, you don't have to worry about it getting to production.

I confess there's a bit of a grey line here - is running the compiler really more of a solution than a CI test and deployment system? Well, it can be a judgement call, but I'd say anything where you have to specifically think about the problem in the future tends toward managed, and if not, tend toward solved. This might change based on your perspective too - a vendor managing a problem might solve it from the perspective of their client, for example.

I'd argue that a programming language ought to aim to solve as many problems as it can, and provide mechanisms to manage the rest. D does a generally good job here, and this is where I feel D's metaprogramming is unique.

D's metaprogramming and solving problems

Lots of languages have some form of reflection and method generation, and the associated syntax sugar to make it work fairly well; you can do a lot of cool things with it.

D has all this... plus integration into the language's mandatory static checks. With the type system plus user-defined constraints and static asserts, you can solve many classes of problems of incorrect code getting live. If you want to add methods that fit particular names, you can do that, easily, with opDispatch. If you want only a particular combination of types, you can check that and abort the build if it isn't good enough.

Then, of course, you still have various tools to manage problems it cannot solve like unit tests, asserts, contracts, and, of course, other startup or runtime exceptions, but those are relatively common. (Though D's startup tests are a step above many languages, I'll write about that another time.)

All this can lead to a new problem though: potential compile time growth. And alas, all we can do is manage that one, but it can be managed, so not the end of the world.

But overall, D is strong at solving problems other languages only manage, and that is part of why I have found and continue to find so much success using it for so many years.

A minor, tangential wish

I sometimes wish we could transform existing types though. Like @transform struct Foo {} would actually generate a different Foo than the one we see in the code.

Of course, mixin templates can kinda do that... but just kinda. They still have to work with the names already there.

What I will sometimes do is something like:

@transform struct Foo_ {}
mixin TransformedStructs;

which removes the tailing underscore. But I don't love it.

So I kinda wish we had some syntax sugar there, even if it is just more easily passing anonymous structs. You can sorta fake it with string mixins however, but meh. And the documentation of these things can be non-obvious. Just another problem to manage :)