Improving today's error handling

Posted 2021-08-23

Last week, I wrote some speculative thoughts on error handling ideas and hinted that I had ideas improve today's things as well. Today, I'll go into some ideas for better exceptions and better error codes in D.

Core D Development Statistics

In the community

Community announcements

See more at the announce forum.

How to improve today's exceptions

Last week, I speculated a bit on some bigger error handling concepts and hinted that there's some things I'd like to improve within the strict confines of today's exception system.

The main idea here is that I consider strings harmful. The fact that the easiest thing to do in D is throw new Exception("some string"); is problematic since that's about the worst kind of exception you can have: it is generically typed and has no structured data; just the string intended for user consumption... and even then, it is more likely assuming the user is a developer since it isn't a terribly friendly interface, has no translation mechanisms, comes with a file and line number from source, printed stack trace by default, etc. This makes it difficult to use for any task other than cancelling the task. It also discourages even adding information that the developer-user might find useful like attached error codes. You need to convert it to string and concatenate it... which is a pain and thus very frequently left out. And of course, this is all hostile to nogc. While I'm quite critical of nogc, the encouraged exception model is needlessly antagonistic to it.

There are plenty of ways to improve this using D's beloved metaprogramming. Generally, I'd say we should focus on using a categorization base which has the string pattern attached statically, then dynamic data is added by a function. Consider the following:

import std.better.exception;

class FileNotFoundException : Exception {
	mixin BetterException!(string);
}

void openFile(string filename) {
	throw exception!FileNotFoundException(filename);
}

The exception function is actually a factory function that could be replaced, but it is responsible for both allocating and creating the actual exception class. The template argument there is the parent class, then it will automatically subclass, if needed, to attach the given details. It's implementation looks something like this:

T exception(Parent, Args...)(Args args, string file = __FILE__, size_t line = __LINE__) {
	class E : Parent {
		Args args;
		this(string msg, string file, size_t line, Throwable next = null) {
			super(msg, file, line, next);
		}

		override void dispose() { theAllocator.free(this); }

		mixin BetterException; // toString and friends from here
	}
	auto e = theAllocator.make!E(Parent.msg, file, line);
	e.args = args;
	return e;
}

It might even take a string as a template argument to create the class, to still provide a distinct type but without needing a separately declared parent, since people are lazy and are still liable to want to put strings in there.

When you catch, you can use typeof:

try {
	throw exception!"my message"(4);
} catch(typeof(exception!"my message"(int.init)) e) {
	// caught
	e.dispose(); // if it was allocated, dispose of it here.
}

Or to alias the type in the top level scope just like any other class, or of course, just catch it through the base class if you don't need the extended information as anything other than a string. This is fairly common use in languages like Javascript too.

A bit more about the allocators: here I implied it might be malloc or something, through the formal allocator system, but realistically I'd probably make it do either a version switch to do that for certain builds, but generally I think it is best to keep it GC managed... but do some tricks to make it nogc compatible. Something like this:

Exception exception(T...)(T args, string file = __FILE__, size_t line = __LINE__) @nogc {
        scope Exception delegate() @nogc lie;
        lie = cast(typeof(lie)) () {
                import core.memory;
                GC.disable();
                scope(exit) GC.enable();

                return new Exception("", file, line);
        };

        return lie();
}

void main() @nogc {
        throw exception();
}

Yes, it just plain lies about being @nogc, but I don't feel guilty about this at all. The GC.disable/enable pair in there means you won't have the collection you fear, but you still get the safety net of the GC collection the exception - if you do ever actually run a collection.... and if you don't, you can probably afford to leak the occasional rare exception or to remember to free it yourself in the catch block - which means broad compatibility with existing code.

If I had a little more time, I might explore adding a reference count to it. If reference count increases to 2, it can do a GC.addRoot so it is manually managed. If reference count decreases to 1, it removes the root, returning to automatically managed; the first reference would be the owning GC rather than any particular instance. So reference going to zero would either be ignored, allowing the GC to clean it up... or I guess it could go ahead and delete this if you wanted to live dangerously and trust the programmer not to screw it up. But I actually do think this covers 99.9% of practical cases, with the GC.disable honoring the spirit of @nogc, even if we lie to the compiler about its specific restrictions. And again, unlike dip1008, this doesn't introduce trouble with existing code that does hold on to the reference since it falls back to normal collections.

Then since the data is passed in as arguments instead of encouraging string concatenation at the call site, those can be captured without allocation too, or at least allocated by the same scheme as the exception object itself (possibly embedded right inside it) and still follow these rules.

In conclusion, by using D's metaprogramming, we can do some small library tweaks to the existing system to make more useful patterns easier to reach for and take care of a few allocation details at the same time. I've experimented with this a little in a module called arsd.exception, but I didn't love that first draft and have never used it outside a demo (though part of that is because I don't want to do an import for just this, given my build chain compatibility guarantees). I think this second draft explained here is generally better, but of course, I haven't actually used it yet either.

improved error codes

There's three things I think we'd want for improved error codes:

1) nodiscard attribute. 2) Implicit construction of return values 3) Some kind of non-local return; a function that can return from its caller, not just from itself.

All three of these are language changes, so unlike the previous section, you can't just write some library functions to realize these improvements. However, #1 is already a dip. #2 is pretty simple: if return x; doesn't compile, automatically wrap it in return typeof(return)(x);. This means you can return sumtypes from a single member. I've talked about that before.

#3 though needs a little more discussion. In some programming languages, there are some automatic return things that checks for errors and propagates the sumtype up:

Sum!(Result, Error) func() { ... }

Sum!(OtherResult, Error) caller() {
	Result r = func().check;
	return OtherResult(r);
}

That check function is where the magic is. It expands to something along these lines:

magic check() {
	auto value = func();
	if(value.isError)
		return_from_caller(value.error);
	return value.value;
}

I do think it is somewhat important for it to be user-defined, so it doesn't marry the compiler to one particular type. But that return_from_caller construct is extraordinary. The closest thing D has to it is throw and... sort of opApply.

In the middle of an expression, a function call can inject a return from the function. The injected return follows the same rules as any other return statement. It would be something similar to a mixin, but there is no return expression today, so impossible to do even with mixin right now.

An objection here might be that there's no indication at the call site that it might return, since it is all user-defined functions. Perhaps some new syntax could be added there, like the return keyword as an expression:

Result r = return func().check;

But I don't know. What I'd propose that does is any return_from_caller thing anywhere in the expression is permitted there to return from that statement or otherwise you get the normal thing.

In Rust, they use a suffix ? to indicate this, but it also requires the use of specifically Result or Option types, known to the compiler. Perhaps it could be a new unary operator that can be defined on the returned aggregate. It needs to be a little big magic but I don't care too much what it looks like, I just want to make sure it is easy to use and at least somewhat expandable.

See, some degree of expandability is important because each of these features to assist improved error codes have multiple other applications as well. In my web framework, I use a sum type for redirections or templates and the implicit construction can make that easier to use. nodiscard can be used for a scheduling api that returns a helper object. And this non-local return thing might even help the error-checking delegate I mused about last week. I want these features for various reasons, the applicability to error checking is just one use.