OpenD at one week old

Posted 2024-01-08

Over 100 people have expressed interest in the OpenD Programming Language over the last week and we've gotten to work. A quick update follows.

Progress

As always, I have a lot of things to do. Nevertheless, we made some good progress on starting to make OpenD a reality.

Website

The website got a small redesign that went live the following day. We compromised on those who wanted Martian red and corporate blue by meeting in the middle with a perfect purple!

There's still a lot of work to do to finish the site content.

Language

We've already merged 8 pull requests in the language, in addition to some of my own code that was tarrying upstream. We have interpolation, octal literals, an extern(C++) bug fix, a module system consistency fix, a new version identifier, and some experimental work on unittests.

There was a lot of discussion about what to do to improve unittests, making the built in option adequate without the need of third-party libraries. Of course, it might be better to simply bundle one of the existing libraries than to change the language, but in the spirit of being open to contributions, we're gonna try a few different things. The first step is allowing unit tests to be named in the language proper, and that is merged and ready to take next steps.

See the full list here: https://dpldocs.info/opend/changes.html and refresh it every few days to keep abreast of changes.

Compiler

We forked repositories, created new repositories, and deleted repositories over the week. There are currently three on the github: the website, a discussion holder (github required this to enable the "Discussions" tab), and a monster of a monorepo.

The monorepo is an aim to reduce duplication and complication in keeping the key assets running: the quick, hackable compiler (based on dmd), the optimizing compiler (based on ldc), and the associated libraries. Instead of writing the code in one repo then having to separately pull it into and adapt it for the other compiler, we're gonna try just combining it all. Building ldc will use the same source and library as dmd. Phobos will be included so it all just works when you clone it and build it.

Using git subtree merges, we will still be able to pull some changes down from upstream.

Again, still some work to be done, but as of this writing, it actually works on my Linux box. Untested elsewhere.

Community Interest

We set up github discussions, discord chat, and an IRC room. There's a total of about one hundred people who have joined these or email me so far, which is much more than I expected. Some of them have already started actively working on the code. Some said they just wanted to watch the technical train wreck and drama of the dumpster fire. Many just want to keep an eye on where it goes before making a decision to get more or less involved.

A couple days last week had significant discussion on language potential, including some former contributors reviving old ideas they had abandoned after a negative experience upstream. However, these last couple days have returned to a quieter normal. Only time will tell if the fork has a lasting invigoration on contributions or not. As I have been trying to warn everyone, I do have a lot of other things to do and thus can't do too much myself, and there's some legitimate technical challenges that are no easier in a fork than they are upstream, that are more challenging than before since not all the relevant expertise has joined the fork. So we have to be realistic about tempering our expectations. Being free of some constraints is not being free of all constraints.

Nevertheless, it nice to feel a new hope.

Debates

There have been some debates among both proponents and critics of the fork.

Naming

Despite not being completely sold on it myself, I settled for the name "OpenD" because it is close enough to show it is a small variant of the D Programming Language while being different enough to show it has changed direction (another thing I considered was to simply call it "D 3.0", but I feared this might be too confusing). Additionally, the word "open" speaks to the key difference: the openness to contribution.

I understand there are some other preferences out there, but I still think this strikes the right balance to acknowledge the heritage of the foundation we're building upon while minimizing confusion of the future. Remember, this fork was not sent into the world to condemn D, but rather that through the fork, D might be saved.

GC

One of the guiding principles of this fork is to embrace the GC as a successful design rather than to shun and avoid it. Does this mean that non-GC strategies are instead the ones shunned and avoided?

I have a fairly easy-going attitude toward this. My biggest benefit out of embracing the GC is simply that it is easy. It is easier for the end users to be productive and correct with a garbage collector - in most cases; there's some important exceptions to this, especially when interfacing with external environments - than it is with most other schemes.

So, my view is if you can deliver on ease-of-use with your other schemes, that's OK. But there is still a concern that this can fragment the userbase and ecosystem.

I have harshly criticized @nogc in the past as putting a disproportionate burden on library authors while being the wrong answer to what can be a perfectly fair question. This leads to trouble on both sides of the equation: library authors, upset that this extra burden is being placed on them, might fail to engage with legitimate problems a programmer with a particular use case is facing, and users may get the impression that GC is a lesser form of memory management and libraries that use are are necessarily worse than those that do. Additionally, @nogc is so strict it is not amenable to any kind of middle group. Sure, in theory, GC code can call @nogc code, so they're compatible, but in practice the interfaces must be quite different - and the documentation more dense - to account for things, meaning such interop becomes harder to use.

Finally, language and library feature discussions are often stopped in their tracks by concerns about @nogc compatibility, without weighing the significant benefits they may bring. "But it needs GC" becomes a feature-killer, and then we're stuck with nothing.

It is these things that I want to avoid with the "embrace GC" statement. I'm not necessarily against supporting other schemes - again, there are sometimes legitimate reasons why they can be beneficial - I just don't want to hurt the common case in the name of a more niche case.

There are, of course, people who disagree with me and have a stronger position. But I think we're all aligned on the goals of keeping the language easy to use, dispelling myths about the GC (while fixing implementation issues when it isn't just a myth), and not letting good work be blocked by the topic.

Upstream Compatibility

Another point of contention is how much compatibility to keep with upstream. I think it generally behooves us to stay close - remember, this is the OpenD Programming Language. D is in its DNA; we're not moving Boulders.

That said, we must not fall intro the trap of favoring compatibility to such an extent we wither and die. A couple breaking changes have already been merged in: module/import name mismatches (if you don't do this, your code doesn't work outside simple examples anyway, so better to have the compiler guide you in the right direction early rather than late), and mutable static initializer references, which is a FAQ that surprises people with how bizarre it is. Both of these caught some trouble right in the test suite, so I expect code will need adjusting, but it is an easy change, you can do it in a backwards-compatible manner, and it fixes a real problem, so I think these are worth it.

Not all potential changes in OpenD will be compatible with upstream though. The named unit tests, for example, if you use them will likely break you away from upstream, as it is a parser change. Of course, there's ways to mitigate this, including simply not using that feature in code you wish to be compatible with both language variants.

What will get dicey is if the same syntax starts doing two separate things. One discussion we had was on changing float.init to be 0 instead of NaN. I think this is worth it, but we haven't actually made the change yet since a is float.init could be trouble. Code should probably transition this to using some name other than .init. We'll think more about it.

But my guiding principle on breaking changes is to weigh the cost and benefit. The cost is potential user confusion between the two and the burden of maintenance and changes. The benefit is... well, whatever it leads to. If the benefit is small, I tend to err more on the side of conservatism, but each decision will be made on a case-by-case basis on its own merits.

And, of course, if someone feels strongly enough about a change to take the time to implement it, that will carry consideration as well. If a decision is neutral, I'd rather accept than reject; rejecting stuff should only happen if it actually causes unnecessary hardship. Of course, hopefully we'd discuss everything before a great deal of time was sunk in.

Future Direction

Ultimately, where this goes is not up to me. Again, remember, I have other work and responsibilities, so I can't do a great deal of work myself, even if I wanted to. So if you want to see something, let's talk about it to make sure there's no big problems, then if you write it, I'll merge it. Or maybe you can get added right to the github org directly and do it yourself. We still have some structure and governance things to work out on the specifics, but again, I err toward accepting rather than denying.

As for what I have in mind, the website has a roadmap here: https://dpldocs.info/opend/roadmap.html which is basically a collection of rough, half-baked notes out of my mind. But you can PR that too if you like. Nothing is set in stone.

If you wanna join the effort, use the github discussion, join the discord, or the IRC (#opend on libera.chat) to start discussing or coding.

We really are Open to your contribution!