"We are at the point where they need a check and balance to keep everybody going smoothly. And I do hope that they listen to us before somebody decides its forkin' time."rikki cattermole, August 23, 2018, D forum
Contributing development work upstream to D leadership has always been a very frustrating process. While the oft-repeated claim that D is a closed-source language is not really true (the D-specific parts of the compiler were GPL'd as early as 2002, leading to an all-GPL compiler (what we now know as gdc) being released in 2004), it is true that D's development methodologies are not especially open and that decision making has very little meaningful input from the community, and this has been true for its entire history.
"PR's languish forever ... Nim is magnitudes of orders easier to contribute to."Timothee Cour, March 26, 2018, https://github.com/timotheecour/D_vs_nim
Make no mistake: D is only what it is today thanks to input from the community. Key features and concepts we think of that make D, D are from community input. The D code in existence is mostly community written. There has been significant work done. I've successfully done much of it myself. Indeed, just last week, one issue I have been pushing lately was merged (after a 4 month delay, but better late than never), and I got several small things merged throughout the year.
But looking only at the successes risks us running afoul of survivorship bias. Walter told the story once (it is a well known story but he's the first one I heard it from) of the World War 2 aircraft coming back with bullet holes. You might first look at that, and say "this is where the planes tend to be shot, let's armor that", but the actual result was the opposite. These were the planes that made it back, meaning the ones shot elsewhere probably went down. Thus it was the areas without bullet holes that needed the attention.
It is important to look at all the data.
"The history of D development is full of cases when it took huge amount of time to adopt something that would be considered a standard practice by that time (like version control, issue tracking or release maintenance) because someone had to "sell" it to Walter in a convincing manner."Dicebot, September 4, 2017, essay "On Leaving D""My stance is that as long as current leadership remains in charge and keep sames attitude, no amount of money or developer time will fix D."
Dicebot, August 23, 2018, D forum thread commenting on his "On Leaving D" essay
While there have been many patches accepted in D's history, there are also many patches that have been left to rot. The code we know know as "druntime" originated as a fork because Walter failed to accept community contributions. In 2004, after attempting to work with upstream, these developers were left with no option but to fork the language to keep their contributions - which they must have felt were necessary to expand the use of D - from being totally lost. This fork was called "Ares" at first, and would later join forces with other community efforts to become "Tango".
Tango called itself "The Developer's Library for D". I didn't understand why at the time, I just knew to use it, you had to install some things from an additional website and I didn't want to do that. But now, knowing what I know now about D, the name was obvious: Tango was where the developers actually were welcome to contribute. And contribute they did: the Tango ecosystem had many things the Phobos ecosystem lacked.
"Walter blesses many ideas. What I'm wondering is how quickly he incorporates the results."jcc7, September 10, 2004, dsource.org forums
Only after four years of persisting in the fork and gaining significant popularity, including capturing D's early commercial users, did upstream finally relent and opened up to a reconciliation, leading to the "druntime" we enjoy today.
I believe that if not for the Ares/Tango fork, D would be merely a footnote today; an experiment from some notable people that ultimately went nowhere in the marketplace. We are often told that this situation nearly killed D, but the truth is probably the opposite: this fork saved D at a critical juncture of its development.
The years that followed, with Tango's key developers brought back into the fold to join the likes of Andrei Alexandrescu, et. al., ushered in a golden age for D. Development was, for once, reasonably open and activity spiked around 2013, with multiple commercial users giving it a chance (note that almost all of D's commercial users have depended directly upon code from the Ares fork, the same code that would have been discarded if Walter's hand were not forced).
However, this would not last. The old habits refused to die and the influx of contributors from the golden age would not stick around, with many explicitly citing the broken process that led to their work going unappreciated, or just fade away as their life circumstances changed, one at a time, with few new people sticking around to take their place, until very few were left.
"As early as spring 2009, I was dead tired of circumstances in the D community, its governance and so on, and although I loved the language (D v1 that is), I was uncertain of the future. I chatted off record with others, and it turned out I was far from alone. Kris was in a similar situation, but also had work drop truckloads on him. "larsivi, December 17, 2012, Tango forums
Phobos development activity fell of significantly in mid 2018, with only two small spikes since then - the addition of CTFE-capable floating point formatting from berni44 and Paul Backus' sumtype module being accepted as std.sumtype.
Last week, I was quite disappointed when no one replied on my anouncement of a new website about D (meanwhile there are three replies, one negative, two positive, but it's somewhat too late). I'm also waiting for almost three months now for a review of my PRs concerning replacing sprintf (just doing thumbs up isn't enough). Together this made me quite sad.So I decided to stay away for a week to calm down and to have time to think about this. Finally I came to the conclusion, that I do not fit in this community. So I will leave.
berni44, Feburary 6, 2020, D forum topic "3 months of waiting..."
berni44 did come back a year later and managed to push through what would turn out to be the most recent spike in Phobos development... but it didn't last. A few months after that return spike, he left again. There are still some of berni44's PRs open, apparently still waiting on a decision or other action from D leadership to this day.
I doubt it is a coincidence that Phobos activity stopped shortly before Andrei Alexandrescu publicly announced he was stepping away from D. As frustrating as it sometimes was to work with him, he had a vision for the library and was willing to work toward that vision. But there was nobody left to continue the work when he stepped down.
If you look at dmd itself during the golden age, Walter himself did only about 1/6 of the work. There were several other people significantly involved. He now does about 1/3 of it himself - it is not that he stepped up; Walter's individual contributions have remained approximately steady. Rather, this proportion has changed because others have stepped down, again, many citing explicitly that their reason is the broken governance model of D.
"I can't work on D while my PRs are ignored for months and even years."Jonathan Marler, October 28, 2018, Github PR comment thread (on string interpolation)
The frustration felt is not limited to open source contributors.
"Every single one of the people rushing to defend D at the time has since come around. There is still some debate on whether, points vs. counter points, choosing D was a good idea, but the overwhelming consensus inside Weka today is that D has *fatal* flaws and no path to fixing them.And by "fatal", I mean literally flaws that are likely to literally kill the language."
"We've known about this problem for ages now. No movement.".
Shachar Shemesh, August 23, 2018, D forums
(Interestingly, one of the people defending D in that thread, Joakim, has since left D - pushed away by D's leaders' direct actions, and another of the D defenders in there, Mike Franklin, would himself shortly part ways with D as well over a lack of attention on his needs, despite him being a considerable contributor himself.)
Of course, this is not news to most people who have been around D for a while, including its leadership.
Around Christmas 2022, Mike Parker announced a campaign where he asked people to send him their list of "D gripes and wishes".
Many people said it was pointless, expecting nothing to come from it, but there were nevertheless a lot of responses.
"For me, dealing with D's leadership has been an infuriating experience time and time again. It's the one reason why I have dialed back my contributions to near zero. There is a lot to fix in D, but when I have to fight for the privilege of cleaning up after you guys while you refuse to even recognize your own bugs, that's just not worth it....I am sure that poor handling of contributors has cost D a lot of talent."
ag0aep6g, December 24, 2022, D Forum "Send me your list of D gripes and wishes"
It has been a year. What came from this? Approximately six months after announcing the campaign, on May 18, 2023, Mike Parker posted a document that collated people's responses into a public document. He said the next steps are to establish goals, prioritize, assign, and execute. As of this writing, the only public artifact to come out of this was setting the following goals:
When asking for the emails, the request said, "But don't write in general terms. Be specific." And people wrote volumes. Yet, when replying to everyone's complaints, those bullet points are all D's leadership had in reply.
The skeptics were right (again). Nothing would come from this.
This isn't the first time similar goals have been listed. Consider these headers from the 2016H1 Vision Document:
It is clear now that setting the goal is not enough. We need action. And we need it a long time ago.
I have been unhappy with trying to contribute up to D for many, many years. Small successes are nice, but don't make up for big failures. Seven years ago, when my improvements to ddoc hit a stone wall, I forked it to make adrdox, the program that powers this website today. In two weeks of work, I accomplished things on my own that had languished for years trying to work with upstream.
And now, I'm going to try to do that again, but this time with the whole compiler. I'm walking away Walter Bright's D and moving to OpenD, a fork I started with several other disgruntled former contributors.
What does this mean for...
This site is currently just barely, but still paid for by patreon. This includes the dub package support, so that will remain operational as long as its costs are covered. It will not receive updates for upstream D, it will probably continue to work well.
The parts dedicated to supporting upstream D specifically will be discontinued. That's why this entry doesn't have the stats anymore. But otherwise, I'll probably still write when I have something to say and the time to say it. The focus has shifted more toward my code than D already anyway. Hence, it is being renamed to "This Week in ARSD" going forward, though I might change it to "This Week in OpenD", the links will still work.
All efforts are discontinued, effective immediately. If anything benefits me, I will merge it into OpenD without even trying to send it upstream.
I've historically spent many hours a week supporting D users of all skill levels. This stops today. I feel bad for the users who will be stumped with misinformation, but I'm moving on.
If you email me asking about upstream D, I might talk to you, but no promises. I'll probably tell you to try OpenD instead.
I will no longer attend any DLF meeting unless they agree to governance changes as a precondition. Then and only then might we have something meaningful to discuss. I do not expect this to happen.
I doubt they'll even invite me. But if they do, I'll decline. I'm going a separate way now.
My D libraries have historically had a commitment to compatibility across internal versions and external compiler updates. This remains, but only for entities with paid support contracts.
For all other users, all compatibility with upstream D is cancelled immediately. It may work, it may not. The code is provided as-is, without warranty of any kind, either expressed or implied, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. The entire risk as to the quality and performance of the program is with you.
If you'd like to purchase a support contract, including potentially supporting a particular upstream D compiler, contact me to discuss terms.
Alternatively, join me in the D fork, where my support for my libraries will continue indefinitely.
I doubt many people are using these, but if you are, same as the library code: compatibility with upstream D is provided as-is, with no promises. I usually have looser compatibility guarantees on applications anyway, since they are only designed to work for me in the first place.
I've spent... a lot of time over the last 16 years on D. My computer programming career is almost entirely thanks to D, and before that, Digital Mars C++. Walter Bright and company have had an enormous effect on my life.
I really hoped this would have gone differently. Walter is a fantastic contributor to a programming language project, and I'd rather work with him than without him. But now, it is time to look back to the future. There's a lot of work to do to make OpenD succeed. There is a very good probability that we'll fail. But at least now, we'll have nobody to blame but ourselves.
(btw the section headings are quoting the video game "Perfect Dark")
Introducing OpenD, a D language fork that is open to your contributions.