new textlayout class, dconf online thoughts, step by step tech progression

Posted 2022-12-19

Right after the DConf Online weekend, I have a brief write up, and some of the design on my new text layout class after a history lesson on why version one and two of that haven't worked out.

Core D Development Statistics

In the community

Community announcements

See more at the announce forum.

DConf Online 2022

My livestream

I did my third livestream for the dconf online which you can watch back at https://www.youtube.com/watch?v=CSefrU-3WJ0. I showed my FF1-nsf program which I blogged about back in January and spent half the time just talking about it rather than coding it, but it is a fun look into some old hardware and some practical minigui implementation code. Then I transitioned into writing a miniature media player and got sidetracked pretty quickly seeing if we could automatically create a dialog from an interface. The answer is no - i only did classes. So I edited minigui.d to add support for interfaces, which worked well enough, but I then wanted it to refer to a changing object and started to add some pointer support and got a bit down the rabbit hole. Then I ran out of time.

I'll go back and finish that functionality later though, as it will be useful.

Other thoughts

I don't really like the format here. You want to watch the talk, but also get serious FOMO about the live chat, so what I did was watch 3 minutes of one at 2x speed, then switch to the other at 2x speed, and back and forth until the end. It actually did kinda work, and I'd only be a few minutes behind on live chat on either. But it is a bit awkward. Thankfully, Mike Parker agreed, and is going to try something else next time. I suggested doing it like a reaction video and he said he is going to give it a try. I think that will work a lot better.

That said, on the content, I have just a few main comments:

1) Atila's push on strings for his reflection lib I think is a mistake. As I've said before here, I prefer to minimize the string use, when you do use them, prefer local names instead of trying to construct fully qualified names, and just do what you want to do in place instead piling on more layers of abstraction than the compiler already has.

2) Robert Schadek has the right idea when it comes to nogc... just don't mess with that nonsense! It is true a lot of javascript can be fairly easily ported to D. Indeed, I've done some myself in this blog, if you remember my "declarative ui in D" post, the noise generator there was a JS port.

3) I liked Steve's model ideas and agree with him that apis should be pleasant to use. He and I have some overlapping projects right now.

4) Brian Callahan is just about in the middle of Steve and I, and Mike Shaw is also in that radius, as well as a few other people I know in the D community. Maybe we'll arrange a mini-dconf at the RPI campus some day in the spring or something.

5) Larry's thing reminded me of some of the old dforms stuff. I actually found it crash-happy and generally buggy when I tried to use it back in the day, but that's part of why I started minigui! So fun times. I've considered writing a custom editor and I might get around to it some day, vim does annoy me sometimes. But first I need a new text layout system... more on that in a moment.

What Adam is working on

arsd user survey

I put out a thread about the arsd.core idea, to see if any users object to the breaking change. See: https://forum.dlang.org/post/ieecciayudxcgmezkcxn@forum.dlang.org so far, only mild pushback with most people saying they don't mind the minor build breakage. We'll see after more people have a chance to see it.

dom.d's XmlDocument

Otherwise, I did a bug fix / breaking change in dom.d's XmlDocument - before it would give <script> and <style> tags the same special treatment as HTML, even though XML is not HTML. I've undone this. But this might break things that depended on the old behavior, which is technically buggy... but I think it also might be important for things like svg. I might be forced to actually read a DTD unless people regularly use the CDATA marker in it. But still, overall the XmlDocument should not have HTML-specific behavior, so I'm going to keep removing these.

TextLayouter

And then the big thing is I've started yet another TextLayouter class. In January of last year, so almost two years ago now, I started writing ExperimentalTextComponent2 and blogged about it here: http://dpldocs.info/this-week-in-d/Blog.Posted_2021_01_25.html#simpledisplay-text. This was supposed to replace the awfully buggy ExperimentalTextComponent upon which minigui on Linux builds its TextEdit (and LineEdit, etc) widgets.

While ExperimentalTextComponent2 looked promising at first, it ended up moving toward one of the same walls as ExperimentalTextComponent v1 and I couldn't find a way around it, and it broke the api on top of it, so it got put indefinitely on hold until I could think of a solution and I left v1 in place - along with all its longstanding bugs - for minigui.

v1 was written before I wrote the OperatingSystemFont class. All the functions it had were tied to the simpledisplay ScreenPainter, so you had to have an instance available for it to do the layout; it would lay out as it draws (or at least pseudo-drew) then cache some info it can use for future work. And since it was tightly coupled to ScreenPainter anyway, it would also draw itself and its parts like selections and cursors (which is why the blink bugs would happen... it doesn't know minigui's focus model).

I also wanted it to support fancy rich text - a goal I always have in mind is doing something like a html editor - and it thus has notions of block elements and inline elements internally. This added complication and didn't really work anyway.

So for v2, now that OperatingSystemFont actually worked, I tried to decouple from ScreenPainter and simplify the scope at the same time, but I still wanted to keep the door open to that html goal. This did make real progress, it avoided the direct ScreenPainter dependency while layout out... but still had a draw(ScreenPainter) method. It avoided some of the mess of v1's internal thing and had some extensibility through its ComponentInFlow interface (a concept I actually intend to keep in v3, but haven't actually written yet so that's subject to change)... but it still exposed too much of its innards to the world, breaking all the apis and limiting what it can do in both capabilities and optimizations.

And now I'm working on v3. Like v2, it is mostly based on OperatingSystemFont... but not exactly. Specifically, it uses the MeasurableFont interface which OperatingSystemFont is one implementation of. I only made MeasurableFont in a refactor a few months ago, but it fits this beautifully and has impacted the direction going here... see OperatingSystemFont is simpledisplay specific, but MeasurableFont, while living in simpledisplay, could just as easily live somewhere else and have totally different implementations. It doesn't even strictly need to do measurements in pixels (though whatever units it does use will have to be compatible with the click coordinate space, you can apply transformations at the edges if you want) meaning it might even work in a character cell terminal.

Now that it is fully decoupled in metrics, the text layouter need not draw itself. Instead, it exposes a method to iterate through renderable segments. These segments have some text, some drawable info like the baseline and bounding box in the abstract coordinate space, and a relatively opaque TextStyle instance. The TextStyle must be able to provide the font metrics - so it has a method to return a MeasurableFont - but otherwise the layouter doesn't really care what it has. It doesn't even care what color the text is, so even that seemingly-basic info is not part of the interface! And it doesn't need to be. (Note: I might change some exact details cuz your draw thing having to cast it back is a bit obnoxious, but I still plan to use the minimal interface internally.) This also means the drawing code might style text based on things totally unknown to the layout engine, for example, syntax highlighting as it draws based on those strings instead of embedding that info in the layout's own data structures.

So, unlike v2 which limited you to the styles I decided to provide, you can now use the layouter with your own user-defined style options.

Similarly, v3 uses a delegate to determine the margin at various points. The driving force here is something like a css float, where the text appears beside an image then wraps around below it. v1 and v2 both had the notion of exclusion zones, represented as Rectangles. v3, instead, uses a delegate, meaning you can define whatever shapes you want (though only on the sides, not in the middle of the text, but I think this is an acceptable simplification. And with the api designed how it is - with the render segment boxes and other things, there's a good chance I can go back and change this later if I do change my mind.) This is simultaneously simpler to code and more flexible than the old approach.

Aside from layout and render, another key component of a text system is navigation and manipulation. v1 did this based on its own methods. v2 never really got to this point before I put its implementation on hold. v3 already has it working, based on the concept of selection objects. You can move around, set anchors and focuses, then delete, insert, get data, etc. This gives you significant access without needing to expose any underlying implementation details - the selection is a fairly opaque object that doesn't tell you things like text offsets - this meaning I don't have to use arrays since the user can't slice it anyway, patching a problem both v1 and v2 had. Sure, my PoC implementation is an array, but I can add gaps or indexes or whatever other data structures later without breaking the user api.

Additionally, these apis end up being pretty easy to use, since you can map them to familiar actions in a text editor you already know.

Finally, I decided to support multiple selections/cursors, which is itself a useful text editing tool to users, and you can make them be controlled by different users, to get that Google Doc style collaboration experience.

I still have 2-4 days of work to do to finish this, and it isn't exactly top priority, but I still think I can find the time to do it over the next couple weeks. It, even unfinished, has already shown bug fixes over v1 and capability improvements over v2, and it has even more potential. Maybe Minigui's custom TextEdit will switch from being an embarrassment to being something we can have pride in!

Release schedule

Once all this work is done, I might go ahead and add the dependency on arsd.core, even if it doesn't actually do much yet, and tag arsd 11.0 in early January instead of waiting for May like I had originally planned. But we'll see how things go with the user survey request and work over the Christmas weeks. I have a lot of things on my list still, but the XmlDocument thing and the arsd.core introduction are the main breaking changes so I can put them out then do the rest in 11.1 etc. We'll see, I'm not in a huge rush to release anything yet.

Final thought

This text thing brings up something I've noticed a few times: there's some things I try to do over and over and just keep failing to do it well, then suddenly there's a time when it gets easy and I wonder why I couldn't just do that a long time ago. I think it is a combination of a few reasons:

Sometimes I just need to give up on something in order to rethink it from another angle, and some time in between attempts helps take that fresh look, and of course, I might have just leveled up my mad skillz in the mean time (such as reading other things that end up being useful in the implementation that I just didn't know befre).

But I also think a lot of it has to do with more infrastructure being written in the mean time. The key difference between ExperimentalTextComponent 1 and 2 was the presence of OperatingSystemFont's metric methods. That plus ScrollMessageWidget have helped a lot in the new code (though I do think the key difference here is the new approach, the new infrastructure definitely helped too).

It reminds me of thought I've had in the past about science and technology in general. You need scientific knowledge to make technology but you need new tech to run new experiments to get more scientific knowledge. And you need group labor on machines to make machines to do more labor to make new infrastructure. So there's a slow circle, step by step that have to be taken in order to enable the next. It isn't a surprise to me that often new things are independently discovered or invented within some time of each other - it often isn't so much individual genius (though that likely helps!), but just that the conditions have now been built up to make it possible to take that next step.

And my... like 15 years now of working on arsd is a microcosm of the same idea, a slow spiral, walking up the steps one at a time.