arsd 10.5 coming this week, new midi code, a FF1 nsf player/editor application

Posted 2022-01-03

I spent much of last week writing more support code and implementing a player/editor application to hack custom music in to the old Final Fantasy 1 game.

Core D Development Statistics

In the community

Community announcements

See more at the announce forum.

What Adam is working on

ff1-nsf

I wanted to see if I could import new songs into the NES FF1 rom to help the randomizer folks do more fun things and decided to write a helper program to play around with it. It is not quite ready for a proper release, but here's the code and a preview:

https://github.com/adamdruppe/nsf-ff1/

A screenshot of the main ui showing the rom's music contents as colored rectangles

The application works by loading up the rom, extracting the song data out, playing it through the nosefart stripped-down NES emulator for audio, and letting you edit it through an interactive UI and a MIDI file importer.

I did the player first, then the display - the idea there was to be able to understand what the bytes in the file actually meant by seeing and changing as I listen - then started the midi importer.

There's a number of technical challenges to working with the old ROM:

  • Each NES game had its own music driver. What works for FF1 might not work for anything else. That's ok, ff1 is my goal here, but it limits the program a bit. I did want the player core for other games, so I factored the code so at least that is reusable, but the majority of it is pretty esoteric. Playing it actually requires emulating parts of the hardware to run the original program.
  • The music driver itself has specific data tables for both the NES and the original game. To figure out note durations, for example, I had to a special function to scan the table, multiply by frame durations, and approximate lengths as best I could. It doesn't always work out, but most melodies are recognizable.
  • Each NES audio channel can play only one sound at a time, so if you want to do some kind of musical chord, it needs to be split across channels, which is very different than midi.
  • Durations are separate across channels though. It should be easy to keep in sync anyway... but my code is still buggy on that.
  • The sounds actually generated aren't exactly standard instruments. The FF1 driver uses 16 "envelope patterns" to provide some different sounds. These vary the volume of the square wave over a timer as it sounds. You can set the speed of the timer to further customize it. But what values mean what? I made a dialog box to show some graphs to recognize them at a glance and then you can live-edit to hear the different results.
  • The rom has a total of about 7200 bytes of data space spread across over three tracks for twenty songs. There's only about 110 bytes on average for each track which must encode all the data it needs... this is the big challenge remaining on the import. The original game has music written with these restrictions in mind and used some nested loops to stretch data as much as possible. I suppose some automatic loop detection can help, but it is going to be tricky to compress into the existing driver.

I solved the emulation problem by simply using some existing code. I had an old nsf player called "nosefart" on my computer that could do the job, so I took its source code, stripped out the core, and imported it into this project. A little control functions, a callback, and a buffer forwarder to my arsd.simpleaudio made it work. And since the core code didn't actually require outside dependencies, it wasn't even that hard to get it to compile for my binary release. Indeed, you should be able to make it work from the github repo too with just a little effort.

It works, on the outside, by loading a NSF file, starting to play a track, and then running the per-frame code as-needed and filling an audio buffer. Pretty straightforward actually, I used 10 lines of bindings to it from D.

A NSF file, meaning "Nintendo Sound Format", doesn't actually come from the Nintendo company; it isn't a formal standard, but rather a hobby project for video game extraction that caught on well enough among people interested in that. Anyway, the file format consists of a 128 byte metadata header, followed by the data and 6502 code pulled out of the game, and a small 6502 initialization subroutine to bridge from the player's generic environment into the game's specific driver code.

Creating a NSF file for a specific game is an exercise for the reader. Of course, for many popular games, it has already been done, but I wanted to pull straight from the rom so I can patch it back up and play the game with modified songs, so while I had a ff1.nsf, I didn't use it directly. Instead, it, along with a FF1 music driver disassembly someone wrote up available on the romhacking.net website, were my guides to do my own extraction. This mostly consisted of finding the offsets into the file that held the data I needed (which didn't always match either the disassembly or the nsf! they did their own rearranging for ease of use/understanding for their purposes, but at least they identified the data blocks) and putting it into my own array.

As a side note, I did a little bit of 6502 programming many years ago. I briefly dabbled in NES homebrew - got about as far as a hello world with controller support, so VERY brief dabbling - but I still found the disassembly to be a pretty easy read. I think there's something to be said about the smaller instruction sets as ease of learning - sure, you have a bunch of concepts to grasp and register/memory juggling to do, but you learn the mnemonic patterns quickly enough to not need to refer back to the docs to know what an instruction does at least.

Anyway, once I got the offsets identified, extracting, modifying, and replacing the data in the rom became simple. And with a header and stub program, feeding it into the NSF player library was also simple enough. I had a music player off the rom!

After that, I added the callback to the frame function to update a UI. But now the UI had to display something meaningful, so this meant diving into the data and that annotated disassembly.

A FF1 song looks like a collection of: envelope select commands, play note commands, wait commands, tempo changes, octave changes, and loop commands. Play note and wait have a duration attached, which is an index into the current tempo table. The tempo table tells how many frames that command persists before moving on to the next one. So my program had to read these durations, keep track of the current tempo, and look up the frame count from the table to appropriately update the UI.

Once it has that info though, I could visually represent each command on a timeline and have a position bar sweep through. The loop commands were a bit tricky - they are basically goto address instructions, meaning I also had to track addresses and adapt to them as it moved. But after some work, the display functioned.

And now, since I'm parsing out the instructions, I added a minigui dialog call to the double click event to let me edit them. The minigui dialog function just auto-generates an edit window based on a struct layout... it isn't exactly beautiful, but it is a functional command editor created in just a few lines of code. Now I can edit the values and listen to the changes.

A screenshot of the raw command edit dialog

(the buttons there are useless for this use, just a side effect of the reflection-based automatic generation of the dialog. They'd be useful if it was a toolbar generated from the struct though.)

Of course, editing values isn't ideal for letting musically-inclined people who aren't necessarily asm hackers to import data, but I do think it is an essential step for me at least to understand the driver's quirks and limitations. I actually think raw edit is going to be necessary to solve the byte space problem to some degree.

But, now that the ROMs data could be played, visualized, and edited, it was time to start getting some more end-user-oriented functionality in with the midi import.

A screenshot of the midi import ui showing track checkboxes

I wrote a midi player with some music staff and piano notation last year, actually based on some old code of mine I wrote in C - yes, it predates my shift to D! But I ported it to D and put some ui on it, and used midi in and out - you can play on a midi instrument and display it in real time, and forward to a system midi output to hear it all.

I started off liberally copy/pasting from that, then started refactoring it into a new module, now called arsd.midiplayer. It has controls for use in my player ui as well as for the preview function in the ff1-nsf program. (And while there, I did some major improvements to my staff display so it now displays more correct note durations, even during live performances! But that will be a post for another day.)

The previewer pulls in the midi stream and displays a timeline style note display, a list of tracks and channels, and lets you play it. When you find something you like, you can overwrite one of the ff1 song channels with the imported data. As of now, this works quite well for some cases, but quickly runs out of space since it can't loop, and sometimes just sounds awful. But still, it is very promising.

Overall the ff1 nsf program is a nice collection of things my arsd libraries can do fairly easily - the UI and audio parts came together fairly quickly and the UDA-based menus and accelerators are REALLY nice, and those auto-generated dialogs are so convenient - but also did highlight a few bugs of mine: most especially in the ScrollableContainerWidget. It works fine for basic things. And its sister class, ScrollMessageWidget, works *beautifully* at the cost of more code for many displays. But the ScrollableContainerWidget is supposed to just work and it certainly doesn't. The minigui layout system currently has no way to communicate size overages back up to parents, which means that container widget can only figure out the scroll bar size if the sum of everything's min sizes didn't fit... and it assumes they are laid out linearly.

Where it works, it is pretty nice, but it clearly still needs a bunch of work to be a real competitor for like a css overflow:hidden div...

I also had some problems with scrolling and redrawing on Windows. Windows puts the application into a kind of nested message loop when it captures the mouse, which it does for scroll bar use and window resizing and a few other things. This is annoying because it doesn't call the simpledisplay custom message pump anymore, meaning queued redraw events (among others) never get triggered until the end!

For some cases, I like that a lot. But scrolling is a place where I want the live update so I know when to stop. I hacked around it by putting a custom event process call in the scroll message handler but.... that's not very good. If the user holds the mouse but doesn't move it, things stop. The web suggests the best way is to turn on a WM_TIMER message but I don't love that either.

I'm mostly a fan of the Windows APIs and standard controls, but this is fairly annoying. There's probably a solution I'm overlooking though, and besides, it basically works good enough as it is. Just ugh.

So yeah, while I was able to pretty quickly make this application with the libraries, there's always something to come back to later.

Other ffr stuff

Some time in the next couple weeks I need to finish the bingo web app for ffr too, hopefully I can write about that soon with the new web stuff it used as well.

adrdox improvement

The doc generator had some O(n^2) nonsense in it that I fixed a couple weeks ago. This significantly - over 10x faster in some case - improves the speed on large modules like druntime's Windows bindings. It still needs more work though.

arsd 10.5

I finally tagged 10.5 which has pieces for day job work and well as for the programs described above. Among the changes:

  • http: wait for requests as they complete, bug fixes
  • midiplayer: new module to provide a turn-key midi player thread with customization callbacks and injection methods
  • simpleaudio: enhancements, suspend method, balance on beeps etc
  • minigui: lots of enhancements to scrolling and other widgets (though still not quite happy with scrollable container), screenshot helper framework for improved documentation (not yet populated though), and a new FileName type for auto file dialogs from uda menu options
  • dom: toPrettyString tweaks
  • cgi: stop method for more programmatic control, Windows hybrid server and dispatcher support. resource not found, http 201, and refresh header helpers in dispatcher.
  • simpledisplay: beginnings of matrix helpers for opengl3+ uniforms, blocking mode flag for event loop functions to do nested temporary blocks (e.g. dialog boxes)
  • apng: hook for custom info in animation files that are stored in the png chunks and processed by your class
  • general: more modules added as dub subpackages like minigui_addons

I may write more details about these in the future when I get around to it.

UPS failure

My uninterruptable power supply started acting up again too, putting me a day behind schedule. I decided to order a new one this time... and when it arrives, I'll have to shut down the computer again.

I'm so disrupted by a computer reboot I really hate doing them. My open windows might look like a mess to you, but to me they all have some meaning. Sigh.