dmd beta, more info coming next time, demo of new web framework initial prototype

Posted 2019-02-18

The new dmd beta came out this week, with lots of objective-C stuff, but I haven't had a chance to try it yet. Meanwhile, we got another LDC release, and in my world, I made some more progress on my new D web framework.

Core D Development Statistics

11 bugs fixed
11 bugs and enhancement requests opened
31 pull requests merged into the language: 18 into DMD, 6 into Phobos, and 7 into druntime.
5 pull requests merged into the website.

In the community

Community announcements

See more at the announce forum.

What Adam is working on

I continue to make progress on the REST code, but I'm still not happy with it yet - I usually don't write about my libraries... well, barely at all, but especially not as I go, so I keep optimistically saying "next week", but I should stop promising that. My wrist gets frequently sore, so even when I have time, I can't always be hacking on code :(

In any case, last week, I wrote about the RPC-style generator, where it wraps method classes with web interfaces, and this week, I got the REST-style generator basically working.

The way it works is you write a pretty basic data class, dispatch to it, and let the library do the rest (pun ALWAYS intended ;P). Behold:

1 import arsd.cgi;
2 import arsd.postgres;
3 
4 class Employee : RestObject!() {
5 	// implement the data members as plain old data
6 	int id;
7 	string name;
8 	string title;
9 	int salary;
10 
11 	// then do the required basic functions as overrides
12 	// create is responsible for creating a new object...
13 	override string create(scope void delegate() applyChanges) {
14 		applyChanges(); // this will apply the changes from the HTTP request
15 		// now the data members are set on this object as set from the web,
16 		// so we can work with it.
17 
18 		// I'm just going to set a random id and save for this demo
19 		import std.random;
20 		id = uniform(2, int.max);
21 		save();
22 
23 		// create returns the URL slug, which I want to just be the ID as a string...
24 		return to!string(id);
25 	}
26 
27 	// save is called after a PUT or PATCH request, to save changes (actually called by a base class
28 	// default implementation of `replace` and `update`, which you can override individually and do
29 	// it differently - but the default just calls load();, then applyChanges(); followed by save();)
30 	override void save() {
31 		// this implementation is nothing special; in fact, I want to later come back
32 		// and add some reflection helpers to automate this with a storage backend of
33 		// your choice too.
34 		auto db = new PostgreSql("dbname=test");
35                 try {
36                         db.query("INSERT INTO employees (id, name, title, salary) VALUES (?, ?, ?, ?)",
37                                 id, name, title, salary);
38                 } catch (DatabaseException e) {
39                         // badly written upsert since I don't have my
40 			// DataObject class ready for postgres here yet :(
41                         db.query("UPDATE employees SET name = ?, title = ?, salary = ? WHERE id = ?", name, title, salary, id);
42                 }
43 	}
44 
45 	// like save, this is called by the default implementations of show(), replace(), and update().
46 	override void load(string urlId) {
47 		// again, nothing special to see here.
48 		auto db = new PostgreSql("dbname=test");
49 		foreach(row; db.query("SELECT id, name, title, salary FROM employees WHERE id = ?", urlId)) {
50 			this.id = to!int(row[0]);
51 			this.name = row[1];
52 			this.title = row[2];
53 			this.salary = to!int(row[3]);
54 		}
55 	}
56 
57 	// Note that DELETE is not implemented on this object, but the reflection doesn't realize
58 	// that yet. So the form will present that option, but it won't do anything.
59 }
60 
61 // a collection can be auto-generated, but it will probably need some custom code
62 // to load, so we can subclass that too. It is templated on the type of the collection..
63 class Employees : CollectionOf!(Employee) {
64 	// and the one method you will want to implement is index.
65 	// later, it will support pagination, searching, and sorting, but
66 	// all I have implemented so far is the basic listing.
67 	override IndexResult index() {
68 		IndexResult ir;
69 
70 		auto db = new PostgreSql("dbname=test");
71 		foreach(row; db.query("SELECT id, name, title FROM employees LIMIT 30")) {
72 			// it is expected to return the object in question with data
73 			// members set. It shouldn't expect ALL members set though, but I haven't
74 			// fully implemented that detail.
75 			auto e = new Employee();
76 			e.id = to!int(row[0]);
77 			e.name = row[1];
78 			e.title = row[2];
79 			ir.results ~= e;
80 		}
81 
82 		return ir;
83 	}
84 }
85 
86 void handler(Cgi cgi) {
87 	// and finally, our request handler will just dispatch the employees path
88 	// to the rest object for the collection.
89 	if(cgi.dispatcher!(
90 		"/employees".serveRestObject!Employees,
91 	)) return;
92 
93 	// and if the dispatcher didn't handle it, it will redirect to the index.
94 	cgi.setResponseLocation(cgi.scriptName ~ "/employees");
95 }
96 
97 mixin GenericMain!handler;

Well, the comments in there speak for the code. You can play with the result of this code live here (assuming it is still live when you read this).

As you go through the generated site, you'll see many imperfections - the code isn't quite done yet. But it basically works and the sample code gives you an idea of where I'm going, though the API is also still likely to change a little as I actually use it.

Speaking of APIs, we can use this same code for JSON work. Just throw ?format=json on the url. for example.

You can also use your HTTP verbs, like with curl:

$ curl -v http://arsdnet.net/cgi-bin/ncgi2/employees?format=json -d name='Name here' -d title='My Title' -d salary=54332> POST /cgi-bin/ncgi2/employees?format=json HTTP/1.1
> User-Agent: curl/7.41.0
> Host: arsdnet.net
> Accept: */*
> Content-Length: 42
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 42 out of 42 bytes
< HTTP/1.1 201 Created
< Date: Mon, 18 Feb 2019 20:37:02 GMT
< Server: Apache/2.4.6 (Unix) OpenSSL/1.0.1e mod_fcgid/2.3.7-dev PHP/5.6.23
< Cache-Control: private, no-cache="set-cookie"
< Expires: 0
< Pragma: no-cache
< Location: /cgi-bin/ncgi2/employees/375317407
< Transfer-Encoding: chunked
< Content-Type: text/html; charset=utf-8
<
* Connection #0 to host arsdnet.net left intact
The object has been created.

Notice the 201 response and the location in the header of the new object. Now, I want to edit it:

$ curl -v --request PATCH http://arsdnet.net/cgi-bin/ncgi2/employees/37 title=Editedt=json -d
*   Trying 74.79.25.214...
* Connected to arsdnet.net (74.79.25.214) port 80 (#0)
> PATCH /cgi-bin/ncgi2/employees/375317407?format=json HTTP/1.1
> User-Agent: curl/7.41.0
> Host: arsdnet.net
> Accept: */*
> Content-Length: 12
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 12 out of 12 bytes
< HTTP/1.1 200 OK
< Date: Mon, 18 Feb 2019 20:44:26 GMT
< Server: Apache/2.4.6 (Unix) OpenSSL/1.0.1e mod_fcgid/2.3.7-dev PHP/5.6.23
< Cache-Control: private, no-cache="set-cookie"
< Expires: 0
< Pragma: no-cache
< Content-Length: 67
< Content-Type: application/json
<
* Connection #0 to host arsdnet.net left intact
{"id":375317407,"name":"Name here","salary":54332,"title":"Edited"}

And you can see the result there is more-or-less what you probably expected, though the code has no special handling for any of this!

Feel free to play with my demo a bit, just be aware I am probably going to wipe the database randomly and might take the demo down later, so don't be alarmed if it doesn't work.

Well, I am going to end here without much more talk, since I need to get off the computer again... blargh. But, next time I get some time, I will write more about the implementation. Much of it is simple enough - just some convention over configuration, basically - but some of the reflection code is still worth talking about, and, of course, the justification of various "RESTful" decisions (the RestObject is not a DataObject per se; it is meant to define api rather than data storage, but I don't wanna go into detail of that... yet) and MVC architecture - or lack thereof - will be interesting. Controversy alert! lol

(And, of course, I still need to finish the Presenter class, which will let you fully customize the generated html!)

BTW: the demo is running in cgi mode, but as always, cgi.d can host in many modes, including the embedded http server... which got another little performance improvement this week. It can run 44k reqs/sec on my computer; not too awful.

PS D rox. The modeling power with the static information is really fun to play with.

Blog Articles