Socket tutorial

Posted 2019-11-11

Core D Development Statistics

In the community

Community announcements

See more at the announce forum.

Sockets tutorial

One of the most surprising feedback I got on my "D Cookbook" back in 2015 was people appreciated my simple socket example with Phobos. This was surprising because I didn't spend a lot of time on it and didn't think many people would care!

So today, I'm going to revisit the topic and write a little more detail.


Sockets are used for communication between two computers, or sometimes, two programs on the same computer. They can be used bidirectionally - to both send and receive data - as long as both sides agree on an address and protocol.

Socket types can be generally broken up into categories based on address type and data type. Common address types include IP, IPv6, and Unix sockets. Common data types include streaming data and datagrams. I'll explain these in more detail later.

With Phobos' std.socket, you pass these types to the Socket constructor. AddressFamily defines the address type and SocketType is the data type. You'll see there are other options in there than the ones I'll talk about here, but they are relatively rare for ordinary applications and I don't want to be writing all night.

Phobos is a thin wrapper around the C API for Berkeley Sockets, so tutorials for them will also apply here, and the information here will also apply to many other languages.

After creating a socket, the next step depends on your data type: if a datagram type, you will bind to an address and start communicating. If a streaming type, there's a difference between client and server. The server must bind to an address, start listening on that address, then accept connections before communicating. The server connects to an existing server address then can start communicating.

Let's get into the details.


The address type varies with what kind of network you and the computer you want to communicate are on.

Computers communicating across LANs or the Internet will use [AddressFamily.INET ] or AddressFamily.INET6 depending on if the network you are communicating on is IPv6 enabled. I'm going to use plain INET for my example, but they work basically the same way. You can also use these addresses for two applications on the same computer, too. These addresses are domain names or numeric sets combined with a port number. For example on port 80 or on port 7654.

Two applications on the same computer may perhaps communicate over AddressFamily.UNIX addresses, which are like filenames, for example /tmp/my_socket. (Despite its name, a UNIX address may be used on newer versions of Windows too, but that's fairly new. On Windows, I'd just use an IP address unless you are sure your program needs UNIX addresses and your target computer support it.)

UNIX address sockets are more efficient than IP sockets and have other benefits like being able to communicate local user account information and not be limited by port numbers, but cannot actually communicate over a network.

With Phobos, you construct the addresses differently:

auto unix = new UnixAddress("/tmp/my_socket");
auto ip   = new InternetAddress("", 80);

But once they are constructed, you use them the same way.

Communication by datagram

Datagrams are packets of data sent and received as a unit. They do not need a connection, meaning you can send and receive as soon as you bind, but there is the possibility that datagrams will get lost on the network and never make it to their destination. They also have a maximum size that the whole network must agree upon (typically <= 4 kilobytes). You might never know that data got lost in transit. Packets may also be rearranged in transit, so ones you send later may arrive first. On the other hand, they tend to be lower latency due to their lightweight nature.

Datagrams over the internet use UDP, the user datagram protocol.

Thus, datagrams' apparent simplicity gets messy if you need more data size or reliability. That's what stream connections are for. But if you can afford to use data, or are using a UNIX address, where there is no actual network to lose your data (though you still must abide by the maximum packet size), the datagram can be useful.

Let's write a pair of example programs to talk to each other. I'll use an IP address here, though both programs will run on your own computer due to the localhost address. If you want to try a UNIX address, simply replace the address constructor line.

First, this program will be set up to receive. It should be running first. This one will wait for a message, then reply to the sender. Make sure you read the inline comments.

1 import std.socket, std.stdio;
3 void main() {
4 	// to use a UNIX address, change the family here and
5 	// then also change the `new InternetAddress` lines to
6 	// `new UnixAddress`.
7 	auto socket = new Socket(AddressFamily.INET, SocketType.DGRAM);
9 	// note that this port is explicitly given so we can
10 	// refer to it from the other program.
11 	socket.bind(new InternetAddress("localhost", 8765));
13 	// A common mistake I see among new socket users is not
14 	// defining a buffer with a size. ALL buffers with std.socket
15 	// must have a size bigger than zero or else it will not work.
16 	char[1024] buffer;
18 	// this will be populated by the receiveFrom function, telling
19 	// us from whom the message came.
20 	Address from;
22 	try_again:
23 	writeln("Waiting for data...");
25 	auto value = socket.receiveFrom(buffer[], from);
26 	// the return value might indicate an error, which
27 	// is not necessarily fatal - it might just be because
28 	// the network was busy and you need to try again later.
29 	// I'll write more about this later.
30 	if(value == Socket.ERROR) {
31 		// Phobos does a bad job exposing errors. It
32 		// offers wouldHaveBlocked as a separate flag,
33 		// and then just a string for lastSocketError.
34 		//
35 		// On Posix systems, it can be important to check
36 		// for EINTR - that the call was interrupted
37 		// and should be retried.
38 		//
39 		// Many times the operating system can do this
40 		// for you, depending on how the signal handler
41 		// is set up. But if you aren't automatically
42 		// retrying, you need to check that code through
43 		// the C errno function.
44 		version(Posix) {
45 			import core.stdc.errno;
46 			if(errno == EINTR)
47 				goto try_again;
48 		}
49 		// otherwise, just throw the string Phobos offers.
50 		throw new Exception(lastSocketError());
51 	}
53 	// if it isn't an error though, it indicates the length of
54 	// data received. With datagram packets, they should never
55 	// have a value of zero, since that indicates the connection
56 	// is closed, and we have no connection anyway!
57 	//
58 	// So I'll just slice the buffer to get the data which may
59 	// be empty.
60 	auto data = buffer[0 .. value];
62 	writeln("Received: ", data);
64 	// and let's send a reply:
66 	auto reply = "got it!";
67 	value = socket.sendTo(reply[], from);
68 	// again, check for error
69 	if(value == Socket.ERROR)
70 		throw new Exception(lastSocketError());
71 	// but otherwise the datagram should be sent in its
72 	// entirety in one go. Anything less means the network
73 	// has some other size limit and you should probably fix
74 	// your code to work with the smaller packets.
75 	assert(value == reply.length);
76 }

And now this one will send a packet to the first one, then wait for the reply. Again, be sure to read the comments. To try for yourself, run the first program, then while it is running, run the second program and see them talk to each other.

1 import std.socket, std.stdio;
3 void main() {
4 	auto socket = new Socket(AddressFamily.INET, SocketType.DGRAM);
5 	// note the address here is set to ANY because it will
6 	// only be receiving replies to data it sends, and thus
7 	// the address does not need to be fixed to something.
8 	socket.bind(new InternetAddress(InternetAddress.ADDR_ANY, InternetAddress.PORT_ANY));
10 	string dataPacket = "Hello!";
11 	writeln("Sending data...");
12 	// datagrams don't need connections, so we just send
13 	// the packet to the address we set up in the other program
14 	auto err = socket.sendTo(dataPacket, new InternetAddress("localhost", 8765));
15 	if(err == Socket.ERROR)
16 		throw new Exception(lastSocketError());
17 	// same length assumption as described above...
18 	assert(err == dataPacket.length);
20 	// and now we will listen for a reply. All sockets can
21 	// be used to both send and receive.
22 	char[1024] buffer;
23 	Address from;
24 	err = socket.receiveFrom(buffer[], from);
25 	if(err == Socket.ERROR)
26 		throw new Exception(lastSocketError());
28 	auto data = buffer[0 .. err];
29 	writeln("Got response: ", data);
30 }
The examples show blocking sockets. If you set the blocking property to false on the socket, instead of waiting to send or receive data, it will return Socket.ERROR and wouldHaveBlocked will return true. If you use event-driven programming - more on this later - your calls will nearly never block anyway, so I usually use blocking mode. But if you want to use nonblocking mode, it is especially important to check those return values!

To recap, the steps are:

  1. Create a Socket with your desired AddressFamily and SocketType.DGRAM
  2. Call Socket.bind with an Address object
  3. Call Socket.sendTo to send packets and Socket.receiveFrom to receive packets. Don't forget to check return values, though datagrams may not report that they got lost on the network! Write your program with the assumption that the data may disappear.

Communication by stream

To solve the reliability problem and size limits incurred by datagrams, stream protocols were developed. On the internet, this is implemented by TCP, the transmission control protocol, so-named because it controls the transmission of data so it arrives in sequence, detects and resends lost data, and can handle arbitrary sizes of data.

This takes a lot of complexity off your hands in many cases, but you also must be able to process data streams instead of just standalone packets, and write a little more code to establish those reliable connections.

The socket API and operating system take care of a lot of details for you, but there are a few more function calls you need and it is extra important to process the return values on send and receive in more detail than on datagrams.

Let's get into the examples. Connection-based streams have very different flows for servers and clients, though once the connection is made, you can communicate each direction equally well.


IMPORTANT - when you receive data, check the data length! It will not necessarily match what the other side sends - TCP allows the network or operating systems to break up and combine packets on either side or in the middle.

Assume your data will not be received in a single call and may have boundaries in the middle. Read the inline comments carefully.

1 import std.socket, std.stdio;
3 void main() {
4 	// like always, first create the socket with the params
5 	auto listener = new Socket(AddressFamily.INET, SocketType.STREAM);
7 	// this is not always the right thing to do, but I often
8 	// use it - this tweaks a setting allowing you to reuse
9 	// an address that was recently used. Normally, TCP keep
10 	// the connection on hold after the program exits for a
11 	// minute or two in order to clean up any data that got
12 	// delayed on the network. But when doing quick restarts
13 	// and tests, that delay is annoying, so I turn it off here.
14 	listener.setOption(SocketOptionLevel.SOCKET, SocketOption.REUSEADDR, true);
16 	// then the server needs to bind to an address that other
17 	// people can connect to... You do not need to give the full
18 	// address unless you only want to listen on one particular
19 	// interface. Just the port is OK if you want to be accessible
20 	// from any network interface the computer is on.
21 	//
22 	// note that this can throw an exception if the address is
23 	// already in use by another program, or if your user account
24 	// doesn't have permission to use the port (ports < 1024 are
25 	// traditionally restricted to root on Linux, for example)
26 	listener.bind(new InternetAddress(2525));
27 	// then start listening. The number here is how many
28 	// connections it will keep waiting instead of rejecting
29 	// when it is busy.
30 	listener.listen(10);
32 	accept_another:
34 	// then the listener socket just accepts connections
35 	// normally, at this point, you would switch to event-driven
36 	// programming and accept new connections only when they
37 	// actually come instead of waiting for one. But for this
38 	// sample, we will just wait until someone connects by
39 	// calling `accept`. I'll show a simple event sample later.
40 	Socket socket = listener.accept();
42 	// and now you can communicate with the connection returned
43 	// by `accept`, which is just another regular socket of the
44 	// same type.
45 	string message = "Hello!";
47 	// now we have to accept the reality that our message may
48 	// be broken up by the network stack. As such we're going
49 	// to loop over until the whole thing is sent.
50 	while(message.length) {
51 		auto ret = socket.send(message[]);
52 		if(ret == Socket.ERROR)
53 			throw new Exception(lastSocketError());
55 		if(ret == 0)
56 			return; // the other side closed, we're done
58 		// advance our message for the next loop so we don't
59 		// resend any part of it.
60 		message = message[ret .. $];
61 	}
63 	// when you are done sending, you can call shutdown
64 	// to indicate that no more will come. You don't have
65 	// to do this though, instead you might just close when
66 	// you're done. But since I want to wait for a reply, I will
67 	// simply shut down sending so it can still receive before
68 	// closing.
69 	socket.shutdown(SocketShutdown.SEND);
71 	// and now we'll wait for the reply.
73 	string reply;
75 	do {
76 		// messages can be received in arbitrary chunks
77 		// too, so I am going to copy it into another
78 		// reply variable to make that more convenient.
79 		//
80 		// you might instead just stream data right through
81 		// this buffer for more efficiency, but remember
82 		// there's *no guarantee* the whole message will come
83 		// in with one call, even if your buffer is huge!
84 		char[1024] buffer;
85 		auto ret = socket.receive(buffer[]);
86 		if(ret == Socket.ERROR)
87 			throw new Exception(lastSocketError());
88 		// if receive returns zero, it means the other side
89 		// shutdown send or closed the connection.
90 		if(ret == 0)
91 			break; // other side indicated they are done
93 		reply ~= buffer[0 .. ret];
94 	} while(true);
96 	writeln("Got from client: ", reply);
98 	// we received our connection, sent it a hello, and got
99 	// a reply. now we're done, so let's close the connection.
101 	socket.close();
103 	// at this point, we could loop back up and accept another
104 	// one; `goto accept_another;` but for this demo I am just
105 	// going to call it finished and exit the program.
106 	//
107 	// closing the listener too is not strictly necessary but
108 	// it is nice to close stuff you are done with.
109 	listener.close();
110 }


The same caveat from the server applies here too! Your data may be broken up or combined. You need to treat it as a stream of bytes and process it, probably buffering too.

1 import std.socket, std.stdio;
3 void main() {
4 	// a client socket can be simpler - it just needs to
5 	// create the socket, connect, and then communicate
6 	auto socket = new Socket(AddressFamily.INET,  SocketType.STREAM);
7 	// note that connect can throw an exception if the connection
8 	// fails (try it without running the other server program!)
9 	socket.connect(new InternetAddress("localhost", 2525));
11 	writeln("Client connected!");
13 	string serverMessage;
15 	do {
16 		// I am using a char buffer here because I know I
17 		// am sending strings on the other side, but you
18 		// would very likely want to use ubyte instead for
19 		// binary protocols.
20 		//
21 		// The size of the buffer is also a bit arbitrary,
22 		// anything from 1 KB (1024) to 4 KB (4096) are
23 		// very common block sizes, but you can do something
24 		// else too if you like. Packets come in various
25 		// sizes and the kernel buffers the stream until
26 		// you receive it too.
27 		//
28 		// Just it must NOT be zero! Do not declare
29 		// `char[] buffer;`, since that has zero length
30 		// so it cannot receive anything.
31 		char[1024] buffer;
33 		// receive will return whatever is available now,
34 		// which could be more or less than we want, up to
35 		// the max size of the buffer. It returns 0 if the
36 		// other side has disconnected, or ERROR if something
37 		// has gone wrong.
38 		auto ret = socket.receive(buffer[]);
39 		if(ret == Socket.ERROR)
40 			throw new Exception(lastSocketError());
42 		if(ret == 0)
43 			break; // got it all!
45 		// now slice the buffer to get the part...
46 		auto part = buffer[0 .. ret];
47 		// and append it to build our complete message.
48 		// may also be possible to process without copying
49 		// in a real program, especially depending on your
50 		// network protocol. I like to define one where a
51 		// message size is given up front, then you read
52 		// a complete message into a buffer and process it
53 		// when it is done. Then messages may span across
54 		// packets and the end of one packet can be a new
55 		// message, but all is handled by just slicing and
56 		// copying when necessary.
57 		//
58 		// But some have other ways to indicate the end, like
59 		// a sentinel value you can detect and slice on, or
60 		// here, it is just everything available on the stream.
61 		serverMessage ~= part;
62 	} while(true);
64 	writeln("Received from server: ", serverMessage);
66 	// send our reply the same way
67 	string reply = "Thanks! Bye.";
68 	while(reply.length) {
69 		auto got = socket.send(reply);
70 		// you might be asking, why doesn't the class methods
71 		// just throw? The main reason is these errors might
72 		// be handleable, like wouldHaveBlocked(), or EINTR
73 		// (see the other example above) and you want those
74 		// to be handled here quickly. An exception could
75 		// be handled too, but the library preferred to simply
76 		// forward the underlying error codes.
77 		if(got == Socket.ERROR)
78 			throw new Exception(lastSocketError());
79 		if(got == 0)
80 			break;
82 		reply = reply[got .. $];
83 	}
85 	writeln("reply sent!");
87 	// we're done, let's close and exit.
88 	socket.close();
89 }

Event-driven programming

You can get events for small numbers of active sockets with the function. Here's an example of a socket server using that function. You might run several copies of the client above to talk to this. To exit, just press ctrl+c in the terminal running this sample, or connect five clients and it will exit once all five are done.

1 import std.socket, std.stdio;
2 import core.time; // for durations for our timeouts
4 void main() {
5 	// it starts off the same as the other server example:
6 	// create the socket, bind to an address, and start to listen
7 	auto listener = new Socket(AddressFamily.INET, SocketType.STREAM);
9 	// want to reuse again for convenience
10 	listener.setOption(SocketOptionLevel.SOCKET, SocketOption.REUSEADDR, true);
12 	listener.bind(new InternetAddress("localhost", 2525));
13 	listener.listen(10);
15 	writeln("listening");
17 	// this is a new concept though: a SocketSet holds the
18 	// sockets you are asking for events on. It must be reset
19 	// each time it is used, but only constructed once, so we
20 	// will make it here, then populate it in the event loop below.
21 	auto readSet = new SocketSet();
23 	// since we can handle multiple clients, let's make an array
24 	// for them too.
25 	Socket[] connectedClients;
26 	bool isRunning = true;
28 	// I want this program to exit after it handles 5 connections,
29 	// so I'll keep count of them here too.
30 	int totalConnectionCount = 0;
32 	// and this is the event loop...
33 	while(isRunning) {
34 		// Here's how to use that SocketSet: first, reset it
35 		// in each loop, then add all the sockets we are
36 		// interested in. Afterward, we call `select`, then
37 		// loop through again and see if there are still
38 		// events in the set.
39 		//
40 		// So first, reset and add our live sockets:
41 		readSet.reset();
42 		foreach(client; connectedClients)
43 			readSet.add(client);
44 		// including the listener for new connections!
45 		readSet.add(listener);
47 		// Now we call ``. It takes arguments
48 		// for up to three sets: one for reads, one for
49 		// writes, and one for exceptional conditions like
50 		// special out-of-band data.
51 		//
52 		// All three sets work the same way and are
53 		// independent. You can pass `null` to ones you
54 		// don't care about.
55 		//
56 		// You can also pass a timeout value if you like, which
57 		// i'll do here for demo purposes. The timeout is
58 		// triggered if no other events come before the
59 		// designated duration.
60 		//
61 		// BTW the technical definition of "event" with select
62 		// is that the given operation - read or write - will
63 		// not block when you call the function exactly once.
64 		//
65 		// A read event is triggered when there's no data if
66 		// the other side closes the connection, for example,
67 		// because it will return 0 immediately on receive.
68 		//
69 		// Similarly, an error condition existing on the
70 		// socket will also trigger the readiness event since
71 		// you can read that immediately. So still check your
72 		// return values!
73 		//
74 		// I am only checking read events, even though you
75 		// arguably should check write events too to avoid
76 		// being forced to wait for a write buffer to open up.
77 		//
78 		// Just that complicates the code a lot and if you are
79 		// only doing small writes, it is unlikely for it to
80 		// keep you waiting as long as reads.
81 		auto eventCount =, null, null, 5.seconds);
83 		if(eventCount == -1) {
84 			// select returning -1 is not necessarily
85 			// an error, it means the function got
86 			// interrupted. For example, a user might
87 			// have triggered a ctrl+c interrupt (with
88 			// a custom handler, by default that exits).
89 			//
90 			// since interruption is not fatal, we can
91 			// check flags from signal handlers (none
92 			// here) and then simply retry.
93 			continue; // retry by continuing loop
94 		}
96 		if(eventCount == 0) {
97 			// it returns 0 on the timeout.
98 			// just print and continue here
99 			writeln("Select timed out.");
100 			continue;
101 		}
103 		// otherwise, it tells us how many events are
104 		// ready. But we don't know which ones without
105 		// looping through everything we had added and
106 		// testing them. (This is why this API is not
107 		// efficient for large numbers of sockets - all
108 		// these extra loops add up.)
109 		if(eventCount > 0) {
110 			// remember to check our listener! if it
111 			// is ready, we can call accept on it.
112 			if(readSet.isSet(listener)) {
113 				// the listener is ready to read, that means
114 				// a new client wants to connect. We accept it here.
115 				writeln("accepting new socket");
116 				auto newSocket = listener.accept();
117 				newSocket.send("Hello!\n"); // say hello
118 				// skipping the error check here for brevity
119 				newSocket.shutdown(SocketShutdown.SEND);
120 				connectedClients ~= newSocket; // add to our list
122 				// we handled one event, so let's
123 				// note that there's less to do now
124 				eventCount--;
125 			}
127 			// I am doing a for loop here rather than
128 			// foreach because I will modify the
129 			// connectedClients inside and that is not
130 			// well-defined with foreach.
131 			for(int idx = 0; idx < connectedClients.length; idx++) {
132 				// if nothing else to do, no point wasting time looping
133 				if(eventCount <= 0)
134 					break;
135 				auto client = connectedClients[idx];
136 				if(readSet.isSet(client)) {
137 					// ready to read from this client!
139 					eventCount--;
141 					char[1024] buffer;
143 					// let's just read from it and
144 					// echo it back
145 					auto got = client.receive(buffer[]);
146 					if(got == Socket.ERROR)
147 						throw new Exception(lastSocketError());
148 					if(got == 0) {
149 						// they disconnected. let's keep count
150 						// and exit if we hit enough
151 						totalConnectionCount++;
152 						if(totalConnectionCount >= 5)
153 							isRunning = false;
155 						// and close it too.
156 						client.close();
158 						writeln("socket disconnected");
160 						// and then when it is closed, we need to remove
161 						// it from our active clients list
162 						connectedClients[idx] = connectedClients[$ - 1];
163 						connectedClients = connectedClients[0 .. $-1];
164 						idx--; // modifying the item in the loop is weird
165 					} else {
166 						// otherwise, we received data and I just
167 						// want to echo it back and shutdown. This
168 						// would change based on the network protocol
169 						client.send(buffer[0 .. got]);
170 						client.shutdown(SocketShutdown.SEND);
172 						writeln("socket sent data");
173 					}
174 				}
175 			}
176 		}
178 		// after handling these events, we loop back up to
179 		// wait for and handle more. Note that select will
180 		// wait until it either timeouts or has something
181 		// new, so it won't eat your CPU.
182 	}
183 }

Reading events from several different sources - like the keyboard or a gui in addition to the network - is something Phobos cannot do. You'll need OS functions or a third party library for that.

Moreover, Phobos' select function is best for small numbers of connections and is not hugely efficient for more (e.g. thousands). For these more complex event models, use operating system functions through the core.sys package, or third party libraries.

About timeouts and blocking modes

In blocking mode, a call to send or receive will wait until a timeout occurs or the data is finished to return. You might want to set these timeouts to a smaller value to make interrupting easier.

To set a send timeout, call

// 3 second example
socket.setOption(SocketOptionLevel.SOCKET, SocketOption.SNDTIMEO, dur!"seconds"(3));

For a receive timeout, use:

// 3 second example
socket.setOption(SocketOptionLevel.SOCKET, SocketOption.RCVTIMEO, dur!"seconds"(3));

On newest Phobos versions (it was buggy until recently) the wouldHaveBlocked() function will return true after the send/receive call returns Socket.ERROR to find this condition.

I sometimes use smaller timeouts to give a loop a chance to check a shouldExit flag, so we can cleanly exit more easily.

Unblocking mode, setting Socket.blocking to false, is basically like setting a timeout to 0.


Phobos' Socket class has a destructor that closes the socket! If you use the socket handle elsewhere, don't let the Socket get away from you. When it gets garbage collected, it will close the connection... even if you kept a copy of the underlying OS handle somewhere else. This can lead to frustrating random bugs!

If you are going to break the phobos Socket abstraction at all, I recommend never using it and instead use the underlying functions yourself. Thankfully, since std.socket is mostly just a thin wrapper around those underlying functions, your knowledge will almost entirely carry over to using them. Biggest downside is the Windows and Posix socket APIs are slightly different - 80% of your code will probably compile either way, but the rest of it might need some version conditions.

Performance tweaks

TCP is a fairly complex protocol and there are a lot of ways to tweak your program to get better performance. Socket.setOption can set these tweaks, but the conditions when you should use them and what specifically you should do is out of the scope of this tutorial. But remember, knowledge from C and other languages about their socket APIs will translate to D pretty easily.

Recap of tips

  • Always check return values on send/receive functions.
  • To use UNIX sockets, change the AddressFamily and Address members.
  • Remember stream data may come in smaller chunks than you expect, or may otherwise have chunk boundaries at inconvenient places.
  • Networks may take some time to detect errors, or not detect errors at all when used with datagram mode.
  • To make a TCP server, create socket, bind address, listen, then accept in loops. Accept returns the socket objects representing individual connections you can communicate with.