04. Protocol Parsing
Our server will be able to process multiple requests from a client, to do that we need to implement some sort of “protocol”, at least to split requests apart from the TCP byte stream. The easiest way to split requests apart is by declaring how long the request is at the beginning of the request. Let’s use the following scheme.
+-----+------+-----+------+--------
| len | msg1 | len | msg2 | more...
+-----+------+-----+------+--------The protocol consists of 2 parts: a 4-byte little-endian integer indicating the length of the following request, and a variable length request.
Starts from the code from the last chapter, the loop of the server is modified to handle multiple requests:
while (true) {
// accept
struct sockaddr_in client_addr = {};
socklen_t socklen = sizeof(client_addr);
int connfd = accept(fd, (struct sockaddr *)&client_addr, &socklen);
if (connfd < 0) {
continue; // error
}
// only serves one client connection at once
while (true) {
int32_t err = one_request(connfd);
if (err) {
break;
}
}
close(connfd);
}The one_request function only parses one request and
replies, until something bad happens or the client connection is gone.
Our server can only handle one connection at once until we introduce the
event loop in later chapters.
Adding two helper functions before listing the
one_request function:
static int32_t read_full(int fd, char *buf, size_t n) {
while (n > 0) {
ssize_t rv = read(fd, buf, n);
if (rv <= 0) {
return -1; // error, or unexpected EOF
}
assert((size_t)rv <= n);
n -= (size_t)rv;
buf += rv;
}
return 0;
}
static int32_t write_all(int fd, const char *buf, size_t n) {
while (n > 0) {
ssize_t rv = write(fd, buf, n);
if (rv <= 0) {
return -1; // error
}
assert((size_t)rv <= n);
n -= (size_t)rv;
buf += rv;
}
return 0;
}Two things to note:
- The
read()syscall just returns whatever data is available in the kernel, or blocks if there is none. It’s the application that is responsible for handling insufficient data. Theread_full()function read from the kernel until it got exactlynbytes. - Likewise, the
write()syscall can return successfully with partial data written if the kernel buffer is full, we need to keep trying when thewrite()returns fewer bytes than we need.
The one_request function did the actual work:
const size_t k_max_msg = 4096;
static int32_t one_request(int connfd) {
// 4 bytes header
char rbuf[4 + k_max_msg + 1];
errno = 0;
int32_t err = read_full(connfd, rbuf, 4);
if (err) {
if (errno == 0) {
msg("EOF");
} else {
msg("read() error");
}
return err;
}
uint32_t len = 0;
memcpy(&len, rbuf, 4); // assume little endian
if (len > k_max_msg) {
msg("too long");
return -1;
}
// request body
err = read_full(connfd, &rbuf[4], len);
if (err) {
msg("read() error");
return err;
}
// do something
rbuf[4 + len] = '\0';
printf("client says: %s\n", &rbuf[4]);
// reply using the same protocol
const char reply[] = "world";
char wbuf[4 + sizeof(reply)];
len = (uint32_t)strlen(reply);
memcpy(wbuf, &len, 4);
memcpy(&wbuf[4], reply, len);
return write_all(connfd, wbuf, 4 + len);
}For convenience, we added a limit to the maximum request size and use
a large enough buffer to hold the request. Endianness used to be a
consideration when parsing protocols, but it is less relevant today so
we are just memcpy-ing integers.
The client code for making requests and receiving replies:
static int32_t query(int fd, const char *text) {
uint32_t len = (uint32_t)strlen(text);
if (len > k_max_msg) {
return -1;
}
char wbuf[4 + k_max_msg];
memcpy(wbuf, &len, 4); // assume little endian
memcpy(&wbuf[4], text, len);
if (int32_t err = write_all(fd, wbuf, 4 + len)) {
return err;
}
// 4 bytes header
char rbuf[4 + k_max_msg + 1];
errno = 0;
int32_t err = read_full(fd, rbuf, 4);
if (err) {
if (errno == 0) {
msg("EOF");
} else {
msg("read() error");
}
return err;
}
memcpy(&len, rbuf, 4); // assume little endian
if (len > k_max_msg) {
msg("too long");
return -1;
}
// reply body
err = read_full(fd, &rbuf[4], len);
if (err) {
msg("read() error");
return err;
}
// do something
rbuf[4 + len] = '\0';
printf("server says: %s\n", &rbuf[4]);
return 0;
}Test our server by sending multiple commands:
int main() {
int fd = socket(AF_INET, SOCK_STREAM, 0);
if (fd < 0) {
die("socket()");
}
// code omitted ...
// multiple requests
int32_t err = query(fd, "hello1");
if (err) {
goto L_DONE;
}
err = query(fd, "hello2");
if (err) {
goto L_DONE;
}
err = query(fd, "hello3");
if (err) {
goto L_DONE;
}
L_DONE:
close(fd);
return 0;
}Running the server and the client:
$ ./server
client says: hello1
client says: hello2
client says: hello3
EOF
$ ./client
server says: world
server says: world
server says: worldThe protocol parsing code requires at least 2 read()
syscalls per request. The number of syscalls can be reduced by using
“buffered IO”. That is: read as much as you can into a buffer at once,
then try to parse multiple requests from that buffer. Readers are
encouraged to try this as an exercise as it may be helpful to understand
later chapters.
Notes on protocols: The protocol used in this chapter is the most simple practical protocol. Most real-world protocols are more complicated than this. Some use text instead of binary data. While text protocols have the advantage of being human-readable, text protocols do require more parsing than binary ones, which are more coding and error-prone. Another thing to complicate protocol parsing is that some protocols don’t have a straight way to split messages apart, those protocols may use delimiters, or require further parsing to split messages. The use of delimiters in protocols can add another complication when the protocol is carrying arbitrary data, as the delimiters in data need to be “escaped”. We’ll stick to the simple binary protocol for later chapters.
Source code:
codecrafters.io offers “Build Your Own X” courses in many programming languages.
Including Redis, Git, SQLite, Docker, and more.