03. TCP Server and Client
We’ll write a TCP server and client. The 2 programs are incomplete and incorrect in many ways, but are sufficient to demonstrate the syscalls from the last chapter.
3.1 Tips for Learning Socket Programming
Look Up Stuff in Man Pages
This is not a reference or beginner’s book, so we will not include every detail of using the socket API. You are expected to look things up.
man socket.2
The above command shows the man page for the socket()
syscall. Man pages are divided into several sections, as specified by
the numerical suffix. Examples:
man read.2
returns theread()
syscall (section 2 is for syscalls).man read
returns theread
shell command (in section 1; not what you want).man socket.7
returns the socket interface overview, not the syscall.
Find Online Resources
Man pages are for looking up things, not for learning; they are not tutorials, and they rarely explain anything. You may not be sure what is relevant, or the man page simplify does not have the answer. However, there are some good online resources that can fill the gap, such as Beej’s Guide.
And Googling is still effective in 2024 if you have a specific question.
3.2 Create a TCP Server
What the server will do: read data from the client, write a response, then close the connection.
Step 1: Obtain a Socket Handle
The socket()
syscall takes 3 integer arguments.
int fd = socket(AF_INET, SOCK_STREAM, 0);
AF_INET
is for IPv4. UseAF_INET6
for IPv6 or dual-stack sockets. This selects the IP level protocol. For simplicity, we’ll only consider IPv4.SOCK_STREAM
is for TCP. UseSOCK_DGRAM
for UDP, which is not our concern.- The 3rd argument is 0 and useless for our purposes.
There are other types of sockets that are created with different arguments, such as Unix domain sockets for IPC. We’ll only use TCP, so you can forget about those arguments for now.
The socket.2
man page is not relevant at this point. How
to create a TCP or UDP socket is documented in tcp.7
and
udp.7
respectively.
Step 2: Configure the Socket
There are many options that change the behavior of a socket. Such as configuring TCP keepalive and Nagle’s algorithm (neither of which are our concern).
However, the socket()
syscall does not have a way to
pass these of options, so another syscall setsockopt()
is
used to configure socket options after the socket has been created.
int val = 1;
(fd, SOL_SOCKET, SO_REUSEADDR, &val, sizeof(val)); setsockopt
- The 2nd and 3rd arguments specifies which option to set.
- The 4th argument is the option value.
- The option value is arbitrary bytes, so its length must be specified.
Socket options are scattered over several man pages. TCP-specific
options are in tcp.7
, while many generic options are in
socket.7
.
Most options are optional, except the SO_REUSEADDR
option; SO_REUSEADDR
is enabled (set to 1) for every
listening socket. Without it, bind()
will fail when you
restart your server. This is related to delayed packets and
TIME_WAIT
, and you can easily find explanations by
Googling.
Step 3: Bind to an Address
We’ll bind to the wildcard address 0.0.0.0:1234
.
// bind, this is the syntax that deals with IPv4 addresses
struct sockaddr_in addr = {};
.sin_family = AF_INET;
addr.sin_port = ntohs(1234);
addr.sin_addr.s_addr = ntohl(0); // wildcard address 0.0.0.0
addrint rv = bind(fd, (const sockaddr *)&addr, sizeof(addr));
if (rv) {
("bind()");
die}
struct sockaddr_in
holds an IPv4 address and port. You
must initialize the structure as shown in the sample code. The
ntohs()
and ntohl()
functions convert numbers
to the required big endian format.
For IPv6, use struct sockaddr_in6
instead. That’s why
the bind()
syscall needs the size of the structure.
The socket API we’re using is derived from the BSD socket
API, which is a simple API except for these socket address
structures; these struct sockaddr_*
are a big mess that
just emulate a tagged union.
Step 4: Listen
The listen()
syscall takes a backlog
argument.
// listen
= listen(fd, SOMAXCONN);
rv if (rv) {
("listen()");
die}
After the listen()
syscall, the OS will automatically
handle TCP handshakes and place established connections in a queue. The
application can then retrieve them via accept()
. The
backlog
argument is the size of the queue, which in our
case is SOMAXCONN
. SOMAXCONN
is defined as 128
on Linux, which is sufficient for us.
Step 5: Accept Connections
The server enters a loop that accepts and processes each client connection.
while (true) {
// accept
struct sockaddr_in client_addr = {};
socklen_t addrlen = sizeof(client_addr);
int connfd = accept(fd, (struct sockaddr *)&client_addr, &addrlen);
if (connfd < 0) {
continue; // error
}
(connfd);
do_something(connfd);
close}
The accept()
syscall also returns the peer’s address.
The addrlen
argument is both the input size and the output
size.
Step 6: Read & Write
Our dummy processing is just 1 read()
and 1
write()
.
static void do_something(int connfd) {
char rbuf[64] = {};
ssize_t n = read(connfd, rbuf, sizeof(rbuf) - 1);
if (n < 0) {
("read() error");
msgreturn;
}
("client says: %s\n", rbuf);
printf
char wbuf[] = "world";
(connfd, wbuf, strlen(wbuf));
write}
Both read()
and write()
return the number
of bytes, which is ignored in our program. This chapter only
demonstrates how to use the socket API, not how to actually do socket
programming.
3.3 Create a TCP Client
What the client will do: write something, read back from the server, then close the connection.
Make a Connection
Nothing surprising for the connect()
syscall.
int fd = socket(AF_INET, SOCK_STREAM, 0);
if (fd < 0) {
("socket()");
die}
struct sockaddr_in addr = {};
.sin_family = AF_INET;
addr.sin_port = ntohs(1234);
addr.sin_addr.s_addr = ntohl(INADDR_LOOPBACK); // 127.0.0.1
addrint rv = connect(fd, (const struct sockaddr *)&addr, sizeof(addr));
if (rv) {
("connect");
die}
char msg[] = "hello";
(fd, msg, strlen(msg));
write
char rbuf[64] = {};
ssize_t n = read(fd, rbuf, sizeof(rbuf) - 1);
if (n < 0) {
("read");
die}
("server says: %s\n", rbuf);
printf(fd); close
Compile and Run
Compile our programs with the following command line:
g++ -Wall -Wextra -O2 -g 03_server.cpp -o server
g++ -Wall -Wextra -O2 -g 03_client.cpp -o client
Run ./server
in one window and then run
./client
in another window:
$ ./server
client says: hello
$ ./client
server says: world
3.4 More to Learn
Some important but not immediately relevant things.
Demystifying `struct sockaddr`
Let’s untangle the struct sockaddr
mess.
struct sockaddr {
unsigned short sa_family; // AF_INET, AF_INET6
char sa_data[14]; // useless
};
struct sockaddr_in {
short sin_family; // AF_INET
unsigned short sin_port; // port number, big endian
struct in_addr sin_addr; // IPv4 address
char sin_zero[8]; // useless
};
struct sockaddr_in6 {
uint16_t sin6_family; // AF_INET6
uint16_t sin6_port; // port number, big endian
uint32_t sin6_flowinfo;
struct in6_addr sin6_addr; // IPv6 address
uint32_t sin6_scope_id;
};
struct sockaddr_storage {
sa_family_t ss_family; // AF_INET, AF_INET6
// enough space for both IPv4 and IPv6
char __ss_pad1[_SS_PAD1SIZE];
int64_t __ss_align;
char __ss_pad2[_SS_PAD2SIZE];
};
All of these structures start with a 16-bit integer
*_family
(despite the different types), which distinguishes
IPv4 from IPv6. They emulate a tagged union:
struct sane_sockaddr {
uint16_t family; // AF_INET, AF_INET6
union {
// whatever ...
};
};
How to use these structures:
struct sockaddr_storage
is large enough for both IPv4 and IPv6. Use it in production instead.struct sockaddr_in
andstruct sockaddr_in6
are the concrete structures for IPv4 and IPv6.- Cast a
struct sockaddr_storage
reference tostruct sockaddr_in
orstruct sockaddr_in6
to initialize/read the structure. struct sockaddr *
is the type used by the socket API, the structure itself is useless. Just cast any structure to this type.
Syscalls, APIs and Libraries
When you invoke any of the syscalls on Linux, you are actually invoking a thin wrapper in libc — a wrapper to the stable Linux syscall interface.
On Windows, the socket API follows the same BSD API, but with many different details. And the interface comes from an OS DLL rather than syscalls.
There are also many high-level libraries built on top of the socket API, but they are not good for learning, because you need the big picture first, without getting lost in small details.
More Socket: Get Addresses
You can get the address (IP + port) for both local and peer.
getpeername()
: Get the remote peer’s address (also returned byaccept()
).getsockname()
: Get my exact address (after binding to a wildcard address).
More Socket: Configure My Local Address
bind()
can also be used on the client socket before
connect()
to specify the source address. Without this, the
OS will automatically select a source address. This is useful for
selecting a particular interface from multiple interfaces.
If the port in bind()
is zero, the OS will automatically
choose a port.
The Linux-specific SO_BINDTODEVICE
option selects the
source interface by name.
More Socket: Domain Name Resolution
getaddrinfo()
resolves a domain name into IP addresses.
There is a sample program in its man page.
Unlike other socket APIs, this is not a syscall and is implemented in
libc because name resolution is a complicated and high-level function. A
name resolution on Linux typically involves reading a bunch of files in
/etc
and then sending and receiving UDP packets with a DNS
server.
The Next Step: Protocol
That’s the bare minimum of the socket API you need to know. But knowing the API of something won’t get you very far. You need to create a protocol to actually communicate over a network.
Source code: