Build Your Own Web Server From Scratch In JavaScript Subscribe to get notified of new chapters and the book's release.
01. Introduction
1.1 Why Web Server?
You probably use a browser to surf the web every day, and HTTP is one of the technologies that powers the web. HTTP is widely used even beyond browser-server communication. It is one of the foundations of today’s software ecosystem. There are many important software products that are related to web technologies. To name a few: NGINX, Apache, and many HTTP-based RPCs.
As an IT person, you have likely been exposed to high-level applications of these techs, such as configuring servers, building apps with web frameworks. But have you ever looked at something low-level? That is, the knowledge and details of understanding and building networked applications.
While many software developers are web developers, not many software developers have peeked into the technology itself. Web servers and network applications are often seen as black boxes. That’s why I started the “Build Your Own X” book series. To learn and teach basic things by the “from scratch” approach, through concise books.
1.2 Build Your Own X From Scratch
Why take on a build-your-own-X challenge? A few scenarios to consider:
- You are a student who has just learned programming, but has no real project experience. A build-your-own-X project will exercise what you learned in school. Without such practice, people will quickly forget what they learned and build suboptimal solutions in their future careers. It is also a nice thing to put on your resume.
- You are a working software developer who seeks to level up your skills, but you are not sure what to focus on. A from scratch project will grant you the understanding of how things actually work. This kind of knowledge lasts much longer than “how to do X in framework/language Y”, which is becoming increasingly devalued as AI advances.
- You are a hobbyist who likes to code. You probably also like to learn how stuff works. Projects like this are very flexible, you can explore whatever topics interest you, or extend the project by adding bells and whistles.
- With the advances in AI tools (2023 CE), producing code is not the problem; making the code work is! The people who excel at software development will be the people who excel at debugging, i.e. the people who understand how things work in detail. This may be your chance to survive and thrive in the AI revolution.
1.3 Build Your Own Web Server
1.3.1 Network Programming
Where to start building a web server from scratch? The first step is to learn network programming, often called socket programming because the API is called socket API. However, socket programming is more than just glueing APIs or libraries together, there are some key concepts and techniques that beginners are likely to overlook.
1.3.2 Protocols & Communication
The next step is to look at HTTP itself. For two parties to communicate over a network, the data transmitted over the network must conform to a specific format, called a “protocol”.
The HTTP protocol is what we will focus on, but once you have learned enough, you can implement other protocols or create your own custom protocols. You can view this book as a generic networking book, using HTTP as an example.
1.3.3 HTTP in Detail
You probably have some vague ideas about the HTTP protocol, such as URLs, different methods like GET and POST, response codes, various headers, and etc. But have you ever thought that you can create all the details of a web server from scratch, through your own code? It’s actually not that complicated, and it’s very rewarding, as you will see.
1.4 The Book
1.4.1 Project Centric
Like my other books, this book follows a step-by-step approach. Each chapter builds on the previous chapters. You start with a simple TCP server. Adding things until you have a basic HTTP server. Then we will explore some common applications like caching, proxies, and etc.
Producing code is not the only thing you do when developing software, debugging & testing are also essential. So this book will also include some tips on debugging network applications.
Readers are advised to do the coding exercise, not just read the text, because the coding is part of the learning experience.
1.4.2 Discussions at the End of Chapter
Getting your own web server running is a very rewarding experience. However, it should not be your only goal. There are many blog posts that show you a toy HTTP server. But how do you get beyond toys? That’s why you are reading this book.
At the end of each chapter, there are additional discussions about:
- What’s missing from the code? The gap between toys and the real thing, such as optimizations and applications.
- Important concepts beyond coding, such as event loops and backpressure. These are what you are likely to overlook.
- Design choices. Why stuff has to work that way. You can learn from both the good ones and the bad ones.
- Alternative routes. You don’t have to follow exactly the path from this book.
1.4.3 Node.js and TypeScript
The project is written from scratch in JavaScript for Node.js without any dependencies, although many things are language agnostic, and you can still learn even if you don’t use JavaScript.
To the make code more readable, the code samples are actually presented in TypeScript with type annotations, although the differences between JS and TS are minor, and you don’t need to be familiar with TS to understand the code.
The code presented in the book is mostly data structures + functions. There are no fancy abstractions to hide important stuff from you, which is good for you as a learner.
1.4.4 Book Series
This book is part of the “Build Your Own X” book series. You can find other books on the official website, including books on building your own Redis, database, and compiler.