Exercise 52: moreweb

Now that you've created a web server using the Python http.server library you can move to the very final project. You are going to create your own web server from abolute nothing using everything you've learned so far. In Exercise 51 you created the majority of the handling that is "above" the http.server module. You didn't do any network connection handling or HTTP protocol parsing. In this final exercise you will implement all the gear necessary to replicate what http.server does for your lessweb server.

Exercise Challenge

To complete this exercise you'll want to read the Python 3 asyncio module. This library provides you with tools for handling socket requests, creating servers, waiting for signals, and most everything else you'll need. If you want an extra challenge doing this, then you can use the Python 3 select module, which provides an even lower level for handling sockets. You should use this documentation to create a series of small little socket servers and clients.

Once you understand how to create servers and clients that talk over a TCP/IP socket you'll need to move to processing HTTP requests. This part of the project is going to be daunting as the HTTP standard is insane and way more complicated than it needs to be. I would start with the simplest HTTP parsing library you can design, and then expand on it with more and more samples. The first place to start is the RFC 7230, but be prepared to experience some of the worst writing humans could devise.

The best way to study RFC 7230 is to first extract out all of the grammar listed in the Collected ABNF appendix. At first glance this seems crazy since this is just a huge grammar specification. You actually learned how to read these in Part V of this book but on a smaller scale. You know how regular expressions, scanners, and parsers work and how to read a grammar like this. All you need to do is study this grammar and implement it a little bit at a time. When implementing this I would completely ignore any of the "chunk" grammar.

Once you've studied this grammar you should start writing a parser for HTTP using what you have already created. Use your data structures, parsing tools, and everything you can to create a valid parser for a small subset of HTTP. Try to cover as much of this grammar as you can. To help you out, there is a set of test files that have valid HTTP requests in them at https://learncodethehardway.org/more-python-book/http_tests.zip. You can download this set of test cases and run them through your parser to confirm it works. I extracted many of these test cases from the excellent And-HTTP server and then augmented them with more basic examples. Your goal is to get as many of these as possible passing.

Finally, once you have a way to write a decent asyncio or select socket server and a way to parse HTTP, you can then put the two together and make your first functioning web server.