Reading a Socket in Chunks
Try it: Reading a Socket in Chunks
How the Python for Everybody socket client reads a web document: it sends a raw HTTP GET, then calls recv(n) in a loop, getting pieces of whatever has arrived, until an empty result says the server closed; the headers end at the first blank line (\r\n\r\n).
How it works
- Connect a socket to the server on port 80 and send 'GET http://host/path HTTP/1.0' followed by an empty line.
- Call recv(n): it returns at most n bytes — only what is waiting in the socket buffer — and blocks while nothing has arrived yet.
- Append each chunk to data; the chunk sizes depend on the buffer size and on packet timing.
- When recv returns b'' the server has closed the connection, so break out of the loop.
- Find the first b'\r\n\r\n': before it are the status line and headers, after it is the document (Content-Length bytes).
Default run (19 steps): The client will fetch http://data.pr4e.org/romeo.txt with a raw socket and read the reply 64 bytes at a time. The server's reply is 335 bytes and will arrive in 4 packets of up to 100 bytes. … Split: data[:164] is the status line and headers; data[164+4:] is the 167-byte body — the document itself (its Content-Length says 167).
Simplified: The network is a model on a simulated clock: the reply arrives in equal-size packets at a fixed interval and the server closes with the last packet. The response headers are a fixed template. Nothing is sent over a real network.
Loading the simulation…