Handling Binary Data and Encoding in Python

From Request to Response

A web browser can be understood as a sequence of low-level operations: create a socket, connect to a web server, send an HTTP request, receive the response in pieces, decode those pieces, and close the connection. This article follows that sequence so you can see how binary data moves through a socket and becomes readable text.

Opening the Connection

The program begins by creating a socket object and connecting it to a server on port 80. Port 80 is the standard port for HTTP, which is the unencrypted form of web traffic described in this exercise. Calling connect() starts a TCP three-way handshake: the client sends SYN, the server replies with SYN-ACK, and the client sends ACK. After this exchange, the socket has a reliable communication channel to the server's listening socket.

import socket

sock = socket.socket()
sock.connect(("remote-server", 80))

Formatting the HTTP Request

Once the socket is connected, the client sends an HTTP GET request. The request starts with GET, includes a resource path and an HTTP version, and ends with two consecutive line endings. The HTTP protocol interprets the blank line represented by \r\n\r\n as the end of the request headers.

The sequence \r\n is a carriage return followed by a line feed. The carriage return is ASCII 13 and the line feed is ASCII 10. In \r\n\r\n, the first pair ends the request line and the second pair creates a blank line. That blank line tells the server that no more headers are coming and that the request is complete.

request = "GET / HTTP/1.0\r\n\r\n"
request_bytes = request.encode()
sock.send(request_bytes)

Reading the Response Stream

The response does not necessarily arrive as one complete object. It arrives as a stream of packets, so the client repeatedly calls recv(512). Each call retrieves up to 512 bytes. The program processes each returned chunk and continues calling recv() until the server has finished sending and the call returns an empty result.

while True: chunk = sock.recv(512) if not chunk: break print(chunk.decode(), end='')

From Bytes to Text

The data returned from the socket is binary data represented as bytes. Those bytes are not yet the readable text that should be printed. Calling decode() converts each received chunk into a string. The program therefore has two distinct representations: bytes while data is moving through the socket, and a string after decoding for display.

Completing the Socket Lifecycle

import socket

sock = socket.socket()
try:
    sock.connect(("remote-server", 80))
    request = "GET / HTTP/1.0\r\n\r\n"
    sock.send(request.encode())

    while True:
        chunk = sock.recv(512)
        if not chunk:
            break
        print(chunk.decode(), end='')
finally:
    sock.close()

The try and finally structure gives the socket a clear lifecycle. It is created first, connected before use, used for sending and receiving, and closed after the communication work is finished. The receive loop ends because the server has no more data to provide, not because the client assumes that one recv() call contains the whole response.

Mistakes to Avoid

• Sending the request as a string instead of bytes.

  Python sockets work with bytes, while the request was constructed as text.

  Fix: Encode the request before sending it: sock.send(request.encode()).

• Leaving out the final blank line.

  The HTTP request needs \r\n\r\n to signal the end of its headers.

  Fix: End the request with two consecutive line endings.

• Calling recv() only once.

  The response arrives as a stream and may require multiple chunks.

  Fix: Place recv(512) inside a loop and continue until the return is empty.

• Printing received bytes without decoding them.

  The received chunk is raw binary data rather than the readable string expected for display.

  Fix: Decode each non-empty chunk before printing it.

• Adding a newline after every received chunk.

  print() adds its own newline, which can insert extra line breaks between chunks.

  Fix: Use end='' when printing chunks continuously.

Practice the Trace

Key Takeaways

1. A connection to port 80 creates a reliable channel through a TCP three-way handshake.
2. An HTTP GET request must be encoded into bytes and terminated with \r\n\r\n.
3. recv(512) retrieves up to 512 bytes at a time, so responses must be handled in a loop.
4. Each non-empty byte chunk is decoded into a string before it is printed.
5. An empty result from recv() signals that the server has finished sending, after which the socket should be closed.