Loop Control and Break Statements
Buffered reading prevents memory exhaustion by reading large files in fixed-size blocks and writing each block to disk before retrieving the next, rather than loading the entire file into RAM at once.
Why Whole-File Loading Fails
When a program loads an entire large file into memory, the file occupies memory all at once. As file size grows, this can exhaust available memory, causing crashes or slowdowns. Buffered reading uses a different strategy: the program retrieves one fixed-size block, writes that block to disk, and only then retrieves the next block.
The goal is not to make the file smaller. The goal is to limit how much of it is held in memory at one time.
The Buffered Transfer Cycle
A buffered transfer repeats a small cycle. First, the source supplies a block whose size is controlled by buffer_size. Next, that block is written to disk. After the write, the program asks the source for another block. This continues until the source returns empty data.
Three-Block Transfer
Trace a source containing three sequential blocks: Block A, Block B, and Block C.
First read: The source supplies Block A. The buffer holds Block A, and the program writes Block A to disk.
Second read: After Block A has been written, the source supplies Block B. The buffer now holds Block B, and the program writes Block B to disk.
Third read: The same process retrieves and writes Block C.
Final read: The source has no remaining data, so the read produces empty data. The loop uses this signal to stop.
The disk receives all three blocks sequentially, while memory holds only the current block during each transfer.
Reading Until the Source Ends
The core loop pattern uses while True to keep requesting blocks. Each iteration calls img.read(buffer_size). The loop cannot stop merely because one block has been processed; it must request the next block and inspect the result. When read() returns empty data, the source has reached the end of the file. The break statement then exits the loop immediately.
while True: block = img.read(buffer_size) if not block: break output.write(block)
Writing Blocks Safely
The output operation belongs inside the loop, immediately after the program confirms that a nonempty block was read. This makes the transfer incremental: each block is written before the next block is retrieved. When writing downloaded files to disk, use binary mode, written as wb. Binary mode is the required mode for this technique.
| Approach | Memory behavior | Transfer behavior |
|---|---|---|
| Load the entire file | The entire file is held in memory | All data is obtained at once |
| Buffered reading | One fixed-size block is held at a time | Each block is written before the next is retrieved |
Mistakes with Loop Control
Loading the entire large file before writing it
The file can consume excessive memory and cause crashes or slowdowns.
Fix:
Read fixed-size blocks and write each block before retrieving the next.Omitting the end-of-file check
The loop lacks the termination signal identified by the buffered-reading pattern.
Fix:
Check for empty data and use break to leave the loop.Writing the block after requesting another block
The program does not follow the incremental read-then-write cycle.
Fix:
Write each nonempty block before the next read.Using text mode for downloaded binary data
The source specifically requires binary mode for writing downloaded files.
Fix:
Use wb when writing downloaded files to disk.
Practice the Control Flow
A source contains four sequential blocks. Describe what happens on each iteration of a while True loop that reads with img.read(buffer_size), stops when the returned data is empty, and writes each nonempty block to output. Include the point at which break executes.
Hints
- Track the current block in memory separately from the blocks already written to disk.
- After the fourth block is written, consider what the next read returns.
- The empty result is the termination signal.
What do you think happens?
After the final nonempty block has been written, what should the next read produce?
Reveal answer
Answer: Empty data
The source states that read() returns empty data at end-of-file. The loop then uses break to stop.
Practical Reach
Buffered reading is useful for real-world network applications because it allows programs to download files of any size without crashing or exhausting system resources. The essential design is consistent: keep only a fixed-size block in memory, write it to disk, and stop when the source reports that no data remains.
Key Takeaways
- Loading an entire large file into memory can cause slowdowns, crashes, or exhausted system resources.
- Buffered reading retrieves data in fixed-size blocks and writes each block to disk before retrieving the next.
- A while True loop can process sequential blocks when it checks each read result for empty data.
- An empty result from read() signals end-of-file, and break exits the loop.
- Downloaded files should be written in binary mode using wb, with a buffer size between 100 KB and 1 MB.