Writing Data to Files
The file handle does not contain file data; it is a connection to the file on disk that the for loop uses to read data incrementally.
The Hidden Work Behind a File Loop
A file loop can look simple: open a file, then use a for loop to process it. The important mechanism is easy to miss. Opening the file does not pull all of its data into memory. The file handle is a connection to the file on disk, and the for loop uses that connection to request the next line when each iteration begins.
Handle, Disk, and Loop
A file handle does not contain the file's complete data. It represents a connection to the file on disk. When a for loop iterates over the handle, each iteration causes Python to fetch the next line. The loop body then works with that line before the next line is fetched.
Only the current line needs to be kept for the loop's immediate work. After that line has been processed, the next line can be fetched. This is more memory efficient than loading an entire file at once and allows files of any size to be processed with only a small, constant amount of memory.
What Each Line Contains
In this pattern, the loop variable line receives one line during each iteration. A line read from the file includes its newline character when that character is present at the end of the stored line. The newline is not automatically removed before the loop body receives the line.
Tracing Three Iterations
Suppose a file contains three lines. The first two lines end with newline characters, but the final line does not. What does the loop variable receive?
Iteration 1: The loop receives the first line, including its newline character.
Iteration 2: The loop receives the second line, including its newline character.
Iteration 3: The loop receives the final line. Because the file does not end with a newline, this value does not include one at the end.
The loop processes one line per iteration, but the final line can differ: it may have no newline character when the file ends immediately after its text.
Placing Data with write()
The write() method puts data into a file exactly as provided and returns the number of characters written.
The string supplied to write() controls exactly what is inserted. In the example, the string includes a newline character, so the written data includes that line ending. The value returned by write() is the character count for the string that was written, including the newline character when it is part of that string.
Consecutive write() calls place their strings sequentially from the current file position. If the first string does not end with a newline, the next string begins immediately after it. If a string does contain a newline, the following string begins after that line ending.
write() Compared with print()
| Operation | Line-ending behavior | What must the programmer do? |
|---|---|---|
| write("A") | Writes A exactly as provided | Add a newline explicitly if a line ending is wanted |
| write("A\n") | Writes A followed by a newline | Place the newline in the string |
| print("A") | Adds a newline automatically | No explicit newline is needed for the ordinary print behavior |
Unlike print(), write() does not automatically append a newline. To create separate lines with write(), include the newline character in the strings you pass to it. Without that character, multiple calls can join their text on the same line.
Mistakes with Reading and Writing
Assuming that opening a file loads all of its data into the file handle
The file handle is a connection to the file on disk. The for loop triggers incremental reading.
Fix:
Think of each loop iteration as fetching the next line through the connection.Assuming every line variable has no newline character
A line read by the loop includes its newline character when one is present in the file.
Fix:
When tracing a line, check whether the file stored a newline at that position.Expecting write() to move to a new line automatically
write() inserts the strings exactly as provided and does not automatically add newlines.
Fix:
Use write("A\n") when the first piece of data should end a line.Ignoring the return value of write()
write() returns the number of characters written, not the written string.
Fix:
Interpret the return value as a character count.
Before predicting a file's final contents, trace every write() call in order and mark whether each supplied string contains a newline. The file result is the direct sequence of those strings; no extra newline is inserted by write().
Predict the File Result
What do you think happens?
A program performs these calls in order: write("red"), write("\n"), write("blue\n"), write("green"). What sequence of text appears in the file?
Reveal answer
Answer: red on the first line, blue on the second line, and green on the third line without an ending newline
The first call writes red without ending the line. The second call supplies the newline. The third call writes blue and another newline. The final call writes green without adding a newline.
Construct a short file-processing loop that counts the lines in a file. Then explain why the loop can process a very large file without loading all of its lines into memory at once.
Hints
- Iterate directly over the file handle with a for loop.
- Increment a counter once for each loop iteration.
- Connect the memory explanation to the fact that each iteration reads one line.
Working Rules to Remember
- A file handle is a connection to file data on disk, not the complete file data stored in memory.
- A for loop over a file handle reads one line per iteration, which keeps memory use small.
- A read line includes its newline character when that character is present in the file.
- write() inserts exactly the supplied string and returns its character count.
- write() does not add newlines automatically, so include newline characters explicitly when separate lines are required.
Key Takeaways
- Opening a file establishes a connection; iteration is what triggers incremental reading.
- A file loop keeps one current line in memory rather than loading the entire file at once.
- Newline characters can be part of the line values read from a file.
- write() writes the supplied string exactly and returns the number of characters written.
- Separate output lines require explicit newline characters when using write().