File Input and Output

Why Files Matter in Programs

Every program you write exists in memory only while it runs. Once the program stops, all variables and data disappear. But many real-world programs need to persist data: a banking application must save account balances, a game must store high scores, a data analysis tool must load measurements from yesterday's experiment. This is where files come in. Files are the bridge between your program's temporary memory and permanent storage on disk. The ability to create, read, and write files is essential to many programs.

The File I/O Workflow

Every interaction with a file follows a predictable sequence. First, your program must open the file, which tells the operating system you want access to it and establishes a connection. Next, your program performs operations: reading data from the file into memory, or writing data from memory into the file. Finally, your program must close the file, which releases the connection and ensures all data is safely written to disk. This three-step pattern—open, operate, close—is fundamental to all file handling.

Opening a File and Choosing a Mode

When you open a file in Python, you must specify a mode that tells the operating system what you intend to do with it. The mode determines whether you can read, write, or both, and critically, what happens to any existing content in the file. Choosing the wrong mode can accidentally erase data or cause your program to fail when it tries to write to a read-only file.

The File Pointer and How It Moves

When a file is open, the operating system maintains a file pointer—a position marker that tracks where in the file the next read or write operation will occur. Think of it like a bookmark in a book: it marks your current location. When you read data, the pointer advances past the bytes you just read. When you write data, the pointer advances past the bytes you just wrote. Understanding the pointer's position is crucial because it determines what data you will read next or where new data will be written.

Context Managers: Safe File Handling

Python provides a feature called a context manager, invoked using the 'with' statement, that automatically handles opening and closing files. When you use 'with', Python guarantees that the file will be closed at the end of the block, even if an error occurs. This prevents resource leaks and ensures data is written to disk. Without a context manager, you must manually close the file, and if your code crashes before the close statement, the file remains open and data may be lost.

Reading File Content

Python provides several methods to read data from a file, each suited to different situations. The read() method reads the entire file into a single string, useful for small files but wasteful for large ones. The readline() method reads one line at a time, advancing the file pointer after each call. The readlines() method reads all lines into a list of strings. The most Pythonic approach is to iterate directly over the file object, which yields one line at a time without loading the entire file into memory.

with open('scores.txt', 'r') as f:
    for line in f:
        score = line.strip()
        print(score)

85
92
78

Writing and Appending Data

Writing to a file is similar to reading, but uses the write() method instead of read(). The write() method takes a string and writes it to the file at the current file pointer position. In write mode ('w'), the file is truncated first, so the pointer starts at byte 0 and any new data overwrites from the beginning. In append mode ('a'), the pointer starts at the end, so new data is added after existing content. Remember that write() does not automatically add newlines; you must include '\n' in your string if you want line breaks.

with open('log.txt', 'w') as f:
    f.write('Starting program\n')
    f.write('Processing data\n')
    f.write('Finished\n')

To add data to an existing file without erasing it, use append mode ('a'). The file pointer starts at the end, so new data is added after the existing content. This is useful for log files where you want to record events over time without losing previous entries.

with open('log.txt', 'a') as f:
    f.write('New event logged\n')

Common Mistakes with File I/O

• Forgetting to close the file after manual open()

  The file remains open, consuming system resources. If the program crashes or runs for a long time, the operating system may run out of available file handles. Additionally, if you later try to write to the same file from another part of your program, the changes may not be visible because the file is still held open.

  Fix: Use the 'with' statement: with open('data.txt', 'r') as f: data = f.read()

• Using write mode ('w') when you meant to append

  If 'log.txt' already exists with previous entries, opening it in write mode immediately erases all existing content. You lose all historical data.

  Fix: Use append mode ('a') instead: with open('log.txt', 'a') as f:

• Trying to read from a file that doesn't exist

  Opening a non-existent file in read mode raises a FileNotFoundError. Your program crashes unless you handle the exception.

  Fix: Check if the file exists first, or use try/except to catch the error. Alternatively, use mode 'r+' or 'a' which create the file if it doesn't exist.

• Forgetting to strip newlines when reading lines

  Each line read from the file includes the trailing newline character. When you print it, you get an extra blank line because print() also adds a newline by default.

  Fix: Use strip(): print(line.strip())

• Not including newlines in write() calls

  All three lines are written consecutively without line breaks, resulting in 'Line 1Line 2Line 3' on a single line in the file.

  Fix: Add '\n' to each write(): f.write('Line 1\n')

Choosing the Right Approach for Your Task

Practice: Building a Simple Data Logger

Summary

1. File I/O follows a three-step pattern: open the file, perform read or write operations, and close the file. The 'with' statement automates the closing step and ensures safe resource cleanup.
2. File modes determine what operations are allowed and what happens to existing content. Mode 'r' reads only, 'w' writes and erases existing content, 'a' appends to the end, and 'r+' and 'w+' allow both reading and writing.
3. The file pointer is a position marker that tracks where the next read or write will occur. It starts at byte 0 in read and write modes, and at the end of the file in append mode. Each read or write operation advances the pointer.
4. Python provides multiple ways to read files: read() for the entire file, readline() for one line at a time, readlines() for all lines as a list, and direct iteration for memory-efficient line-by-line processing.
5. Always use the 'with' statement to open files. It is the standard Python idiom, prevents resource leaks, and handles errors gracefully. Never forget to include newline characters ('\n') when writing multiple lines.