File I/O and Data Persistence
The pickle module converts Python objects into byte streams for storage and retrieves them back into memory.
From Memory to Persistence
A Python object normally exists in your program's memory while the program is running. The pickle module provides a way to convert that object into a byte stream for storage and later retrieve it back into memory. This creates a simple persistence lifecycle: an object is created, serialized into bytes, written to a file, read later, and reconstructed as a Python object.
The file is independent of the variable that originally referred to the object. Once pickle.dump() has stored the object in the file, deleting the variable removes that variable from memory but does not remove the stored data from the file.
Saving with pickle.dump
To save an object, call pickle.dump(object, file). The first argument is the Python object to serialize. The second argument is an opened file. For pickle operations, the file must be opened in wb mode: binary write mode. The dump operation converts the object into a byte stream and writes that stream to the file.
After pickle.dump() completes, the list has been serialized and written to shoplist.pkl. This example does not print the list; its result is stored data in the file.Restoring with pickle.load
To retrieve the stored object, open the file in rb mode: binary read mode. Then call pickle.load(file). The load operation reads the stored byte stream and returns the object back into memory. The returned object can be assigned to a new variable such as storedlist.
["apples", "bread", "milk"]The Variable Can Disappear
What do you think happens?
The variable shoplist is created, saved to a file, then deleted. Later, it is loaded into a new variable called storedlist. What do you expect storedlist to contain?
Reveal answer
Answer: The list that was saved
The del keyword removes the variable from memory, but the data has already been stored in the file through pickle.dump(). The file persists independently of whether shoplist still exists.
Saving, deleting, and loading a list
Trace shoplist as it is saved, removed from memory, and loaded into storedlist.
Create: shoplist refers to a list in the program's memory.
Serialize: pickle.dump(shoplist, file) converts the list into a byte stream and writes it to the file opened in wb mode.
Delete: del shoplist removes the variable from memory. It does not affect the data already stored in the file.
Restore: Opening the same file in rb mode and calling pickle.load(file) retrieves the stored data and assigns the resulting object to storedlist.
storedlist contains the list that was saved, even though shoplist no longer exists in the program.
The important distinction is between a variable in memory and the data stored on disk. del shoplist destroys the reference to the list in the program, but it does not erase the pickled file. Loading the file later creates access to the stored data through storedlist.
Binary Modes Compared
| Operation | Function | File mode | Direction |
|---|---|---|---|
| Save an object | pickle.dump(object, file) | wb | Object to file |
| Restore an object | pickle.load(file) | rb | File to object |
The binary mode must match the pickle operation.
Objects, Bytes, and Files
These are different stages of the same persistence process. The Python object is the in-memory value your program works with. The byte stream is the serialized form produced for storage. The file holds that stored byte stream. pickle.load() reverses the storage direction by retrieving the data and returning it as an object in memory.
Mistakes That Break the Workflow
Opening the file in a text mode instead of a binary mode
Pickle operations work with byte streams, so the file mode must be binary.
Fix:
Use wb when saving with pickle.dump() and rb when restoring with pickle.load().Forgetting to close the file after dumping
Forgetting to close files is identified as a common file-handling error in this workflow.
Fix:
Close the file after the dump operation.Assuming del removes the saved data
del removes the variable from memory, while the file persists independently.
Fix:
Treat the in-memory variable and the stored file as separate parts of the lifecycle.
Practice the Lifecycle
Write a short Python sequence that creates a list named shoplist, opens a file in wb mode, saves the list with pickle.dump(), closes the file, deletes shoplist, then opens the file in rb mode and loads the data into storedlist. Finally, print storedlist.
Hints
- Import pickle before using pickle.dump() and pickle.load().
- Use pickle.dump(shoplist, file) when saving.
- Use pickle.load(file) when restoring.
- Use wb for the saving step and rb for the restoring step.
- The loaded value should be assigned to storedlist.
Before running your solution, trace four states: the list exists in memory, the list has been serialized to the file, shoplist has been deleted, and storedlist contains the restored object.
Key Takeaways
- pickle.dump(object, file) serializes a Python object into a byte stream and saves it to a file.
- pickle.load(file) reads the stored byte stream and returns the object back into memory.
- Use wb for binary writing during serialization and rb for binary reading during deserialization.
- Deleting a variable removes it from memory but does not affect data already stored in the pickled file.
- Close files after completing pickle operations.