Opening and Closing Files in Python
The pickle module converts Python objects into byte streams for storage and retrieves them back into memory.
From Memory to Storage
A Python object normally exists in your program's memory. The pickle module provides a way to convert that object into a byte stream for storage in a file. Later, pickle can retrieve the stored data and restore it into memory.
What do you think happens?
The variable shoplist is created, saved to a file, then deleted. Later, the file is loaded into a new variable called storedlist. What do you expect storedlist to contain?
Reveal answer
Answer: The saved list
del shoplist removes the variable's reference from the program's memory. It does not affect the data that pickle.dump() already stored in the file.
The Object Lifecycle
The lifecycle has two directions. During saving, pickle.dump(object, file) converts a Python object into a byte stream and places that stream in the file. During retrieval, pickle.load(file) reads the stored data and brings the object back into memory. The variable name can change during this process: the original object may be called shoplist, while the restored object may be called storedlist.
Saving with dump
To save an object, open the file in binary write mode, wb. Then pass the object and the opened file to pickle.dump(). After the operation is complete, close the file. The binary mode matters because pickle works with byte streams rather than text data.
import pickle shoplist = ["apples", "bread", "milk"] file = open("shoplist.pkl", "wb") pickle.dump(shoplist, file) file.close() del shoplist
The essential saving pattern is pickle.dump(object, file) with the file opened in wb mode. The object is serialized into bytes for storage.
Restoring with load
import pickle file = open("shoplist.pkl", "rb") storedlist = pickle.load(file) file.close() print(storedlist)
['apples', 'bread', 'milk']The original shoplist variable is no longer needed for loading. pickle.load() uses the data in the file and creates the restored object assigned here to storedlist. This demonstrates that the file persists independently of whether the original variable still exists in memory.
Choosing the File Mode
| Task | File mode | Pickle operation | Data direction |
|---|---|---|---|
| Save an object | wb | pickle.dump(object, file) | Object to file |
| Restore an object | rb | pickle.load(file) | File to object |
The binary mode must match the direction of the pickle operation.
Text modes are a common source of errors in pickle programs. The required modes for these operations are the binary modes wb and rb, because pickle stores and retrieves byte streams.
Mistakes to Avoid
Opening the file in a text mode instead of a binary mode.
pickle converts objects into byte streams, so its file operations require the binary modes wb and rb.
Fix:
Use wb with pickle.dump() and rb with pickle.load().Using pickle.load() while trying to save an object.
pickle.load() retrieves data from a file; it does not perform the save direction.
Fix:
Use pickle.dump(object, file) to save an object.Using pickle.dump() while trying to restore an object.
pickle.dump() saves an object to a file; it does not perform the retrieval direction.
Fix:
Use pickle.load(file) to restore the stored object.Forgetting to close the file after dumping.
Forgetting to close files after dumping is identified as a common error.
Fix:
Complete the pickle operation and close the file afterward.Assuming del removes the pickled data.
del removes the variable from memory, while the pickled data persists independently in the file.
Fix:
Treat deletion of the variable and storage in the file as separate actions.
Check Your Understanding
You have a Python object named scores that must be saved to a file named scores.pkl and later restored into a variable named saved_scores. Write the two pickle operation calls and identify the correct binary mode for each file opening.
Hints
- Saving moves the object toward the file.
- Restoring moves stored data from the file into memory.
- Use wb for saving and rb for restoring.
Tracing the Direction
Which operation belongs in each blank: saving scores to scores.pkl or restoring saved_scores from scores.pkl?
Saving: Open scores.pkl in wb mode, then use pickle.dump(scores, file). The direction is from the Python object to the file.
Restoring: Open scores.pkl in rb mode, then use saved_scores = pickle.load(file). The direction is from the file back to a Python object.
Closing: Close the file after the dump or load operation has been completed.
Saving uses wb with pickle.dump(); restoring uses rb with pickle.load().
Key Takeaways
- pickle.dump(object, file) serializes a Python object into a byte stream and saves it to a file.
- pickle.load(file) retrieves stored pickle data and restores it into memory.
- Use binary write mode wb for dumping and binary read mode rb for loading.
- Deleting a variable with del removes it from memory but does not remove the data already stored in the file.
- Close the file after completing the pickle operation.
Key Takeaways
- pickle moves Python objects between memory and files by converting them to and from byte streams.
- Use pickle.dump() with wb to save and pickle.load() with rb to restore.
- A variable can be deleted after saving without removing the data from the file.
- Complete the dump or load operation before closing the file.