Alternative Data Formats: JSON and CSV
Data persistence allows you to store Python objects in files so they survive program termination.
When Memory Is Not Enough
A Python object can exist in memory while a program is running, but that existence is temporary. When the program ends, the object is no longer available in memory. Data persistence solves this problem by storing a Python object in a file so it can survive program termination. Pickle is the mechanism described in this lesson for moving an object from temporary memory into persistent file storage and later bringing it back.
The Persistence Boundary
RAM and file storage play different roles. An object in RAM is fast to access and modify, but it is volatile: it disappears when the program ends or the computer loses power. A file on disk is persistent: it survives program termination and power loss. Pickle bridges these two forms of storage. It converts the object into data that can be written to a file, and the file can later be used to reconstruct the object in memory.
| Location | State of the data | What happens after program termination |
|---|---|---|
| RAM | Volatile | The object disappears from memory |
| File on disk | Persistent | The stored data remains available |
The source distinguishes temporary objects in RAM from persistent file storage.
Persistence does not mean that the original in-memory variable remains alive. It means that enough information has been stored in a file for the object to be reconstructed later.
Saving an Object with Pickle
Pickling is the saving stage. The pickle module analyzes the object's type, structure, and nested values, then serializes that information into a binary format. The resulting bytes are written to a file with pickle.dump(). The file must be opened in write binary mode for this operation.
- Import the pickle module.
- Create or identify the Python object to save.
- Open a file in write binary mode.
- Pass the object and the file to pickle.dump().
- Close the file after the object has been stored.
Restoring the Original Object
Unpickling is the retrieval stage. pickle.load() reads the bytes from a pickle file, decodes the binary data, and reconstructs the object in memory. The file must be opened in read binary mode. The restored object is identical to the original in type, structure, and values.
- Open the stored file in read binary mode.
- Pass the file to pickle.load().
- Allow pickle to read and decode the bytes.
- Receive the reconstructed Python object in memory.
The Shopping List Cycle
Saving, deleting, and restoring a shopping list
Follow what happens when a shopping list is stored in a file, removed from memory, and then retrieved.
Create: A shopping list is created as a Python list in memory.
Pickle: The file shoplist.data is opened in write binary mode, and pickle.dump() stores the list in it.
Remove: The shoplist variable is deleted, so the original variable no longer exists in memory.
Unpickle: The same file is opened in read binary mode, and pickle.load() retrieves the stored list.
Restore: The retrieved object is a list with the same type, structure, and values as the original.
The shopping list is recovered from the file even though the original shoplist variable was deleted.
What do you think happens?
After the shoplist variable is deleted, can the shopping list still be recovered?
Reveal answer
Answer: Yes, because the stored file contains the serialized object.
pickle.dump() stored the list in shoplist.data before the variable was deleted. pickle.load() later reads that file and reconstructs the list in memory.
Common Persistence Mistakes
Treating pickle.dump() as if it only writes a text version of the object
Pickling performs serialization. It analyzes the object's type, structure, and nested values and encodes them into a binary format.
Fix:
Think of pickle.dump() as converting the complete object into bytes for storage.Using the wrong file mode for saving
The source specifies write binary mode for pickling.
Fix:
Open the file in write binary mode before storing the object.Using the wrong file mode for loading
The source specifies read binary mode for unpickling.
Fix:
Open the file in read binary mode before retrieving the object.Believing that deleting the variable also deletes the saved data
The source example deletes the variable after saving and still restores the list from the file.
Fix:
Separate the in-memory variable from the persistent file that contains its serialized data.
A Reliable Save-and-Restore Routine
Keep the two halves of the process distinct. During saving, start with the object in memory, open a file in write binary mode, and use pickle.dump(). During restoration, start with the stored file, open it in read binary mode, and use pickle.load(). Closing the file after saving is part of the demonstrated cycle. This separation makes it easier to identify whether a problem occurred while writing the persistent representation or while reconstructing the object.
| Stage | Starting point | Operation | Result | File mode |
|---|---|---|---|---|
| Pickling | Python object in memory | pickle.dump() | Bytes stored in a file | Write binary |
| Unpickling | Bytes in a file | pickle.load() | Python object in memory | Read binary |
Check Your Understanding
A program creates a Python object, saves it to a file with pickle.dump(), ends, and starts again later. Describe the file mode, function, and data movement required in each stage.
Hints
- Saving and retrieving use different binary file modes.
- pickle.dump() moves an object toward a file.
- pickle.load() moves stored data back into memory.
Explain why deleting a variable does not prevent recovery when the object was already pickled to a file.
Hints
- Compare the variable in RAM with the serialized data on disk.
- Consider what pickle.load() reads.
Persistence in One View
- Data persistence stores Python objects in files so they survive program termination.
- RAM is volatile, while a file on disk is persistent.
- pickle.dump() serializes an object into bytes and writes those bytes in write binary mode.
- pickle.load() reads the bytes in read binary mode and reconstructs the original object.
- The reconstructed object has the same type, structure, and values as the original.
Key Takeaways
- Persistence allows data to survive after a program terminates.
- Pickling moves a Python object from volatile memory into a persistent binary file.
- Unpickling reads that file and reconstructs the object in memory.
- Saving uses pickle.dump() with write binary mode; retrieval uses pickle.load() with read binary mode.
- Deleting the original variable does not remove the serialized data already stored in the file.