Writing Objects to a File

Imagine a drawing program that saves its pictures to a file. Later, the same program is updated, and the new version may store its information a little differently than the old one did. Will the new version still open the files the old version saved? The honest answer is that it depends on how the data was represented in the file from the beginning.

This article looks at two problems that appear when a file is tied too closely to the exact layout a program uses today. It also looks at two design habits that reduce those problems: storing a version number inside the file, and choosing a format that a person can read and understand. Along the way you will trace what happens to the values in a file as the program changes, so the problems are visible rather than abstract.

A file written without a version number is a long sequence of values. Nothing in the file names each value. Its meaning comes entirely from its position: the reader takes the values in the same order the writer produced them. The reader is only correct if it assumes the same positions the writer used.

The example below is an invented illustration, not one from the source. Version 1 of a program saves two values in this order: a width at position 0 and a height at position 1. Version 2 adds a depth value and places it between them, so the layout becomes width, depth, height. Before you look at the diagram, decide what the old reader will see when it reads position 1.

The diagram shows the cause of the failure. Nothing in the file was damaged. The height value still exists at position 2, and the depth value sits exactly where the old reader looks for height. Only the assumed positions have changed, so the old reader quietly reads the wrong number. Because no error is raised, this kind of failure can go unnoticed for a long time.

A file that depends on exact program internals is fragile because its meaning is tied to a specific set of rules inside the program. The file itself does not say which rules were used. If those rules change, the same bytes can be interpreted in a new and incorrect way.

The source gives a concrete case. Suppose the program adds a new property to curves, so that it now expects an extra value for each curve. Every older file becomes unreadable, unless the user happens to remember which version of the program created that file. The data is not damaged in any physical sense. What has changed is the set of rules needed to interpret it.

A well-designed data file begins with a small header that describes the file itself. In the format discussed in the source, the first line names the program that created the file. This gives the program a quick test. When a user picks a file to open, the program can read the first word and confirm the file is the correct type before it tries to read anything else.

The same first line also carries a version number. In the source example it is 1.0. The version answers a different question from the program name. It asks which rules were used to write the rest of the file. Each time the layout changes, the number changes with it, and a higher number means a newer layout.

Follow the diagram for a file from a known version. The program confirms the type, reads the version, recognizes it, and reads the rest of the file using the rules for that version. Now follow a file from a newer version. The program still confirms the type and reads the version, but it does not recognize the version. The source says the program should explain that a newer version is needed rather than guess at the contents. A guess could produce the wrong values silently, which is exactly the failure shown earlier.

A human-readable format uses clear labels so that a person looking at the file can tell what each piece of data means. In the source's example, a line in the file begins with the word background and is followed by three numbers. Those numbers are the red, green, and blue components of the color. The label removes the guesswork, so the line explains itself.

The same idea appears in the curve data. Each curve is wrapped between the markers startcurve and endcurve. Inside those markers the file holds the curve's color, its symmetry property, and the xy-coordinates of each point. Because the structure is spelled out in words, a reader can follow the file from top to bottom without running a program.

Readability has practical value in three ways. Files in this format can be created or edited by hand, and the source notes that the example file was in fact written in a text editor rather than produced by the program. A readable file is easier to debug, because a missing marker or a misplaced number is visible on the page. And labeled data is easier to extend, since a new option can be added as a new labeled line without forcing every reader to memorize a fixed position.