Data Quality and Production Data Access

Imagine a team preparing to deliver a data system to a customer. The customer has handed over a sample export, several documents describing the fields, and a verbal summary of how the business runs. The team reads through the sample, finds it tidy, and begins designing around what they see. Weeks later, the system meets the real data and behaves in ways nobody planned for. The source describes this pattern as one of the most common failures in FDE engagements, and it usually starts with a quiet assumption made early in the project.

This article names that assumption, explains why it is wrong, and walks through the three problem types the source identifies: gaps between what was described and what exists, schemas that disagree, and formats that do not match across records. It then shows how these problems turn into failed systems and unplanned work, and what to do about it.

The misconception at the center of this article is the assumption that a data sample is representative of production data. The reasoning feels sensible. A sample comes from the same source, it has the same columns, and it looks well-behaved when inspected. So it seems reasonable to treat whatever is true of the sample as true of the full system.

The source states the gap directly: the data in the sample is clean, but the data in production is not. A sample can be clean for reasons that have nothing to do with the production data it was meant to represent. It may have been assembled by hand, filtered by someone who wanted it to look right, or pulled from a narrow slice of the business. Whatever the reason, a clean sample is evidence about the sample. It is not automatically evidence about everything else.

Follow the path a wrong assumption takes. The customer's documents describe the fields, the sample shows what those fields look like in practice, and the team designs the system from both. Nothing in that path touches the production data itself. The first time the built system meets production data is the first time the assumption is actually tested, and by then the design is already in place.

In the diagram, the three inputs on the left all feed one design decision. The wrong assumption enters at that decision point, because the team treats the sample and documents as a description of production. The built system is then built on that description. When it reaches production data, the mismatch surfaces at the point where the system receives real records, not at the point where the design was made.

Once the sample assumption is set aside, the problems become easier to sort. The source groups the common issues into three kinds. Each one is a place where the data the customer described and the data the customer actually holds have drifted apart. The three problem types share a single root: the expectation built from documentation and samples diverges from what the production data contains.

The source notes that a system built around the documented structure can fail on the actual data, and that the bridging work needed to close the gap is often absent from the original estimate. The sequence runs in a predictable order. The team designs from the documents and sample. The system meets production data. Failures appear that the design did not account for. Someone must diagnose what the data actually contains, then rework the system to handle it. The diagnosis and rework were not in the plan, so they arrive as added effort.

The added effort is not a sign of carelessness in the design. It is the cost of an assumption that was never tested. Recognizing that the cost comes from the data rather than the code helps a team plan for it at the start, by allowing time to examine production data before committing to a design.