Audit Trails in Quality Management

A high completeness score only shows that required fields are present; it does not show that the values are correct.

Programming

A Report That Looks Finished

Imagine a team reviewing the first results of an automated inspection reporting model. The model turns the audio recording of an inspector into a structured record. The first numbers look encouraging: nearly every required field is filled in. It is tempting to read that result as a sign the model is ready. This section and the ones that follow explain why that conclusion is premature, and how a team in an aerospace quality setting found a remedy that made the record trustworthy enough to pass an audit.

What do you think happens?

If every required field in an inspection record is filled in, is the record guaranteed to be accurate?

Yes, a complete record is an accurate record
No, presence of a field does not show that its value is correct

Reveal answer

Answer:

No, presence of a field does not show that its value is correct

A high completeness score only shows that required fields are present. It does not show that the values are correct.

Why Completeness Feels Like Quality

The belief at the center of this topic is intuitive: if every field in a report is present, the report is good. Completeness feels like a proxy for quality, because a gap in a form is visible at a glance while a wrong value often is not. A checklist with every box ticked looks finished. The trap is that the checklist only tells you that something was written in each box, not whether what was written is true.

Key point

A field being present says nothing about where its value came from. A reviewer reading the finished report cannot easily tell a value that was spoken from one that was guessed.

Present Versus Correct

The gap between present and correct is where the failure lived. The evaluation treated completeness and accuracy as separate questions. Completeness asks whether the required fields are filled in. Accuracy asks whether the values match the inspection. The diagram below places the two results from the first model run side by side.

How can every required field be filled in and the record still be wrong? Completeness and accuracy measure different things, so one can be high while the other is low.

The two columns above were measured separately. The model cleared the completeness criterion at ninety-six percent, which means it was usually filling in the required fields. It missed the accuracy criterion at eighty-one percent. A single headline number would have hidden that difference, which is why the team kept the criteria apart.

The Three Validation Criteria

The team evaluated the model against a rubric with three criteria. The evaluation set was forty real inspections, each labeled by the customer's quality manager. The criteria in this source are framed as three different questions: structural completeness, factual accuracy, and audit-readiness. Because they ask different questions, a model can do well on one and poorly on another.

How do the three criteria relate to one another, and what does each one check that the others do not? Completeness checks presence, accuracy checks values, and audit-readiness checks whether a reviewer can trust the record.

Read the arrows as a chain of questions rather than a ranking. A record can pass the first question and fail the second. Audit-readiness is the question the customer cares about most in the end, and the source shows that it moved with accuracy rather than with completeness: the remedy did not raise completeness, yet it raised audit-readiness to one hundred percent.

Tracing the First Model Run

Worked example

Reading the first evaluation results

The first model run scored 96% on completeness and 81% on accuracy. Decide what each number tells the team, and whether the record is ready for audit.

1
Read the completeness score
96% means the model was usually filling in the required fields. It says nothing about whether those values are right.
2
Read the accuracy score
81% means nearly one in five records had values that did not match the inspection on this criterion. The gap is the important signal.
3
Find the cause
The quality manager traced the gap to a specific behavior: the model was inferring fields the inspector had not said aloud.
4
Judge the audit risk
An inferred value can look like a legitimate entry while having no basis in the inspection. A reviewer cannot easily spot it, so the record is not audit-ready.

Result

The high completeness score did not make the record trustworthy. The accuracy gap, caused by inference, was the real problem.

Mistakes When Reading a Completeness Score

Common mistake
Treating a high completeness score as proof of accuracy
What it looks like
A report scores 96% on completeness, so the team signs it off without checking values.
Why it fails
Completeness only shows that required fields are present. It does not show that the values are correct.
Do this instead
Evaluate accuracy separately, against the source inspection, before judging readiness.
Common mistake
Assuming a filled field was spoken
What it looks like
A reviewer sees a value in a field and assumes the inspector said it aloud.
Why it fails
A field being present says nothing about where its value came from. The model may have inferred it.
Do this instead
Require that every value can be traced to something the inspector actually said.
Common mistake
Collapsing the three criteria into one number
Why it fails
Each criterion asks a different question, so a single score can hide a failure on another criterion.

Constraining the Model to Spoken Fields

The remedy was explicit and simple to state. The model could only fill fields the inspector had spoken. Any field the inspector did not address had to remain missing, and it had to be flagged so that the inspector could complete it manually. The model was no longer allowed to fill gaps with its own inference. The diagram below shows how one unspoken field changes between the two behaviors, while a spoken field is unaffected.

What changes in the record when the model is restricted to spoken fields, and which fields become empty as a result? Only the field the inspector never addressed changes: it moves from an inferred value to an empty, flagged field.

The source does not list which specific fields were affected, so the diagram shows a generic unspoken field rather than a named one. What the source does state is the effect on the scores. The constraint lowered the completeness score, because fields the model would previously have guessed now stayed empty. It raised the audit-readiness score to one hundred percent. The customer signed the validation milestone after this result.

Checking a Report Before Sign-Off

MEDIUM

A generated inspection record has a completeness score of 100%. A reviewer notices that one required field contains a value the inspector never mentioned in the recording. Explain whether the 100% completeness score is enough to make the record audit-ready, and describe what you would check next.

Hints

Ask which of the three criteria the 100% score actually measures.
Think about where the unmentioned value came from, and which criterion it would fail.
Consider whether the value should have been left empty and flagged under the spoken-fields constraint.

Your attempt (not saved)

Key Takeaways

A high completeness score only shows that required fields are present; it does not show that the values are correct.
The three criteria ask different questions: structural completeness, factual accuracy, and audit-readiness. A model can pass one and fail another.
In the first run, completeness reached 96% while accuracy reached 81%, and the gap traced to the model inferring fields the inspector had not said aloud.
Restricting the model to spoken fields, and flagging the rest for manual completion, lowered completeness but raised audit-readiness to 100%, at the cost of leaving some fields empty.