Diagnosing Genuine AI Engineering Engagements

Picture an AI engineering engagement a few weeks in. Meetings are on the calendar, status updates go out, and documents and artifacts keep appearing. From the inside, the work feels productive. The source for this article makes a pointed observation: most engagements in AI engineering do not announce that they are going badly. Work continues, meetings happen, and artifacts get produced, so an engagement can feel productive even when it is not.

The problem is not that the warning signs are hidden in some dramatic way. The problem is that the normal activity of an engagement masks them. This is the situation the cargo-cult engagement diagnostic is meant to address. Its purpose is to help a practitioner notice this kind of trouble while there is still time to act on it, rather than discovering the failure only after it has become impossible to recover from.

The key to the whole tool is the phrase from the inside. An outside reviewer may bring distance, but the diagnostic is written for the person in the middle of the engagement. That person needs a way to check their own situation without waiting for a retrospective to reveal the problem.

The word pattern does a lot of work here. The source explains that the recognizable pattern of a cargo-cult engagement is consistent across the engagements its author has seen go wrong. Because the pattern repeats, it does not have to be rediscovered from scratch each time. It can be reduced to a short list of probes.

That compression is the reason the diagnostic has five questions rather than a long checklist. The source describes the diagnostic as built around a small set of questions. A short set is easier to run consistently, and it asks the practitioner to look at the engagement as a whole rather than auditing every corner of it.

Timing is part of the diagnostic itself. The source states that the five questions are run on every engagement at the four-week mark. Read the timeline below as the single path the source describes: the engagement runs for some time, reaches the four-week checkpoint, and then the questions are applied. Notice where the checkpoint sits relative to the rest of the work.

The word every matters. The diagnostic is not reserved for engagements that already look suspicious. It is a routine checkpoint applied across the board. This is what makes it useful: trouble that is hard to see early can still surface at a predictable point, without anyone needing to first feel uneasy about the work.

The source gives a clear rule for interpretation: answers that come back wrong point toward the cargo-cult pattern, and they should be read as a whole rather than one question at a time. This matters because no single answer is meant to be a verdict. Each question contributes to a picture, and the picture is what carries the signal.

The flowchart below traces the decision the practitioner faces once the questions have been applied. The path splits on how the answers, taken together, read. An engagement whose answers keep it on the genuine path moves in one direction. An engagement whose answers come back wrong is pointed toward the cargo-cult pattern. The source notes that the composite illustration of the wrong-answer outcome is not reproduced in the available material, so the outcome side is described only at the level the source supports.

A short set of questions can work as a recognition tool because the failure it targets has a consistent shape. The source explains that the author has seen the same pattern across engagements that went wrong, and the diagnostic compresses that pattern into five probes. Each probe is a way of looking at the engagement from a slightly different angle, and the source's instruction to read the answers together means the angles reinforce one another.

Think of it this way: a single surprising answer might have an ordinary explanation. A set of answers that all point the same direction is much harder to explain away. That is why the diagnostic asks for a whole-picture reading. The recognition happens when the pattern becomes visible across the set, not when any one answer is examined alone.