Prompt Design for Contract Review

Most people picture a software project as a plan written in advance and then carried out step by step. The contract review system described in this module grew in a different way. It began as a small working version, was shown to the people who would use it, and was then reshaped by what those people noticed. Its development story is less about a finished design and more about a loop: build something, watch it run, learn from the reactions, and build again.

This article follows that story stage by stage. You will look at the first prototype, the feedback that came out of a demonstration, the prompt revisions and new output fields that followed, and the way the tool was eventually connected to the firm's existing systems. At each stage, the question to keep in mind is the same: what did the system look like, and what triggered the next change?

The first version was deliberately modest. The source describes it as a prototype made of three Python files, put together over a single afternoon. Its job was not to be complete. Its job was to give the team something concrete enough to demonstrate, so that the questions that really matter could surface quickly.

The diagram below traces the progression at the level the source supports. Each arrow is a real step the source describes, and each box is the state the system was in afterward. Read it from left to right and notice what triggered each transition.

The debrief is the hinge of this progression. The source records a two-hour conversation after the demo, and that is where a library entry for the working version was written. The working version is referred to there as the slice. The debrief turned what the team observed into decisions about what to change next.

Notice that the later states are not separate products. The same system was modified in place. The later version still contained the earlier work, but it now did more, and each new capability was a response to something the team had learned.

The source describes development as a loop rather than a sequence with an end point. The team builds something, observes it running in front of its intended audience, learns from the reactions, and builds again. Each pass through the loop starts from the system as it currently exists, not from a blank page.

The diagram shows the loop as three stages with a return arrow. The return arrow is the important part. Feedback is not a report that ends the project; it is the input that determines what the next build should do. The source describes this same cycle when it summarises the approach as build, watch, learn from reactions, and build again.

During the demo, the deputy general counsel identified several patterns. The source names three. Each one describes a place where the system's behavior did not match what a careful lawyer would expect.

The source states that these patterns became prompt revisions during the following week. Revising a prompt is a lightweight kind of change. It alters how the system reasons about a document without requiring the surrounding code to be rebuilt. That is one reason an iterative team can respond to feedback quickly: the feedback can be turned into a changed instruction rather than a new program.

The output schema was extended to require a confidence field for each surfaced clause. Why would a firm want a number attached to each clause? The source answers directly: the firm used the confidence field to decide which memos to route to an associate for a faster review pass.

This is a practical use of a signal that an iterative process can produce. The first version of the system had no such field. Once the team could see which clauses the system was less sure about, it could build a routing decision around that information. The feedback loop therefore did not only fix errors. It created a new capability the firm could act on.

A review tool that works only on its own is of limited use to a firm. Within a month of the first demo, the slice had grown into a workflow connected to the firm's existing infrastructure. Integration changed what the system was for. Before these connections existed, a reviewer had to move work in and out of the tool by hand. Once the review run was wired to incoming email and outgoing case records, the tool became one stage in an existing process rather than a separate place to visit.

Read the diagram from left to right. The source names four connection points: the firm's email, where contracts arrive; the review run, which produces a redline and memo; the case management system, where the memo is posted; and the assigned attorney, who is notified. The connections are ordered in that sequence. The source does not describe the internal mechanics of any single connection, so the diagram shows the hand-offs and their order, not how each one is implemented.

The source presents this integration as part of the same iterative growth that produced the confidence field, which is why the two topics belong together in one story. Each new connection came after the team had observed the tool running and understood what the surrounding process needed from it.