Application Development
Application development is the work of closing the gap between a general engine and a specific job.
Why a Working Model Is Not Yet a Product
Once a model exists, it is used to produce outputs. That use is called inference. Inference optimization is the set of techniques that aim to make this use of a model better optimized. This article introduces the main techniques in that set so you know what the rest of the module covers before you go into the details.
The article then turns to the layer where people actually build things with a model. Models have become easy to get hold of, so almost anyone can now start using one to build something. The layer where this happens is called application development. Keep one question in mind as you read: if a model is already capable, what work is left for the developer? The rest of the article answers that question.
Inference Optimization as a Family of Techniques
Inference optimization techniques are techniques used to optimize inference. The overview names three of them: quantization, distillation, and parallelism.
The three techniques are introduced together as peers under one umbrella term. Each one aims at the same goal, which is making the use of a model better optimized, but each approaches that goal differently. This overview gives only the names and their role. It does not explain how each technique works internally.
Where This Overview Sits in the Course
The source states that all three techniques are discussed in Chapters 7 through 9. The detailed mechanics are found in those chapters rather than in this overview. Read this overview as a map: it tells you what each technique is called and where to find it, and it tells you that application development comes next.
The diagram below shows the structure of the topic as the source presents it. The three techniques sit beneath the umbrella term. Application development is shown as a separate topic that the source discusses after the inference optimization techniques. The source does not describe how application development depends on these techniques, so the diagram shows only the order in which the topics appear.
From a General Engine to a Specific Job
Application development is the process of using a model to build an application by supplying good prompts and the context the model needs.
A useful way to think about this is that the model supplies a capability, and the developer decides how that capability is aimed. A model on its own is a general-purpose engine that responds to whatever text it receives. An application is narrower. It has a particular job, particular users, and a particular shape of output. Application development is the work of closing the gap between a general engine and a specific job.
Tracing the Path From Model to User
The source describes how work moves through this layer. Prompts and context flow into the model. The model's output becomes the application's result. That result must then pass through evaluation and reach users through an interface. Evaluation and interface are not optional stops at the end of the process. They sit on the path between the model and the people who use the application.
What do you think happens?
A model's answer reads smoothly and sounds confident. Is that enough to trust it for the job?
Reveal answer
No, fluency gives little signal about accuracy, so the output must be checked
The source explains that a model can produce output that reads well and still be wrong for the task. Evaluation is the deliberate step of testing outputs against what the job requires, rather than trusting the first pleasing result.
Prompts and Context as the Two Inputs
Two inputs do most of the work in this layer. The first is the prompt, which is the instruction the model reads. The second is the context, which is the material the model needs in order to act on that instruction sensibly. The two are related but not the same, so each is examined separately below.
A Worked Trace Through a Support Assistant
Tracing one request through a support assistant
A software company builds an application that drafts a short summary of customer support tickets for human agents. Trace how the prompt, context, model output, evaluation, and interface fit together for one ticket.
- 1Write the prompt
The prompt states the task and the expected form of the output: summarize the customer's problem in two sentences and suggest one next step.
- 2Supply the context
The context is the ticket text and the relevant product notes, the background the instruction does not contain. Without it, the model cannot act on the instruction sensibly for this product.
- 3Receive the model output
The model returns a draft summary. This draft becomes the application's result, but it is not yet ready for users.
- 4Evaluate against the job
The draft is checked against what the job requires: does it match the ticket, and is the suggested step appropriate? A pleasing draft that misstates the problem fails this check.
- 5Present through the interface
The agent sees the summary beside the original ticket text, so they can judge the summary before acting on it. The interface is designed alongside the prompt and the evaluation, not added afterwards.
The application delivers a checked summary through an interface that lets the agent judge it. The model supplied the capability; the developer supplied the prompt, the context, the evaluation step, and the interface that together aim that capability at the support job.
Why Evaluation Is a Required Stage
The source states plainly that this layer requires rigorous evaluation, so outputs must be checked against the job before they are trusted. The reason is practical. A model can produce output that reads well and still be wrong for the task, and a fluent answer gives little signal about its accuracy.
Evaluation is the deliberate step of testing an application's outputs against what the job actually requires, rather than trusting the first pleasing result.
Why the Interface Is Designed Alongside the Rest
The last requirement the source names is that good applications demand good interfaces. The interface is the surface through which a person gives the application input and receives its results. A model may produce an excellent answer, yet the application still fails if that answer is hard to find, hard to act on, or presented without the information the user needs to judge it.
Design the interface alongside the prompts and evaluation rather than bolting it on after the model work is finished. The source gives this as the reason the interface belongs in the design from the start.
Mistakes Developers Make in This Layer
Common mistake Treating the model as a finished application
What it looks likeA team connects a model directly to users and ships whatever it returns.
Why it failsA model is a general-purpose engine. The application has a particular job, users, and output shape that the model does not know on its own.
Do this insteadSupply a clear prompt and the context the job needs, and treat the result as a draft to be checked.
Common mistake Trusting a fluent answer
What it looks likeA reviewer accepts a confident, well-written summary without comparing it to the source ticket.
Why it failsA model can produce output that reads well and is still wrong for the task.
Do this insteadBuild evaluation into the path between the model and users, testing outputs against what the job requires.
Common mistake Adding the interface at the end
What it looks likeThe team finishes the prompt and model work, then decides how users will see results.
Why it failsAn excellent answer still fails if it is hard to find, hard to act on, or shown without the information needed to judge it.
Do this insteadDesign the interface together with the prompts and the evaluation step from the beginning.
Common mistake Assuming the three named techniques are the only inference optimization techniques
Practice Tracing a Request
A teacher builds an application that drafts feedback on student essays. Identify which parts of the design are the prompt, which part is the context, which step checks the model's output before it reaches students, and which part determines how the feedback is shown to the teacher.
Hints
- The prompt is the instruction the model reads, including the expected form of the output.
- The context is the background material the model needs, such as the essay and the grading rubric.
- Evaluation tests outputs against what the job requires, before users rely on them.
- The interface is the surface through which the teacher gives input and receives results.
Key Takeaways
- Inference optimization techniques are techniques used to optimize inference, and this overview names quantization, distillation, and parallelism; the list is introduced with including, so it may not be complete.
- The detailed mechanics of the three techniques are in Chapters 7 through 9, and application development is the topic that follows them.
- Application development is the process of using a model to build an application by supplying good prompts and the context the model needs, closing the gap between a general engine and a specific job.
- Rigorous evaluation is required because fluent output can still be wrong, and good interfaces are required because a correct answer still fails if users cannot find, act on, or judge it.