Planning AI Applications

Pause for a moment and imagine a product team that wants to add an intelligent feature, such as summarizing customer emails or drafting replies to them. Most people would picture a group of machine learning researchers, a large pile of training data, and a long build process before anything reached customers. Today a single developer can often add that same feature in a few days. That shift is the reason a new discipline has appeared almost overnight.

This article explains what that discipline is, why it has grown so quickly, and how its pieces fit together. The central question is simple: when a product uses a powerful AI model, which part of the system does the engineer actually build?

The key phrase in that definition is on top of. The AI engineer is usually not building the underlying model from scratch. Instead, the engineer builds the layer that makes a general-purpose model useful for a particular purpose. A foundation model is broad and open-ended; an AI application narrows it to one job, with defined inputs, a defined way of presenting results, and a way of judging whether the results are good.

The source names three forces that drove the rapid growth of the field. They are increased investment after the success of ChatGPT, a low entrance barrier enabled largely by the model as a service approach, and the ability to build applications with minimal coding or in plain English. The diagram below shows how these forces feed into the growth itself.

To see where an AI engineer works, follow a single request. The diagram below shows the path a user's input takes from the interface to the model and back again. Read it as a sequence: each arrow is one handoff, and the steps on the application side are where the engineer's work concentrates.

Notice where the foundation model sits. It is behind the API, and the application sits between the user and the model. The source's own description puts the engineer's work in the application layer between user and model, which is the part of the path the engineer controls.

The layering in the previous section explains why AI engineering is more than calling an API. The model is general and open-ended. The application layer decides which task the model performs, what inputs it receives, and how its outputs are presented and evaluated. The hierarchy below shows that the foundation model is the base on which the application is built.

Because foundation models are general, the range of possible applications is very wide. The source notes that the number of potential applications seems endless and that listing every use case is impossible. Categorizing them is also difficult, because different surveys use different schemes. Amazon Web Services, for example, groups enterprise generative AI use cases into three buckets: customer experience, employee productivity, and process optimization. Other groupings exist, such as eight categories that include programming, data analysis, customer support, marketing copy, research, web design, and art, and groupings by the value captured, such as cost reduction, process efficiency, growth, and accelerating innovation.

A single application can belong to more than one category. A companion bot might provide companionship and also aggregate information. An application might extract structured data from a document and then answer questions about it. The categories are therefore lenses for describing an application, not sealed boxes. The source also notes that some categories look rare in open source projects simply because those applications are rarely released openly, and that builders of education, data organization, and writing applications may find enterprise use more suitable.

Not every promising use case is a good first project. The enterprise world generally prefers applications with lower risk. The source reports that companies deploy internal-facing applications, such as internal knowledge management, faster than external-facing applications, such as customer support chatbots. Internal applications let a company build its AI engineering expertise while limiting exposure to data privacy, compliance, and potentially catastrophic failures.

The source also observes that although foundation models are open-ended, many applications built on them are still close-ended, such as classification. Classification tasks are easier to evaluate, which makes their risks easier to estimate. A practical way to judge an application's risk, then, is to ask how easily its output can be checked.