Foundation Models and How They Are Adapted

Imagine a product team that wants to add an intelligent feature, such as summarizing customer emails or drafting replies. Until recently, most people would picture a group of machine learning researchers, a large pile of training data, and a long build process. Today, a single developer can often add that same feature in a few days. That shift is the reason a new discipline has appeared almost overnight. This article explains what that discipline is, why it has grown so quickly, and how its pieces fit together.

The key phrase in that definition is 'on top of.' An AI engineer is usually not building the underlying model from scratch. The engineer builds the layer that makes a foundation model useful for a particular purpose. A foundation model is general: it can be asked many kinds of questions and can produce many kinds of output. The application layer narrows that generality into a specific job, such as answering questions about a company's internal documents or classifying incoming support messages. The foundation model supplies the broad capability, and the AI engineer supplies the purpose, the inputs, and the way results are presented and checked.

Several terms are used for building applications on top of foundation models, including ML engineering, MLOps, AIOps, and LLMOps. The source explains why AI engineering is the preferred label. ML engineering was not adopted because working with foundation models differs from working with traditional machine learning models in several important ways, and the term ML engineering would not capture that difference. ML engineering remains a good umbrella term that covers both kinds of work.

Terms ending in 'Ops' were set aside for a related reason. Operational components do exist in this work, but the central focus is on tweaking, or engineering, foundation models so they do what you want. To check the choice, the author surveyed 20 people who were developing applications on top of foundation models and asked which term they would use. Most preferred AI engineering.

The source identifies three forces that drove the growth of AI engineering. The first is investment, the second is a low entrance barrier produced largely by the model as a service approach, and the third is the ability to build applications with minimal coding or in plain English. The three forces reinforce one another: investment funds the models and tools, the service model lowers the cost of using them, and the ease of working with them brings in more builders, which in turn attracts more investment.

The clearest way to see where an AI engineer works is to follow a single request. Trace it step by step through the diagram below, then read the walkthrough that follows it.

The request path shows a layered structure. The foundation model is the bottom layer, reached through a model API. Above it sits the application layer, which the AI engineer usually builds and manages. The layer the user touches is the presentation of the application, which displays the result. The diagram below stacks these pieces so you can see which parts sit inside the application and which sit behind the API.

This layering explains why AI engineering is more than calling an API. The model is general and open-ended, so the application layer decides what task the model performs, what inputs it receives, and how its outputs are presented and evaluated. Two applications built on the same model API can behave very differently because their application layers differ.

Because foundation models are general, the range of applications is very wide. The source notes that the number of potential applications seems endless and that it is impossible to list every use case. Categorizing them is also hard, because different surveys use different categories. Amazon Web Services has grouped enterprise generative AI use cases into three buckets: customer experience, employee productivity, and process optimization. Other groupings exist, such as eight categories including programming, data analysis, customer support, marketing copy, other copy, research, web design, and art. Some groupings are organized by value captured, such as cost reduction, process efficiency, growth, and accelerating innovation.

A single application can belong to more than one category. A bot might provide companionship and also aggregate information, and an application might extract structured data from a document and then answer questions about it. The source also notes that some categories look rare in open source projects simply because those applications are rarely released openly. Builders of education, data organization, and writing applications may find enterprise use more suitable.

Business motivation varies too. In a 2023 Gartner survey of 2,500 executives, 7% cited business continuity, meaning the risk that an organization could go out of business if it did not adopt generative AI, as their reason for embracing it. The source reports this as a single data point, so it shows that business continuity was a minority motive rather than the dominant one.

Not every promising use case is a good first project. The enterprise world generally prefers applications with lower risk. Companies deploy internal-facing applications, such as internal knowledge management, faster than external-facing applications, such as customer support chatbots. Internal applications let a company build its AI engineering expertise while limiting exposure to data privacy, compliance, and potentially catastrophic failures. The comparison below sets the two kinds of starting points side by side.

The source also observes that although foundation models are open-ended, many applications built on them are still close-ended, such as classification. Classification tasks are easier to evaluate, which makes their risks easier to estimate. When you design an application, asking how easily its output can be checked is a practical way to judge its risk.

Once the request path is clear, the engineer's role is easy to locate. The foundation model and its API are provided by someone else, and the engineer works in the application layer between the user and the model. That layer determines what the model is asked to do, how inputs are prepared, how outputs are shaped and checked, and what the user sees.