Evaluating Foundation Models

Pause and consider what a small team would once have needed to add an intelligent feature to a product, such as summarizing customer emails or drafting replies. Most people would picture a group of machine learning researchers, a large amount of training data, and a long build process. Today a single developer can often add that same feature in a few days. That shift is the reason a new discipline has appeared almost overnight. This article explains what the discipline is, why it has grown so quickly, and how its pieces fit together.

The key phrase in that definition is on top of. The AI engineer is usually not building the underlying model from scratch. The engineer builds the layer that makes a general-purpose model useful for a particular purpose. A foundation model is broad and open-ended, so the work of deciding what it should do, what it should receive, and how its answers should be presented falls to the engineer.

The clearest way to see where an AI engineer works is to follow a single request from start to finish. Before reading the steps, try predicting: when a user submits a question, which of the components in the path do you think the application developer builds, and which is the model that someone else may have built?

Notice where the foundation model sits. It is behind the API, and the application sits between the user and the model. The engineer typically works on the application layer: deciding how the request is packaged, what happens to the output, and how the result is evaluated before it reaches the user. The model itself is usually something the engineer consumes rather than builds.

The rapid growth of AI engineering is explained by three forces that reinforce one another. Each is described in the subsections below, and the diagram shows how they combine to produce growth in the discipline.

Several terms are used for building applications on top of foundation models, including ML engineering, MLOps, AIOps, and LLMOps. ML engineering was not adopted because working with foundation models differs from working with traditional machine learning models in several important ways, and the term would not capture that difference. ML engineering remains a good umbrella term that covers both kinds of work.

Terms ending in Ops were also set aside. Operational components do exist in this work, but the central focus is on tweaking, or engineering, foundation models so they do what you want. The author surveyed 20 people who were developing applications on top of foundation models and asked which term they would use. Most preferred AI engineering.

Because foundation models are general, the range of applications is very wide. The source notes that the number of potential applications seems endless and that listing every use case is impossible. Categorizing them is also hard, because different surveys use different categories. The table shows two of the groupings the source mentions.

A single application can belong to more than one category. A bot might provide companionship and also aggregate information, and an application might extract structured data from a document and then answer questions about it. The source also notes that some categories look rare in open source projects simply because those applications are rarely released openly. Builders of education, data organization, and writing applications may find enterprise use more suitable.

On business motivation, the source reports one data point: of 2,500 executives surveyed by Gartner in 2023, 7 percent cited business continuity, meaning the risk that an organization could go out of business if it did not adopt generative AI, as their reason for embracing it.

Not every promising use case is a good first project. The enterprise world generally prefers applications with lower risks. Companies deploy internal-facing applications, such as internal knowledge management, faster than external-facing applications, such as customer support chatbots. Internal applications let a company build its AI engineering expertise while limiting exposure to data privacy, compliance, and potentially catastrophic failures.

The source also observes that although foundation models are open-ended, many applications built on them are still close-ended, such as classification. Classification tasks are easier to evaluate, which makes their risks easier to estimate. When designing an application, asking how easily its output can be checked is a practical way to judge its risk.