Foundation Models and Emerging Applications

When you first learn that large language models are more capable than small ones, a natural guess follows: a more capable model should be able to learn from fewer examples. If the model is smarter, it surely needs less help from labeled data. This belief feels reasonable, and it shapes how many people estimate the cost of building an AI application. It is also wrong in the way that matters most for budgeting. This article explains why, then follows the layering idea that sits underneath many of today's AI products.

The mistake comes from what the comparison measures. One way to test a large model is to give it the same data a small model received and ask whether it does as well. That test is not the one builders usually care about. A builder wants the strongest performance the application can reach, so the relevant question is how much data the large model needs to reach its own best performance.

Seen this way, the answer changes. A large model has more capacity to absorb patterns, and feeding it only the small dataset leaves much of that capacity unused. Scaling up the model is therefore usually paired with scaling up the training data, so that the bigger model has enough examples to make use of its extra size. The diagram below traces the two questions side by side.

Labeling data is a service with a price, and that price is not fixed across projects. The actual cost depends on three main factors. The first is the complexity of the task, meaning how difficult it is for a labeler to make each judgment. The second is the scale of the dataset, where larger datasets typically come with a lower cost per sample. The third is the labeling service provider you choose, since providers set their own prices.

Notice that scale behaves differently from the other two factors. Complexity and provider choice change the price you pay for each sample. Scale tends to push the per-sample price down, but it also means you are buying more samples in total. Per-sample price and total budget are separate quantities, and a good estimate tracks both.

Amazon SageMaker Ground Truth offers a concrete illustration of volume pricing. As of September 2024, it charged 8 cents per image for labeling fewer than 50,000 images, and only 2 cents per image for labeling more than 1 million images. The price per image drops by a factor of four as the project moves from the smaller tier to the larger one. Prices change over time, so these figures are a snapshot of one provider at one date, not a permanent rate.

The per-sample cost is the quoted price for the tier your project falls into, because each image is one sample. The total is that price multiplied by the number of images. Both steps depend on choosing the correct tier first.

Many of the AI tools people use today look like independent products, each with its own interface and purpose. A useful question to ask about any of them is what it is actually running on. The source describes many existing applications as built on top of foundation models. That means applications are best understood as layers resting on a shared base. The foundation model is the common base, and each application is a layer that uses it for a particular purpose.

Imagine tracking a single foundation model and asking, for each product that uses it, where that product gets its capability. Pick three products that look quite different: a writing assistant, a customer support chatbot, and a tool that summarizes long documents. Each one is an application, and the question is what each one rests on.

The shape of the diagram is the point. Several applications depend on the same base, so improving or understanding that base affects all of them. Each application adds its own purpose and interface on top. When you read a new AI product, asking what it rests on tells you which parts are shared and which belong only to that product.

The source makes a forward-looking claim: there is no doubt that the number of applications will grow exponentially in the future. This is the author's prediction, and it should be read as a prediction. It describes the direction and shape of expected growth. It does not give a measured rate, a time frame, or a calculation.

The source connects this growth to the development process. It says that more systematic approaches to AI engineering will make development easier, and that this ease will enable even more applications. In other words, the number of applications is linked to how easily they can be built. Better engineering practice is presented as one of the conditions for growth. The chain is: a shared foundation model, applications built on it, systematic engineering that makes building easier, and therefore more applications.