Model Build Versus Buy Decision

Imagine a team that has finished building a language model and now wants people to use it. Someone has to keep the model running on a machine, accept incoming questions, pass them to the model, and send the answers back. The team then has to decide who does this work: the team itself, or an outside company that already runs the model and sells access to it. That choice is the build versus buy decision.

Neither option is correct in every case. The right answer depends on the situation, and the situation can change over time. This article first defines the pieces involved, then follows how a query moves through them, and finally looks at the factors that tip the decision one way or the other.

A model cannot reach a user by itself. Some machine has to host it and keep it running. The service that does this hosting is called an inference service. It has three jobs: it receives queries from users, it runs the model to generate a response for each query, and it returns those responses to the users.

The interface that users actually interact with is called the model API. The model API is the front door, and the inference service is the workshop behind it where the model does its work. Keep these two roles separate in your head, because the build versus buy question is really about who runs the workshop.

A model API is not always run by the company that built the model. It can come from three kinds of provider. Model providers are companies that build the models themselves. Cloud service providers run large cloud platforms and offer model access as one service among many. Third-party API providers are companies that specialize in serving models, often built on top of models they did not create.

The same model can be available through several APIs, and those APIs may differ in features, constraints, and price. The source gives a concrete case: GPT-4 is available through both OpenAI and Azure APIs. Because the underlying model is the same, you might expect identical behavior everywhere. The source warns against that assumption. Different APIs may use different techniques to optimize the same model, which can lead to slight differences in its performance. For that reason, you should run thorough tests whenever you switch from one model API to another.

Openness also affects who can offer a model. Commercial models can only be accessed through APIs licensed by their developers. Open source models can be supported by any API provider, which gives you room to choose the provider that works best for you. Some companies follow a mixed strategy: Cohere and Mistral, for example, open source some of their models and provide APIs for others.

The source also points to an incentive effect. For commercial model providers, the models are their competitive advantage. For API providers that have no models of their own, the API is their competitive advantage. This means API providers may be more motivated to offer better APIs and better pricing. The source adds that building scalable inference services for larger models is nontrivial, which is one reason third-party inference and finetuning services have multiplied on top of open source models.

Data privacy asks where your information travels when it is used with a model. When you use an externally hosted model API, your queries are sent outside your organization. For companies with strict data privacy policies that forbid sending data outside the organization, externally hosted model APIs are ruled out entirely. Some countries also have laws that forbid sending certain data across their borders. A model API provider that wants to serve those customers would have to set up servers in those countries.

A real incident shows how this risk plays out. Samsung employees put proprietary Samsung information into ChatGPT, which leaked the company's secrets. It is unclear how Samsung discovered the leak or how the leaked information was later used. The incident was serious enough that Samsung banned ChatGPT in May 2023.

There is a second privacy channel beyond the act of sending data. If you use a model API, the provider could use your data to train its own models. Most model API providers say they do not, but their policies can change. In August 2023, Zoom faced a backlash after users discovered that the company had quietly changed its terms of service to let Zoom use service-generated data, including product usage and diagnostics data, to train its AI models.

Why does training on your data matter? Research in this area is still sparse, but some studies suggest that AI models can memorize their training samples. The source cites Hugging Face's StarCoder model, which was found to memorize 8% of its training set. Memorized samples can be accidentally leaked to users, or intentionally exploited by bad actors. So data that enters a training pipeline may later reappear in outputs.

Data lineage is about where the data used to train a model came from and who owns it. The source states plainly that for most models there is little transparency about what data a model was trained on. It gives an example: in Gemini's technical report, Google described the models' performance in detail but said nothing about the training data, beyond a statement that data enrichment workers are paid at least a local living wage. The source also notes that OpenAI's CTO could not give a satisfactory answer when asked what data was used to train its models.

Published training data does not settle the question either. Open publication of training data does not guarantee that anyone has verified it, which is one reason some companies prefer commercial models with protective contracts.

The legal status of AI-related intellectual property is also still changing. The source reports that the US Patent and Trademark Office made clear in 2024 that AI-assisted inventions are not categorically unpatentable, but that an AI application's patentability depends on whether the human contribution to an innovation is significant enough to qualify for a patent. It is also unclear whether, if a model was trained on copyrighted data, a product built with that model can have defensible IP. Because of this uncertainty, companies whose existence depends on their IP, such as gaming and movie studios, may hesitate to use AI in creating their products until the rules are clearer.

Lineage concerns can push a company in several directions: toward open source models, toward proprietary models, or away from both. The two directions are worth examining separately.

The source treats the answer to whether a team should host a model itself or use a model API as dependent on the use case, and it notes that the same use case can change over time. It lists seven axes to weigh: data privacy, data lineage, performance, functionality, costs, control, and on-device deployment. The first two, data privacy and data lineage, are developed in the most depth, which is why the sections above focus on them.

The source also notes that the answer can change as the use case changes. A choice that suits a prototype may not suit a product that handles sensitive records, and a choice that suits a regulated business today may need review if its policies or legal situation shift. For that reason, the comparison is best treated as ongoing rather than a one-time decision.