Production System Considerations
Production work is about keeping a live service running reliably and handling growing demand.
From Promising Result to Live Service
Many machine learning projects begin in a notebook or a short script. Someone loads data, tries a few approaches, and finds a model that performs well on the examples at hand. At that point it is tempting to assume the next step is simple: put the same code on a server and let users call it. The source material for this article explains why that assumption often fails. The choice of programming language is one of the places where the gap between an analytics result and a production service shows up most clearly.
Consider what happens after an analytics team has found something that works. The source describes a relatively common response: the solution is reimplemented inside the larger framework that the company already runs, using a high-performance language. The analytics work is not thrown away. Its logic is carried over into the language that the production system is built on.
Two Kinds of Work
The source separates two kinds of work. Analytics work is about exploring: trying an idea, checking whether it holds up, and changing direction quickly when it does not. Production work is about keeping a live service running reliably and handling growing demand. Because these goals pull in different directions, the tools that suit one are not always the tools that suit the other.
Analytics teams often use Python and R to test ideas quickly, while production teams use Go, Scala, C++, and Java to build robust, scalable systems.
The difference in languages follows from the difference in goals. A language that makes it quick to try an idea and revise it is valuable during exploration. A language suited to a service that must stay up and absorb rising demand is valuable once the idea has earned a place in a live system. The source states that using different languages for these two tasks makes sense.
Tracing the Handoff
The handoff is the point where a working analytics solution leaves the analytics team and enters the production environment. The source describes the sequence in which the analytics team finds the solution and the production team carries that solution into a high-performance language inside the existing framework. Follow the solution through each stage and notice who owns it at each step.
Handoff from analytics team to production team
The output of this handoff is a component of a live system rather than a notebook. The analytics work is the starting point, but the thing that reaches users is a piece of the production framework written in its language.
Reimplementing or Embedding
There are two broad ways to bring an analytics solution into a production environment written in another language. One option is to embed the whole library or programming language that the analytics solution depends on. The other is to reimplement the solution itself in a high-performance language. The source states that reimplementation can be easier than embedding, because embedding requires converting data between formats when moving from and to the different environments.
The before side of the comparison shows the cost the source points to: a dependency on an external library or language, plus a conversion step each time data crosses the boundary. The reimplemented side removes that foreign dependency by rewriting the logic, so the solution is written in the same language as the framework it serves. The embedded side keeps the original logic but carries the external environment with it.
Where the Solution Sits in the Framework
A reimplemented solution does not float free. It is placed inside the larger framework the company already runs, and that framework is written in a high-performance language. The diagram shows the containment relationship. The solution lives inside the framework, and the framework is what keeps the service running.
Where the solution sits within the production system
Following One Model Through Production
Tracing a scoring model from notebook to service
An analytics team has a notebook in which a model scores incoming records and performs well on the examples it was tested on. The company's live service is built in a high-performance language inside an existing framework. Trace what happens to the solution, and decide whether reimplementing or embedding is the better fit for this description.
- 1Stage 1: exploration
The analytics team loads data, tries approaches, and finds a model that performs well on the examples at hand. The work is about testing an idea, which is the exploratory goal the source describes.
- 2Stage 2: handoff
The working solution moves toward the production team. The analytics logic is not discarded; it is carried over into the production environment.
- 3Stage 3: choose an approach
Because the framework is written in a high-performance language, the production team weighs two options. Embedding would keep the Python library and require converting data each time it crosses the boundary. Reimplementing would rewrite the logic in the framework's language.
- 4Stage 4: result
The source states that reimplementation can be easier than embedding because embedding requires format conversion. Under this description, reimplementation is the common response, and the output is a component of the live system rather than a notebook.
The solution's logic survives the handoff, but it is most often reimplemented in the framework's language so that it becomes part of the live service.
Mistakes About Python in Production
What do you think happens?
Does the source conclude that Python has no place in production systems?
Reveal answer
No, Python is used in production in many organizations
The source is careful not to turn the handoff pattern into a rule against Python. It states that Python and scikit-learn are used in production systems in many organizations, including international banks and global social media companies.
Common mistake Assuming the notebook code can simply be copied onto a server
Why it failsThe source explains that this assumption often fails, because analytics work and live service work have different requirements.
Do this insteadTreat the analytics result as the starting point of a handoff, and plan how the logic will enter the production environment.
Common mistake Believing that Python has no place in production
Why it failsThe source says Python and scikit-learn are used in production in many organizations.
Do this insteadRecognize that the question is whether a particular production framework can include Python, which depends on the company's infrastructure.
Common mistake Treating a difficult integration as a failure
Why it failsThe source notes that many companies have complex infrastructure, and that including Python is not always easy. It treats the use of different languages for different kinds of work as a reasonable situation.
Do this insteadAccept that the two kinds of work may legitimately use different languages, and choose between reimplementing and embedding on that basis.
Common mistake Assuming embedding is always the simpler option
Why it failsThe source states that reimplementation can be easier than embedding, because embedding requires converting data between formats.
Do this insteadCompare the cost of carrying a foreign dependency and converting data at every boundary against the cost of rewriting the logic.
Practice Choosing an Approach
A team has a Python scoring model that works well in analysis. Their company runs a high-performance language framework and wants the model available to live users. Before reading the answer, decide which two options the team should compare, and state what each option costs in the terms the source uses: what gets rewritten, what stays, and where data must be converted.
Hints
- The source names two broad options: embedding the whole library or language, or reimplementing the solution in the high-performance language.
- Think about which option requires converting data between formats each time it crosses the boundary between environments.
Key Takeaways
- Analytics work is about exploring ideas quickly, while production work is about keeping a live service running reliably and handling growing demand, so the two may use different languages.
- A common handoff pattern has the analytics team find a working solution and the production team carry its logic into a high-performance language inside the existing framework.
- Reimplementing a solution can be easier than embedding a whole library or language, because embedding requires converting data between formats at each boundary.
- Python is used in production systems in many organizations; the real constraint is that complex infrastructure can make including Python difficult, which is a reasonable reason to use different languages.