Different Requirements for Data Analysis and Production Systems
Analytics teams often use Python and R because they allow quick testing of ideas, while production teams often use Go, Scala, C++, or Java to build robust, scalable systems.
From Promising Analysis to Running Service
Imagine a small analytics team that has spent a few weeks testing ideas for predicting customer churn, meaning which customers are likely to stop using a product. They have a model that performs well on historical data, and they built it quickly using Python and scikit-learn. The natural next question is how that work reaches the customers and systems that need it. The path from a promising analysis to a running service is not always straightforward, and this article explains why.
What do you think happens?
The churn model works well on the analytics team's laptops. What do you think is the most common way a company carries it into a production system?
Reveal answer
Reimplement the solution inside the production framework using a high-performance language
The source describes this as a relatively common response. The solution's idea is kept, but the code that carries it is written again in the language the production system already uses. The Python code is not moved across unchanged.
Two Teams, Two Jobs
In many companies the work is divided between two kinds of teams. Data analytics teams work with languages such as Python and R, which make it easy to test ideas quickly. Production teams work with languages such as Go, Scala, C++, and Java, which they use to build robust and scalable systems. The division is not accidental. It reflects two different jobs.
The source makes the central point directly: data analysis has different requirements from building live services, so using different languages for these tasks makes sense. An analyst cares most about how fast a question can be explored and changed. A production engineer cares most about whether a service stays reliable and can handle large volumes of work. The same model may interest both groups, but they judge it by different standards.
| Question | Analytics team | Production team |
|---|---|---|
| Typical languages | Python and R | Go, Scala, C++, and Java |
| Main priority | Testing ideas quickly | Robust and scalable systems |
| What they care about most | How fast a question can be explored and changed | Whether a service stays reliable under large volumes of work |
| How they judge a model | By how quickly it can be tested and revised | By whether it runs reliably as part of a live service |
Both teams may look at the same model, but they weigh it against different goals.
Tracing a Solution from Prototype to Service
The handoff between the two teams can be read as a sequence of stages. The analytics team does the exploratory work and arrives at a solution. That solution is not copied across unchanged. It is rebuilt in a high-performance language and placed inside the larger framework that the production team maintains. The handoff therefore happens at the level of the solution, not at the level of the original Python environment.
Reimplementing the Solution
Reimplementation keeps the idea behind the solution and writes the code that carries it again, in the language the production system already uses. The analytics team's Python code is not the thing that ships. What ships is a rebuilt component inside the larger framework.
The source gives a practical reason for this choice. Reimplementation can be easier than embedding a whole library or programming language and converting data formats between the two sides. It also avoids the work of converting data to and from the different formats that the two sides would otherwise have to share. In other words, the rebuilt component lives entirely in one language, so the production system never has to translate data across a language boundary.
Embedding Instead of Rewriting
There is an obvious alternative to reimplementation. Instead of rebuilding the solution, the team can embed the whole Python library, or the whole programming language, inside the production system. The production code then calls into Python while it runs. This keeps the original analytics code in use, but it means the two sides must share data, and that data has to be converted between the formats each side uses.
Read as a trade-off, reimplementation spends effort on rewriting the solution once, while embedding spends effort on sharing data across the boundary between the two languages. The source identifies the conversion work as the cost that reimplementation avoids. Whether that trade is worthwhile depends on the system at hand, which is why the source frames reimplementation as common rather than mandatory.
Walking a Churn Solution Through the Handoff
Moving the churn model into production
A small analytics team has a churn prediction model built with Python and scikit-learn that performs well on historical data. The production team wants the prediction to run inside their existing framework. Trace how the solution moves from the analytics environment to the running service using reimplementation.
- 1Explore
The analytics team tests ideas quickly in Python over a few weeks. This is the job their language is chosen for.
- 2Identify the solution
The team settles on a model that performs well on historical data. The idea of the solution is now clear, even though its Python code is not ready to ship.
- 3Reimplement
The production team, or the analytics team working with it, writes the solution again in a high-performance language the production system already uses. The idea is kept; the code is new.
- 4Place inside the framework
The rebuilt component is placed inside the larger framework the production team maintains. No Python runtime has to be added to the production system, and no data has to be converted across a language boundary.
- 5Run as a service
The component runs as part of the live service, judged by the production team's standard of reliability and capacity for large volumes of work.
The handoff happened at the level of the solution. The Python environment stayed with the analytics team, and the production system received a rebuilt component in its own language.
Mistakes When Moving Analytics Work to Production
Common mistake Assuming the analytics team's Python code can be deployed as it is
Why it failsThe source describes the common response as reimplementing the solution, not moving the Python code directly into the production environment.
Do this insteadTreat the analytics work as the source of an idea, then rebuild the solution in the language the production framework uses.
Common mistake Judging the model only by how fast it was built
Why it failsAnalysts and production engineers judge the same model by different standards. Speed of exploration matters to analysts, while reliability and scale matter to production.
Do this insteadCheck the model against the production team's requirements, not only the analytics team's.
Common mistake Concluding that Python has no place in production
Why it failsThe source notes that Python and scikit-learn are used in production systems in many organizations, including very large ones.
Do this insteadRecognize that Python can reach production, but integrating it can be difficult in complex infrastructure, which is why reimplementation is a common route.
Common mistake Treating reimplementation as the only correct choice
Why it failsThe source explicitly says reimplementation is not a universal rule.
Do this insteadCompare the effort of rewriting with the cost of embedding a library or language and converting data, then choose for the specific system.
Deciding Where a Solution Should Run
A small analytics team has a churn model that works well in Python and scikit-learn. Their company runs a large production system built in Java, and the production team has said it will not accept a Python runtime inside that system. Explain which approach the source describes as common here, what the handoff should carry, and one trade-off the production team would weigh against embedding.
Hints
- Look for the reimplementation response in the source and what it keeps versus what it rewrites.
- The source says embedding requires converting data between the two formats.
- Think about who judges the model and by what standard.
Key Takeaways on Team Requirements
- 1Analytics teams often use Python and R because they allow quick testing of ideas, while production teams often use Go, Scala, C++, or Java to build robust, scalable systems.
- 2Data analysis has different requirements from building live services, so the two kinds of work judge the same model by different standards.
- 3A common response is to reimplement the analytics solution inside the production framework using a high-performance language, keeping the idea but rewriting the code.
- 4The alternative is to embed a whole library or programming language, which requires converting data between the two sides. Reimplementation can be easier and avoids that conversion, though it is not a universal rule.
At a glance
- Analytics teams often use Python and R to test ideas quickly, while production teams often use Go, Scala, C++, or Java to build robust, scalable systems.
- Analysts and production engineers judge the same model by different standards: speed of exploration versus reliability and scale.
- A common route from analysis to production is reimplementing the solution in a high-performance language inside the production framework.
- Embedding a whole library or language is the alternative, but it requires converting data between formats. Reimplementation is not a universal rule.