Choosing Numeric Types for Computation

Imagine a small program that keeps a running count of the items it processes. Each time an item arrives, the program adds one to the count. Most of the time nothing interesting happens, because the count stays small. But suppose the program runs for a long time, or it processes a very large batch. At some point the count may reach a value the chosen numeric type cannot hold. Nothing crashes at that moment. The program simply keeps going with a value that no longer matches reality, and the error may only show up later, far from its cause.

This is the central problem of this article. Every numeric type a program uses has properties of its own, and those properties determine what values the type can hold and how it behaves. A type that works well in one place may be unsuitable in another. Choosing a numeric type without looking at its properties is a guess, and that guess can go wrong quietly.

Adequacy is a relationship between two things: a type and a requirement. The same type can be adequate for one program and inadequate for another. A type chosen for a small, bounded task is not automatically safe for a task whose values keep growing. This is why adequacy has to be judged against the specific job, not against a general feeling that a type is good.

Two ideas connect everything in this article. The first is access. The properties of numeric types are accessible through numeric_limits, so you do not need to know them from memory or assume them. The second is purpose. You use numeric_limits to check that the numeric types you rely on are adequate for their use. Access gives you the information, and checking turns that information into a decision.

When you query numeric_limits for a specific numeric type, you receive that type's properties. The properties named in this course's material include the minimum value, the maximum value, the number of digits the type can represent, and epsilon, which describes the gap between a value and the next representable value. Each type has its own entries, so the answer you get depends on which type you ask about.

Checking adequacy is a sequence, not a single glance. Work through the steps in order. Start by stating what values the task requires, including the largest value it will realistically reach. Next, query numeric_limits for the candidate type and read its properties. Then compare the type's maximum and minimum against the required range. Then compare the precision the type offers against the precision the task needs. If either comparison fails, choose a different type and run the checks again. If both pass, the type is adequate for this task. Because requirements change over time, return to the first step whenever they do.

Return to the running count. Suppose the count is stored in a type whose maximum is reached during processing. The value stored just before the overflow is still a correct reading of the count. The arithmetic result that goes past the maximum is no longer something the type can represent, so the variable no longer holds a correct count. The diagram below traces that change. The important point is that the variable gives no obvious warning at the moment it changes.