Systems-Level Concurrency in C++
Concurrency is the execution of several tasks simultaneously, and the source notes that all modern programming languages provide support for it.
Why a Program Gets Stuck Waiting
Imagine a program that must fetch information from somewhere far away. While it waits for that reply, the user still wants the rest of the screen to respond to clicks and typing. A program that can only do one thing at a time has a problem here. It must either stop everything and wait, or ignore the user until the reply arrives. Neither option feels good.
What do you think happens?
What happens to the user's clicks and typing while a one-at-a-time program waits for a reply?
Reveal answer
The program must either stop everything and wait, or ignore the user until the reply arrives.
A program that does only one thing at a time has no other part of itself that can keep working while one part is stuck waiting. Concurrency is one of the main ways programmers avoid this choice.
The idea this part of the course introduces is concurrency. As you read the definition in the next section, keep one question in mind: what would a program need to be able to do in order to keep working while part of it is stuck waiting?
Concurrency as Overlapping Tasks
Concurrency is the execution of several tasks simultaneously. The source treats it as a widely used capability rather than a rare special feature, and notes that all modern programming languages provide support for it.
The key word in that definition is simultaneously. Concurrency is about several tasks making progress during the same span of time. The diagram below contrasts two ways of handling the same work. In the first, the tasks run one after another, so each waits for the previous one to finish. In the second, the waiting task is set aside while another task keeps going.
In the top row, nothing else can happen while the program waits for the reply, so the clicks must wait too. In the bottom row, handling clicks overlaps with the waiting. The two rows contain the same steps; only the arrangement in time differs. That arrangement is what concurrency changes.
Tracing a Program That Runs One Step at a Time
Following a Single Sequence of Steps
A program must fetch information from a distant source, and the user keeps clicking during the wait. Trace what happens when the program has only one sequence of steps.
- 1Step 1: The request is sent
The program starts the fetch and reaches the point where it must wait for a response.
- 2Step 2: The sequence reaches the waiting step
Because there is only one sequence, the whole program is carried by that sequence. The program as a whole has nothing else it is able to do while this step waits.
- 3Step 3: The user clicks
The click arrives while the program is still waiting. The program can only stop everything and wait, or ignore the user until the reply arrives.
- 4Step 4: The reply arrives
Only now does the sequence move on, so the click is handled after the wait rather than during it.
With a single sequence, the waiting step blocks everything else. The user's input is delayed or ignored until the wait is over.
Notice what the trace reveals: the problem is not that the fetch is slow. The problem is that a single sequence has no way to let another part of the program progress while the first part is stuck. Concurrency addresses exactly that gap.
Two Reasons Programs Use Concurrency
The source gives these two reasons as separate goals. It helps to keep them apart, because a program can be designed around one of them without necessarily being designed around the other. The comparison below lays them side by side.
| Aspect | Improving throughput | Improving responsiveness |
|---|---|---|
| Main concern | How much total work is completed | How quickly the program reacts |
| How concurrency helps | Several processors work on a single computation | One part progresses while another waits for a response |
| Typical situation | A computation that can be split across processors | A program waiting on a distant reply while still needing to respond to the user |
The two reasons a program uses concurrency, as separate goals
What the C++ Standard Library Provides
Concurrency is supported by all modern programming languages, and C++ is one of them. The support the C++ standard library provides is described in the source as a portable and type-safe variant of concurrency support that has been used in C++ for more than 20 years. Portable means the facilities are meant to be usable across different environments rather than tied to one platform. Type-safe means the facilities are designed so that the compiler checks how values are used, which helps prevent a class of mistakes.
The source also notes that this kind of support is almost universally supported by modern hardware. The hardware programs run on generally offers what concurrency needs, so the standard library has real hardware to build on.
An important boundary is stated directly in the source. The standard library's support is primarily aimed at systems-level concurrency. It does not directly supply sophisticated higher-level concurrency models. Those models can be provided by libraries that are built using the standard library's facilities. The diagram below shows this layering, with the standard library as a foundation and richer models on top.
Mistakes Learners Make with Concurrency
Common mistake Treating concurrency as a rare feature that only some languages include
Why it failsThe source treats concurrency as widely used and states that all modern programming languages provide support for it.
Do this insteadThink of concurrency as a general capability that programmers across languages are expected to be able to reach for.
Common mistake Assuming throughput and responsiveness are the same goal
What it looks likeA program that splits one computation across processors is aiming at total work done, not necessarily at how quickly it reacts to a click.
Why it failsThe source gives these as two different reasons, and a program can be designed around one without being designed around the other.
Do this insteadAsk which goal a design is serving before judging whether it uses concurrency well.
Common mistake Expecting the standard library to supply every high-level concurrency model
Why it failsThe source states that the standard library does not directly supply sophisticated higher-level concurrency models.
Do this insteadExpect the standard library to provide the foundation, and look to libraries built on top of it for richer models.
Checking Your Understanding of the Two Goals
A program needs to finish a large computation as quickly as possible in total, and the computation can be divided among several processors. Which of the two reasons for concurrency does this scenario primarily illustrate, and why?
Hints
- Look for the phrase that describes what concurrency does for total work.
- Think about whether the goal is total work completed or how quickly the program reacts to input.
A program fetches data from a distant server while a user is still typing into a form. Explain in your own words how concurrency lets the typing keep working, and name the goal this illustrates.
Hints
- Consider which part of the program is waiting and which part is still progressing.
- The source names one specific goal for this situation.
Key Takeaways
- Concurrency is the execution of several tasks simultaneously, and all modern programming languages provide support for it.
- Programs use concurrency to improve throughput, by using several processors for a single computation, or to improve responsiveness, by letting one part progress while another waits for a response.
- A program limited to one sequence of steps must stop and wait, or ignore the user, while a slow step runs.
- The C++ standard library offers a portable and type-safe variant of concurrency support, primarily aimed at systems-level concurrency.
- Higher-level concurrency models are not supplied directly by the standard library and are instead built as libraries on top of its facilities.