Matching Patterns with regex_search
The pattern used by the program is written as a raw string literal.
Scanning Postal Codes Line by Line
Many searches need more than a yes-or-no answer. Suppose you want to scan a file for U.S. postal codes and report each one you find. A program for this task reads the file one line at a time, searches each line with a regular expression, and prints the matched text. When a search succeeds, it prints a line number and the complete match. It may also print a second, indented line containing a smaller piece of that match.
That smaller piece is called a submatch. This article explains where submatches come from, how the program reaches them, and how to reason about the output when a submatch is present or absent. Read the program first and try to predict its output, then compare your prediction with what the code actually does.
A regular expression search reports two levels of detail. The first level is the complete match, the entire stretch of text that the pattern matched. The second level is a submatch, a piece of that stretch that corresponds to one part of the pattern. The complete match always comes first in the results, and submatches follow it in the order their parts appear in the pattern.
Reading the Pattern Piece by Piece
The pattern used by the program is written as a raw string literal. Raw string literals let you write backslash sequences such as \w and \d directly, without doubling every backslash. The pattern has four parts, and the program's search depends on which of them are wrapped in parentheses.
The first three tokens always have to match. The last token is the only parenthesized part, so it is the only source of a submatch. The question mark after the closing parenthesis makes that group optional, so the pattern matches with or without the four-digit extension. Because the parentheses wrap the hyphen as well as the digits, the submatch text includes the hyphen. This detail matters when you predict output.
How Parentheses Create a Submatch
A submatch is the text captured by a parenthesized part of a regular expression. The entry matches[0] holds the complete match, and each parenthesized group after it fills the next index in order of appearance in the pattern.
An smatch object is the container that regex_search writes into. Its contents are valid only after a successful search. You never read matches before checking the result of regex_search, because a failed search leaves the container without the results you would expect.
Read the diagram as a rule. The complete match always occupies matches[0], covering everything the pattern matched, including the unwrapped tokens. Only the parenthesized group produces the entry at matches[1]. The unwrapped tokens still count toward the complete match, but they do not get their own submatch slot.
Tracing a Line Without the Extension
The following program is a small generated example that follows the same pattern as the source program. It searches one line and prints the complete match. It prints a submatch only when the submatch took part in the match.
#include <regex>
#include <iostream>
#include <string>
int main() {
std::regex pattern(R"(\w{2}\s*\d{5}(-\d{4})?)");
std::string line = "TX77845";
std::smatch matches;
if (std::regex_search(line, matches, pattern)) {
std::cout << matches[0] << "\n";
if (1 < matches.size() && matches[1].matched) {
std::cout << " " << matches[1] << "\n";
}
}
}- 1The pattern is built from the raw string literal, so the backslashes reach the regex engine unchanged.
- 2
regex_searchscans TX77845. The first two word characters are TX, the optional whitespace matches nothing, and the five digits 77845 match. - 3The optional group is absent, so the match is only TX77845 and matches[1].matched is false.
- 4The if statement first checks 1 <
matches.size(), then checks matches[1].matched. The second check is false, so the submatch line is skipped.
regex_searchsucceeds, so the results in matches are valid- matches[0] is printed as the complete match
- The guard check fails for matches[1] because the optional group did not take part
TX77845Tracing TX77845 through the guard
What does the program print for the line TX77845?
- 1Match the required tokens
TX matches the two word characters, and 77845 matches the five digits. Nothing is needed for the optional spaces.
- 2Try the optional group
The line has no hyphen after the digits, so the group does not take part. The pattern still succeeds because the group is optional.
- 3Record the complete match
The complete match is TX77845, which is stored in matches[0].
- 4Apply the guard
1 <
matches.size()is checked first, and then matches[1].matched is checked. The second check is false, so the submatch is not printed.
Only TX77845 is printed, with no indented submatch line.
Tracing a Line With the Extension
What do you think happens?
For the line TX77845-1234, what does the indented submatch line print?
Reveal answer
-1234
The optional group is parenthesized, and the parentheses wrap the hyphen as well as the digits. So matches[1] holds -1234, including the hyphen. The guard check passes because matches[1].matched is true.
The two lines differ in one place. Without the extension, matches[1] exists as a slot, but its matched flag is false, so only the complete match is printed. With the extension, the same slot holds -1234 and its matched flag is true, so the program prints the complete match followed by the indented submatch. The pattern and the search are identical in both cases; only the input text changes which result the optional group produces.
Guard Checks Before Printing a Submatch
The source program checks two things before it reads a submatch. The first part, 1 < matches.size(), confirms that the container holds a submatch slot before the program reads matches[1]. The second part, matches[1].matched, confirms that the submatch took part in this particular match. Both parts are needed: the first protects the index, and the second avoids printing a submatch that did not match.
Tracing an Unexpected Submatch
When an output line surprises you, find the exact token that produced it. For a submatch line, the source of the text is the parenthesized group. If the printed text is wrong, look at what is inside the parentheses. If the line is missing, look at the matched check, because that is where the program decides whether to print.
Mistakes Beginners Make with Submatches
Common mistake Expecting matches[1] to hold the whole postal code
What it looks likeReading matches[1] for TX77845 and expecting TX77845
Why it failsOnly the parenthesized group creates a submatch. The whole postal code is in matches[0].
Do this insteadUse matches[0] for the complete match and matches[1] only for the text inside the parentheses.
Common mistake Forgetting that the hyphen belongs to the submatch
What it looks likeExpecting the submatch for TX77845-1234 to be 1234
Why it failsThe parentheses wrap the hyphen as well as the digits, so the captured text includes the hyphen.
Do this insteadPredict -1234, including the hyphen, when the optional group participates.
Common mistake Reading the submatch before checking the search result
Why it failsThe smatch object is valid only after a successful search, so reading it after a failed search gives meaningless results.
Do this insteadCheck the return value of regex_search before reading any entry in matches.
Common mistake Printing matches[1] without checking matched
Why it failsAn optional group may not take part in a match, so its text is not a real submatch for that line.
Do this insteadCheck matches[1].matched, and check 1 < matches.size() first to protect the index.
Practicing Submatch Prediction
The program prints only one line for the input TX77845 and no indented line. Explain which check in the program prevents the submatch line, and name the value of matches[1].matched that causes it.
Hints
- Look at which part of the guard is responsible for deciding whether to print.
- Think about whether the optional group took part in the match for TX77845.
Key Takeaways
- A submatch is the text captured by a parenthesized part of a regular expression, while matches[0] always holds the complete match.
- Submatches appear in matches in the order their parenthesized parts appear in the pattern, and the captured text includes everything inside the parentheses, including literal characters such as a hyphen.
- An optional group may not take part in a match, so check matches[1].matched and 1 <
matches.size()before printing a submatch. - When a submatch looks wrong, trace the text back to the parenthesized group, and when a submatch line is missing, look at the matched check.