Regular Expression Patterns

Searching text for a pattern is one of the most common jobs a program does. A postal code, an order number, or an identifier can all be described by a pattern rather than by a fixed list of strings. To work with such a pattern, a program needs two things: a way to test whether a line of text contains the pattern, and a way to read back exactly what was matched. In C++, both jobs are handled by the function regex_search together with an object of type smatch.

Regular expressions describe the shape of text in a short, terse notation. Instead of listing every string you want to find, you write one pattern, and a search tool looks for text with that shape. The standard library supports this notation through the std::regex class and its supporting functions, and the rest of this article builds up a single working example: a pattern that recognizes a U.S. postal code of the form XXddddd-dddd and its variants.

A pattern is read from left to right, and each piece describes a stretch of text that the match must cover. The postal code pattern has four pieces. The first piece requires two word characters, which the source describes as two letters. The second piece allows any amount of whitespace, including none. The third piece requires exactly five digits. The last piece is a group that is optional: it allows a dash followed by four digits, or nothing at all.

Keep one question in mind while reading any pattern: what does this pattern accept, and at what point would a candidate string stop matching? The pattern is itself just a string. The std::regex object takes that string and turns it into a pattern that can be used for matching. Each quantifier controls how many times a piece may repeat, and the question mark makes a piece optional.

The call regex_search(line, matches, pat) has three parts, and each has a distinct job. The first argument, line, is the text to be searched. The third argument, pat, holds the regular expression, which is the description of what to look for. The second argument, matches, is where the results are stored if a match is found.

The function searches the line for anything that matches the pattern stored in pat. If it finds a match, it stores the match information in matches. If no match is found, the function returns false. This return value is what makes an if statement work: the body runs only when the search has succeeded. The flow below shows how a single line travels from the call to the decision about printing.

The matches variable has the type smatch. The letter s stands for sub or string, and an smatch is a vector of sub-matches, where each element has the type string. A vector is an ordered collection, so each sub-match can be reached by its position. This is why the code writes matches[0] to access one element and matches.size() to ask how many elements there are.

The first element, matches[0], is the complete match: the full piece of text that the pattern matched. The elements after it are sub-matches, which are the parts captured by parentheses in the pattern. In the postal code pattern, the parenthesized group is optional, so it may or may not take part in a given match. A learner reading the results should think of matches as a list whose first item is the whole match and whose later items are the captured parts.

Because the optional group may not participate, the program should check matches.size() and matches[1].matched before reading a submatch. The diagram below shows the state of matches after a successful search on the line Send to AB12345 now, where the optional dash group does not appear.

Consider a small program that reads a file one line at a time. For each line, it searches for the postal code pattern and prints the match. The pattern is defined before the loop, each line is read in turn, and a result is printed only when the search succeeds.

The pattern is stored in a regex object and declared as regex pat with a raw string literal. Its text, written in the notation you will see in regular expression references, reads as two letters, optional space, five digits, and an optional dash with four digits. Each token below contributes one part of that reading.

The quantifiers {2} and {5} set exact counts, while the star in \s* allows zero or more and the question mark after the group makes the whole group optional. The parentheses do two jobs at once: they group the dash and digits together, and they make that group capturable, which is why it can appear as matches[1].

Now trace the pattern across the text TX 77845. The first two positions are consumed by the two-letter piece, the third position by the optional whitespace piece, and the last five positions by the five-digit piece. Because the optional group does not appear in this text, it contributes nothing, and the match ends after the fifth digit.

A raw string literal starts with R" followed by an opening parenthesis, and ends with a closing parenthesis followed by a quote. Inside it, backslashes and quotes are taken exactly as written, so they do not need to be escaped. Patterns tend to contain many backslashes, which is why raw strings are particularly suitable for regular expressions. The pattern appears in the definition in the same form you would write it on paper.

Both versions define the same pattern. The conventional form forces every backslash to be doubled, which makes the pattern harder to read and easier to mistype. The raw form keeps the pattern in the notation used in regular expression references, so the text you read in the code is the text that describes the matching.

Most errors with patterns come from confusing the string syntax with the pattern syntax, and from misreading the smatch that regex_search fills in. The three mistakes below follow directly from the material in this article.