Regular Expression Notation

A string is one of the most common kinds of data a program handles. Names, messages, file paths, and user input are all strings. When you write a string literal or read text from a user, you usually think of the text as a single value. Working with strings effectively, however, requires a more detailed picture of what that value is and how a program can reach into it. This article introduces that picture. It is the starting point for the rest of the chapter, because the three ways of working with string contents that the section introduces all depend on it.

The central idea is that a string is an ordered sequence of characters. Order is what makes a string more than a bag of letters. The first character comes before the second, the second before the third, and so on. Because of this order, every character in a string has a definite place that can be referred to.

A convenient way to picture this is as a row of numbered slots, with one character per slot. The slot numbers are the positions. Reading a character means naming its position. Searching, pattern matching, and iteration all reduce to ways of visiting or comparing these slots in some order. The picture below shows the general model rather than any particular machine's layout: each slot holds one character, the slots are arranged in a definite order, and the position number is what lets a program say which character it means. The model applies to any string, whatever its length.

Once a string is seen as a sequence, the next question is how positions are counted. Many programming languages, including C++, number the characters of a string starting from 0 rather than 1. The first character is at position 0, and a string with n characters has positions running from 0 up to $n-1$. The last character is therefore at position length minus 1, not at position length. This convention affects nearly every operation that touches a string's contents, so it is worth fixing in your mind early.

The section lists three topics that build on the sequence model. Each answers a different question about a string. Searching asks where a piece of text occurs. Regular expression notation describes patterns that text may follow. Iterators provide a systematic way to move through a string one element at a time. All three rely on the fact that positions are ordered, which is why they are best learned in the order the chapter presents them.

The notation is not fixed to that single default. The source states that it can be adjusted to match various standards. The same general idea of a pattern can therefore be written under different notational conventions, and you select the convention your situation requires. The key point is that a pattern's text alone does not fix its meaning; the notation in force does. When you read or share a pattern, the notation it was written for should be clear.

Many learners conclude that regular expressions are a kind of code that only its author can ever read. A single line can hold a dense cluster of symbols, and once written it seems to resist later review. The source names this belief directly: regular expressions can easily become a write-only language. The belief is partly true because a regular expression packs a lot of meaning into very few characters. A reader has to recognize each symbol, understand how symbols combine, and keep track of which part of the pattern each character belongs to. When a pattern grows without discipline, these combinations become hard to follow, even for the person who wrote them.

The source gives one specific modifier: the question mark ?. Placing ? in a pattern makes that pattern lazy. A lazy pattern matches as little text as it can while still satisfying the pattern, as opposed to matching as much as possible. Because the modifier is a single symbol, the change it makes is easy to overlook when reading a pattern. Always check for it when you are predicting what a pattern will match.

The comparison below uses an original example to make the difference visible. Suppose the input text is the HTML fragment <b>bold</b>. The greedy pattern <.+> begins at the first less-than sign and extends as far as it can, so it matches the whole fragment, through the final greater-than sign. The lazy pattern <.+?> begins at the same place but stops at the first greater-than sign it reaches, so it matches only the opening tag <b>. The two patterns differ by a single symbol, but they stop in different places.

The notation includes \i, which the source describes as allowing you to express a subpattern in terms of a previous subpattern. Instead of spelling out the same subpattern again, you refer back to an earlier one. The result reads as a composition of named ideas rather than one long sequence of symbols. The source does not provide further detail about \i in this section, so this article does not describe its exact behavior beyond that statement.

Regular expressions are not only used to test a single string. The source explains that you can use regex iterators to iterate over a stream while looking for a pattern. The pattern is applied repeatedly as you move through a stream of text, and each match the iterator finds is a point where your program can act. The sequence below is an original illustration of that process. The source names the regex iterator and its purpose but does not spell out every step, so treat it as a way to picture the work rather than a full specification of the library.

Notice what the iterator does at each stage. It keeps a place in the stream, so each search starts from where the previous match ended rather than from the beginning again. When it reaches the end of the stream with no further matches, the process is finished. Each match is an opportunity for your program to act, which is what makes an iterator useful for repeated passes over text rather than a single lookup.