Pattern Matching with re.search() and re.match()
re.findall() is a Python method that extracts all non-overlapping substrings matching a regex pattern from a string.
One Line, Many Matches
When a string contains several occurrences of the same pattern, re.findall() collects them into one result. It scans the string from left to right, records each non-overlapping match, and returns the collected results. The important question is not only which text matches, but also how the regular expression is grouped, because capturing groups determine the shape of the returned value.
The Left-to-Right Scan
Imagine findall() moving through a line from position 0 onward. It scans forward until it finds text that can begin a match. After the pattern matches, findall() records that result and resumes scanning at the position immediately after the match ends. Because the next scan begins after the completed match, a later result cannot overlap the earlier one.
Tracing repeated matches
Suppose a pattern matches the two-character sequence ab in the generated text ababa.
First scan: The first ab is found and recorded.
Resume position: Scanning resumes after that complete match, so the next search begins after the first ab.
Second scan: The later ab is found and recorded. The possible match beginning inside the first match is not returned because findall() produces non-overlapping matches.
The result contains the two non-overlapping occurrences identified by the scan.
Capturing Groups Shape Results
A capturing group is a set of parentheses that creates a subpattern. The number of capturing groups controls what each item in the findall() result represents. With no capturing groups, findall() returns a flat list of strings containing the complete matches. With exactly one capturing group, it still returns a flat list of strings, but each string contains only the content captured by that group. With two or more capturing groups, it returns a list of tuples, and each tuple contains the captured groups in their order.
| Capturing groups | Returned item | Meaning |
|---|---|---|
| None | String | The complete match |
| Exactly one | String | Only the content of that group |
| Two or more | Tuple | The captured groups in order |
The return structure depends on the number of capturing groups.
Collecting Repeated Occurrences
A practical use of findall() is extracting every occurrence of a repeated pattern from one line, such as several email addresses in an email log line. The method scans toward the first possible address, records the complete match when the pattern succeeds, then continues after that match until it finds the next one. The final result contains all matches that the pattern identified without overlap.
Consider a generated log line containing three email addresses. If the regular-expression pattern matches an entire email address and contains no capturing groups, findall() returns a flat list of the three complete address strings. If the same pattern is reorganized into two capturing groups, each result becomes a tuple containing the two captured parts in order. The text being scanned is the same; the result structure changes because the pattern's groups changed.
Diagnosing Unexpected Output
When findall() returns something different from what you expected, inspect the pattern before changing the surrounding program. First check whether the pattern actually matches the intended text. Next count its capturing groups, because adding or removing a group changes the return structure. Then inspect quantifiers such as *, +, and ?, which may match more or fewer characters than intended. By default, quantifiers are greedy and match as much as possible; adding ? after a quantifier makes it lazy.
Expecting complete matches when the pattern has one capturing group
With exactly one capturing group, findall() returns the group's content rather than the entire match.
Fix:
Count the capturing groups and decide whether the group content or the complete match is needed.Expecting strings when the pattern has multiple capturing groups
Two or more capturing groups cause findall() to return a list of tuples.
Fix:
Treat each tuple as the captured groups in their pattern order.Assuming findall() returns overlapping matches
findall() resumes scanning after the previous match ends, so overlapping results are not returned.
Fix:
Trace the end of each recorded match and identify the position where the next scan begins.Ignoring greedy or lazy quantifiers
Quantifiers affect how much text the pattern consumes. By default they are greedy; adding ? after a quantifier makes it lazy.
Fix:
Inspect *, +, and ? and verify whether the pattern should consume as much as possible or use lazy behavior.
- Test the regular-expression pattern against the text you believe it should match.
- Count the capturing groups in the pattern.
- Check whether the pattern matches complete text or only captured subparts.
- Inspect *, +, and ? for unintended match lengths.
- Check whether the expected matches overlap; findall() returns non-overlapping results.
Check Your Prediction
A generated pattern finds two occurrences in a line. In the first version, the pattern has no capturing groups. In the second version, it has two capturing groups. Predict the result structure in each version: will each result be a string or a tuple, and will the complete match or captured parts be represented?
Hints
- Start by counting the capturing groups.
- Remember that zero groups and one group produce lists of strings.
- Remember that two or more groups produce lists of tuples.
Checking the return structure
Determine the result shape for two non-overlapping matches when the pattern has no groups, then determine it when the pattern has two groups.
No groups: The result is a flat list of strings, with each string representing a complete match.
Two groups: The result is a list of tuples, with each tuple containing the two captured groups in order.
Match count: The number of collected occurrences is based on the non-overlapping scan, while the item structure is based on the capturing-group count.
The scan determines how many results are collected; the capturing groups determine whether each result is a string or a tuple and what it contains.
Key Takeaways
- re.findall() scans a string from left to right and extracts all non-overlapping matches.
- After recording a match, scanning resumes at the position after that match ends.
- With no capturing groups, findall() returns complete matches as strings.
- With one capturing group, findall() returns the captured content as strings.
- With two or more capturing groups, findall() returns tuples containing the captured groups in order.
- Unexpected results usually require checking the pattern, group count, quantifiers, and intended match boundaries.
Key Takeaways
- findall() collects every non-overlapping match it can identify while scanning left to right.
- The number of capturing groups controls the structure and contents of each returned item.
- Zero or one capturing group produces strings; two or more produce tuples.
- Debugging starts by checking the pattern, then its groups, quantifiers, and match boundaries.