Python's re Module: search() and findall()
grep stands for Generalized Regular Expression Parser. It is a command-line program built into Unix, Linux, and macOS systems that searches through files and prints out every line that matches a regular expression pattern you provide. Think of it as the command-line equivalent of the search() function you use in Python—but instead of searching within a string variable in your code, grep searches through entire files on disk.
From Strings to Files
Regular expressions can be used in more than one setting. In Python, the search() function is used to search within a string variable. grep applies a related idea from the command line: it searches through files on disk and prints every line that matches the regular-expression pattern you provide.
grep stands for Generalized Regular Expression Parser. It is a command-line program built into Unix, Linux, and macOS systems that searches files with regular expressions.
The Line-by-Line Search
When grep runs, it reads the file line by line. For each line, it tests whether the line matches the regular-expression pattern. A matching line is printed in its entirety to the terminal. A nonmatching line is skipped. This continues until grep reaches the end of the file.
grep returns matching lines, not just the part of a line that matched. Its terminal output is called standard output, or stdout.
The Basic Command Shape
The basic grep command has three important parts: the command name, a quoted regular-expression pattern, and the filename to search. Its general form is grep 'pattern' filename.
Finding sender lines
Find every line in mbox-short.txt that starts with From:.
Choose the file: The filename is mbox-short.txt, so grep reads that file.
Choose the pattern: The pattern is ^From:. The caret anchor means the text must occur at the start of a line.
Run the search: grep tests every line against the pattern and prints each line that matches.
The terminal displays every line in mbox-short.txt that begins with From:.
In this command, ^ is important because it restricts the match to the beginning of a line. Without that anchor, the pattern would not express the specific requirement that From: must be the first text on the line.
Python Search versus grep
| Tool | Search target | Result described in the source |
|---|---|---|
| Python search() | A string variable | Used to search within the string |
| grep | One or more files on disk | Prints every matching line to the terminal |
The comparison is about the search target and the output method. Python search() is described here as operating on a string variable. grep operates on files and sends matching lines directly to the terminal. Both use regular expressions, but they should not be assumed to support exactly the same features.
Syntax Mismatches
A pattern that works in Python may not work unchanged in grep. The source identifies one specific difference: grep does not support the \S shorthand. In Python, \S matches any non-whitespace character, meaning a character that is not a space, tab, or newline.
| Goal | Python notation described in the source | grep notation described in the source |
|---|---|---|
| Match a non-whitespace character | \S | [^ ] |
| Meaning of the grep form | Not applicable | Match any character that is not a space |
Broadening a Restrictive Pattern
A search can be too restrictive when its pattern requires more exact text than the file actually contains. For example, the source pattern ^From: requires both the beginning of the line and the exact text From:. If the required line form is broader, revise the pattern instead of assuming the file has no relevant lines.
Using Python's \S shorthand in grep.
The source states that grep does not support the \S shorthand.
Fix:
Use [^ ] when the intended grep pattern is any character except a space.Forgetting that ^ changes the required position of the match.
The caret anchor restricts the match to the beginning of a line.
Fix:
Use ^ only when the desired text must occur at the start of each matching line.Expecting grep to print only the matching fragment.
grep prints the entire line when the line matches.
Fix:
Read the complete output line as grep's matching result.
Practice at the Terminal
You need to find sender lines in a file named mbox-short.txt. Write a grep command whose pattern matches only lines that start with From:.
Hints
- Use the caret anchor for the beginning of a line.
- Place the filename after the quoted pattern.
What do you think happens?
What will grep print when a line matches the pattern?
Reveal answer
Answer: The entire matching line
grep tests each line and prints the entire line when the line matches. Nonmatching lines are skipped.
A Python regular expression uses \S, but the equivalent grep search does not work. Rewrite the relevant part of the pattern using the grep notation described in this article, then explain what that notation excludes.
Hints
- The replacement uses square brackets and a caret.
- The source describes it as matching any character that is not a space.
Key Takeaways
- grep is a command-line regular-expression search tool that searches files and prints matching lines.
- The basic structure is grep 'pattern' filename.
- grep reads files line by line, tests each line, and sends every matching line to standard output.
- Python search() works within a string variable, while grep searches files on disk.
- Regex syntax is not identical across tools: grep does not support Python's \S shorthand, so [^ ] is used for the documented non-space match.
Key Takeaways
- grep applies regular expressions to files rather than to a string variable.
- It examines files line by line and prints complete matching lines to the terminal.
- The caret in ^From: restricts matches to lines that begin with From:.
- Python and grep do not share every regular-expression feature; grep does not support \S.
- When a search fails, inspect anchors, exact text requirements, and syntax differences between tools.