Sorting and Ordering Data Structures
Finding unique words involves a pipeline: read file → split lines into words → check membership to avoid duplicates → sort alphabetically
From Text to Vocabulary
Imagine trying to discover every distinct word used across thousands of pages by reading the pages and tracking each new word manually. A program can perform this task as a sequence of transformations: read the file, split each line into words, keep only words that have not already been collected, and sort the final list alphabetically. The important idea is not one isolated list operation. It is the movement of data through a pipeline, where the data changes form at each stage.
Tracking the Growing List
The unique-word list grows one word at a time. When a word is extracted from a line, the program checks whether that word is already in the list. If the word is not present, it is added. If the word is already present, the program skips it. This check-before-adding pattern is the exact point where duplicates are filtered.
A Small Membership Trace
Trace the unique list while processing the words the, sun, the, moon.
Process the: The list is empty, so the is not in it. Add the.
Process sun: sun is not in the current list. Add sun.
Process the again: the is already in the list. Skip it, so the list does not gain another copy.
Process moon: moon is not in the current list. Add moon.
The resulting unique list is [the, sun, moon] before sorting.
Reading and Splitting Lines
Before membership testing can occur, the program must turn each line into separate word strings. The split() function performs this transformation. When called on a string without arguments, split() breaks the string at whitespace boundaries such as spaces, tabs, and newlines, returning a list of the resulting pieces. This changes the program's view of the data from one complete line to several individual words that can be processed one by one.
line = "the sun rises" words = line.split() print(words)
Sorting the Final Collection
After every line has been read and every extracted word has passed through the membership check, the program sorts the remaining list alphabetically. Without this final operation, the list reflects the order in which words were first encountered. Sorting makes the result easier to search, verify, and understand because the output follows a consistent alphabetical arrangement.
["moon", "sun", "the"]Complete File Workflow
unique_words = [] with open("words.txt") as file: for line in file: words = line.split() for word in words: if word not in unique_words: unique_words.append(word) unique_words.sort() print(unique_words)
The code follows the conceptual sequence directly. Reading happens before splitting, splitting happens before membership testing, and sorting happens only after the file's lines have been processed. This order matters: sorting an incomplete collection would not produce the vocabulary of the entire file, and adding every extracted word without a membership check would preserve duplicates.
For romeo.txt, the expected result is a list of 26 unique words sorted alphabetically: Arise, But, It, Juliet, Who, already, and, breaks, east, envious, fair, grief, is, kill, light, moon, pale, sick, soft, sun, the, through, what, window, with, yonder. The placement of Arise and But before already reflects the ordering of uppercase letters before lowercase letters.
Debugging Divergent Output
When the output differs from what you expect, trace the collection after each stage rather than looking only at the final print statement. Ask which lines were read, what split() produced, which words passed the membership test, and whether sorting occurred after the complete collection was built. The first stage where the state differs from the expected state identifies the likely source of the problem.
Adding every extracted word without checking membership
Repeated words enter the collection multiple times, so the result is not a list of unique words.
Fix:
Check if the word is not in the list before appending it.Forgetting to sort the completed list
The output reflects first-appearance order rather than alphabetical order.
Fix:
Sort the final list after all lines and words have been processed.Not reading the entire file
Words from unread lines cannot appear in the result.
Fix:
Trace the file-reading loop and confirm that all lines are processed.Ignoring case sensitivity or punctuation
Case affects the displayed order, and punctuation can affect the extracted strings.
Fix:
Inspect the actual word strings produced during extraction and compare them with the expected output.
Practice the Trace
Suppose two lines produce these word sequences: ["bright", "moon"] and ["moon", "bright", "night"]. Trace the unique list after each word, then determine the final sorted list.
Hints
- Start with an empty list.
- For each word, decide whether it is already present before adding it.
- Apply sorting only after all words from both lines have been processed.
Practice Check
Find the unique sorted result for bright, moon, moon, bright, night.
Filter duplicates: The first bright and moon are added. The second moon and second bright are already present and are skipped. night is added.
Sort: Arrange the remaining words alphabetically after the membership checks are complete.
The final sorted list is [bright, moon, night].
Pipeline Takeaways
- Read the file and process its lines.
- Use split() to transform each line into separate word strings.
- Use membership testing before appending so duplicates are excluded.
- Sort the completed unique list alphabetically.
- Debug by comparing the collection's state after reading, splitting, filtering, and sorting.
Key Takeaways
- Unique-word extraction is a pipeline that moves from file text to lines, words, unique words, and finally sorted output.
- split() creates a list of separate word strings from a line.
- The condition word not in the unique list is the point where duplicates are filtered.
- Sorting after the complete file has been processed makes the vocabulary easier to search, verify, and understand.
- Tracing the collection after each stage reveals whether a problem comes from extraction, duplicate filtering, incomplete reading, case, punctuation, or sorting.