Sorting Dictionary Output by Frequency
Nested loops are essential for processing multi-level data structures like lines and words in a file. The outer loop iterates through lines, and the inner loop processes each word within that line.
From File Layers to Word Counts
Text in a file arrives in layers. The outer layer consists of lines, and each line contains individual words. Counting every word therefore requires two levels of processing: an outer loop that selects each line and an inner loop that processes the words inside that line. The dictionary records the result by using each word as a key and its frequency as the associated value.
The inner loop completes all of its iterations for one line before the outer loop moves to the next line. This is what allows the program to count every word in every line.
The Two-Level Loop Trace
Imagine an input file containing two lines: 'the cat sat' and 'the dog sat'. The outer loop first selects the line 'the cat sat'. The inner loop then processes 'the', 'cat', and 'sat' in sequence. Only after those three words have been processed does the outer loop select 'the dog sat'. The inner loop then processes 'the', 'dog', and 'sat'.
Building the Counts Dictionary
For each word, the program checks whether that word is already a key in the counts dictionary. If the word is not present, the program adds it with a value of 1. If the word is already present, the program increases its existing value by 1. The if-else structure is important because the first occurrence needs to create the key, while later occurrences need to update the value.
counts = {} for line in fhand: words = line.split() for word in words: if word not in counts: counts[word] = 1 else: counts[word] += 1
A Complete Count Trace
Tracing Two Input Lines
Count the words in the two lines 'the cat sat' and 'the dog sat'.
First line: the: The word is not yet a key, so the dictionary receives the entry 'the': 1.
First line: cat: The word is new, so the dictionary receives 'cat': 1.
First line: sat: The word is new, so the dictionary receives 'sat': 1. After the first line, the dictionary is {'the': 1, 'cat': 1, 'sat': 1}.
Second line: the: The word is already a key, so its value increases from 1 to 2.
Second line: dog: The word is new, so the dictionary receives 'dog': 1.
Second line: sat: The word is already a key, so its value increases from 1 to 2.
The final dictionary is {'the': 2, 'cat': 1, 'sat': 2, 'dog': 1}.
What do you think happens?
After the first line 'the cat sat' has been processed, what should the dictionary contain?
Reveal answer
Answer: {'the': 1, 'cat': 1, 'sat': 1}
Each of the three words has appeared once. The repeated word cases have not occurred yet because the second line has not been processed.
Incrementing Counts Compactly
| Expanded assignment | Compound assignment | Meaning |
|---|---|---|
| counts[word] = counts[word] + 1 | counts[word] += 1 | Read the current value, add 1, and store the result back |
| counts[word] = counts[word] - 1 | counts[word] -= 1 | Subtract from the current value and store the result |
| counts[word] = counts[word] * 1 | counts[word] *= 1 | Multiply the current value and store the result |
| counts[word] = counts[word] / 1 | counts[word] /= 1 | Divide the current value and store the result |
From Raw Output to Frequency View
After the nested loops finish, printing the dictionary shows each word as a key and its count as the value. The source notes that dictionary output may appear random because the keys are stored in hash order rather than in the order in which they were added. This does not mean the counting failed: the word-to-count entries are still the important result.
A frequency-oriented view is useful when the goal is to identify the most common words quickly. In the source's Romeo and Juliet example, 'is', 'the', and 'and' appear 3 times, 'sun' appears 2 times, and many other words appear once. The dictionary captures these frequencies, but its unsorted display does not make the most frequent entries easy to spot. Additional processing is therefore needed to present the results in a more readable frequency order.
Control-Flow Mistakes
Processing only one word from each line
The file contains lines, and each line contains multiple words. Without the inner loop, the program does not process every word within each selected line.
Fix:
Use an outer loop for lines and place the word-processing loop inside it so the inner loop completes for every line.Resetting the dictionary for every line
Earlier line counts would be discarded when the dictionary is recreated for the next line.
Fix:
Keep one counts dictionary for the complete counting process so entries from earlier lines remain available when later words are processed.Assigning 1 every time a word appears
A repeated word would not accumulate its frequency; its existing count would be replaced by 1.
Fix:
Check whether the word already exists. Add it with 1 when it is new, and increment its current value when it is repeated.Assuming an apparently random dictionary display means the counts are wrong
The source explains that dictionary output may appear random because keys are stored in hash order rather than insertion order.
Fix:
Inspect the key-value pairs and use additional organization when you need a frequency-focused display.
Practice the Trace
Trace the input lines 'red blue red' and 'blue green'. Write the dictionary after the first line, then write the final dictionary after the second line. For each word, identify whether the program creates a new key or increments an existing value.
Hints
- Process all three words from the first line before moving to the second line.
- The second occurrence of red should use the existing red entry.
- The second line contains a repeated blue and a new green.
What do you think happens?
After processing both practice lines, which final dictionary is correct?
Reveal answer
Answer: {'red': 2, 'blue': 2, 'green': 1}
The first line gives red two occurrences and blue one occurrence. The second line adds one more blue and one green, producing red: 2, blue: 2, and green: 1.
What to Remember
- The outer loop processes one file line at a time, while the inner loop processes every word in the selected line.
- A dictionary stores each word as a key and its frequency as the value.
- A new word receives a count of 1; a repeated word increments its existing count.
- counts[word] += 1 is a compact form of counts[word] = counts[word] + 1.
- The dictionary contains the frequency data, but additional organization can make the most frequent words easier to identify.
Key Takeaways
- Nested loops navigate the two layers of a text file: lines first, then words within each line.
- The dictionary's keys are words and its values are occurrence counts.
- The if-else structure distinguishes a word's first occurrence from later occurrences.
- The += operator updates an existing count concisely without changing the operation.
- Sorting or organizing the completed dictionary output helps reveal which words occur most frequently.