Sorting Dictionary Data
Use a dictionary to count word frequencies by checking if each word exists as a key and either initializing it to 1 or incrementing its count.
Why Frequencies Matter
When you need to understand what is inside a file, one useful question is: how many times does each item appear? This is a frequency analysis problem. A dictionary is well suited to it because each unique item can be a key and its frequency can be the associated value. As the file is processed, the dictionary grows when a new item appears and changes an existing value when an item appears again.
First Dictionary Updates
Suppose a file contains the lines “the cat sat on the mat” and “the dog ran”. Split the lines into words and process the words from left to right. The first word, “the”, is not in the empty dictionary, so it becomes a key with value 1. When “the” appears again, it is already a key, so its value becomes 2. The same decision is made for every later word.
Processing the first five words
Trace the dictionary while processing the words the, cat, sat, on, the.
Start: Begin with an empty dictionary: {}.
the: The key is new, so add the: 1.
cat: The key is new, so add cat: 1.
sat: The key is new, so add sat: 1.
on: The key is new, so add on: 1.
the again: The key already exists with value 1, so increase it to 2.
{'the': 2, 'cat': 1, 'sat': 1, 'on': 1}
The Counting Algorithm
The algorithm starts with an empty dictionary. For each line in the file, split the line into words. Then process each word individually. Membership testing determines whether the word is already a dictionary key. A missing key receives value 1; an existing key has its value increased by 1. At the end, every unique word is paired with its frequency.
counts = {} for line in file: words = line.split() for word in words: if word not in counts: counts[word] = 1 else: counts[word] += 1
Reading Dictionary State
Tracing means recording the dictionary after each item is processed. This makes the internal change visible: a first occurrence adds a key, while a repeated occurrence changes only that key's value. For the source example, the final dictionary contains each unique word and the number of times it appeared. The word “the” has value 2 because it occurs twice; the other words shown occur once.
If a word has an unexpected count, print the dictionary inside the loop after each word is processed. The trace can show whether the word was never encountered, whether it was treated as a new key, or whether its existing value was incremented incorrectly. Once the logic is confirmed, remove the debug print statements.
Structured File Histograms
The same counting pattern works when the input is structured rather than ordinary prose. A mail log can contain lines beginning with “From”, followed by an email address, while the day appears in a predictable position. The program can filter for lines that start with “From”, split those lines into fields, extract the day from the third word, and count each day in a dictionary.
The dictionary logic has not changed. Only the item being counted has changed: instead of every word, the program counts extracted days. The source describes a result in which Friday has 20 messages, Thursday has 6, and Saturday has 1. The resulting dictionary is a histogram because its keys are categories and its values are frequencies.
Other Frequency Categories
A mail log can also be used to count email addresses. For each line beginning with “From”, extract the email address and use it as the dictionary key. The value records how many messages came from that sender. The source example reports that cwen@iupui.edu appears 5 times, while some other senders appear once.
| Item being counted | Dictionary key | Dictionary value |
|---|---|---|
| Words | Each unique word | Number of appearances |
| Mail days | Each extracted day | Number of messages on that day |
| Email senders | Each extracted email address | Number of messages from that sender |
Mistakes in Frequency Counting
Incrementing a key before giving it an initial value
A new word does not yet have a stored count to increment.
Fix:
Check membership first. Add counts[word] = 1 for a new key, and increment only when the key already exists.Counting every line in a structured log
Only lines with the required format should contribute to the selected category.
Fix:
Filter lines by their format, such as checking whether a line starts with “From”, before extracting fields.Extracting the wrong field
The dictionary would count a different piece of text from the intended category.
Fix:
Split the line and use the documented predictable position, such as words[2] for the day in the source example.Skipping a state trace when the result looks wrong
The final result does not reveal when a key was missed or incremented incorrectly.
Fix:
Print the dictionary after each word or extracted item to locate the first incorrect update.
Practice the Pattern
A file contains several lines of text. Write the counting logic that produces a dictionary of word frequencies. Then trace the dictionary after processing the words “red”, “blue”, “red”, and “red”.
Hints
- Start with an empty dictionary.
- Use a membership check for each word.
- The first “red” creates a key with value 1.
- The second and third “red” increment the existing value.
Practice trace result
Determine the final dictionary after processing red, blue, red, red.
red: The key is new, so its count is 1.
blue: The key is new, so its count is 1.
red again: The existing red count increases from 1 to 2.
red once more: The existing red count increases from 2 to 3.
{'red': 3, 'blue': 1}
Key Takeaways
- A frequency dictionary stores each unique item as a key and its occurrence count as the value.
- The essential decision is whether the current item is already a key: initialize it to 1 if it is new, or increment it if it exists.
- Tracing the dictionary after each update reveals when keys are added and when counts change.
- Structured files can be counted by filtering lines, extracting a predictable field, and applying the same dictionary pattern.
- Words, days, and email addresses use the same counting logic; only the extracted item changes.
Key Takeaways
- Use dictionaries to map each unique item to its frequency.
- Initialize a missing key with 1 and increment an existing key.
- Trace dictionary state to debug counting logic.
- Filter and parse structured records before counting an extracted category.
- The same pattern builds histograms for words, days, and email addresses.