Sorting and Ordering Data

From Text to Top Words

Imagine a text file containing thousands of words. Printing every word count would produce a long, unsorted result, so the useful question is not only how often each word appears, but which words appear most often. The solution has two phases: first count each word, then organize the counts so the largest frequencies rise to the top.

Building the Frequency Dictionary

The counting phase reads the file line by line. Each line is processed by removing punctuation and converting it to lowercase, and then the program examines each word. For a word that is not yet in the dictionary, the program creates an entry with a count of 1. For a word that is already present, the program increases its existing count by 1.

counts = {} for word in words: if word not in counts: counts[word] = 1 else: counts[word] = counts[word] + 1

Reversing the Pair for Sorting

After counting is complete, the dictionary contains each unique word as a key and its frequency as the value. To sort by frequency, take each word-count pair and create a tuple with the count first and the word second. This changes a pair shaped like word, count into count, word.

items = []

for key, val in counts.items():
    items.append((val, key))

Tuples are ordered and immutable sequences. They are suitable here because each result has a fixed two-part structure: frequency followed by word. When tuples are compared for sorting, the first element is considered first, and the second element can be considered when the first elements tie.

Ordering by Frequency

Sort the list of count-word tuples in reverse order. Because the frequency is the first element, reverse sorting places the largest frequencies at the beginning. The list now represents the document ordered from the most frequent words to the least frequent words.

items.sort(reverse=True)

Selecting the Most Common Words

Once the tuple list has been sorted, select the first N entries with a slice. For the ten most common words, use lst[:10]. The resulting slice contains up to ten tuples, and a loop can print each tuple's frequency and word.

for val, key in items[:10]: print(key, val)

Complete Two-Phase Pattern

The complete solution keeps the jobs separate. The first phase reads and counts words. The second phase changes the dictionary entries into frequency-first tuples, sorts them in reverse order, and prints the requested number of results.

counts = {}

for line in file:
    line = line.lower()
    for word in line.split():
        if word not in counts:
            counts[word] = 1
        else:
            counts[word] = counts[word] + 1

items = []
for key, val in counts.items():
    items.append((val, key))

items.sort(reverse=True)

for val, key in items[:10]:
    print(key, val)

Mistakes with Frequency Sorting

• Keeping the word first in each tuple

  Sorting compares the first tuple element first, so the word becomes the primary sorting value rather than the frequency.

  Fix: Create tuples in the order (count, word).

• Sorting without reverse order

  The highest frequencies will not be placed at the beginning by the required descending-frequency process.

  Fix: Sort the frequency-first tuples in reverse order.

• Reading the tuple fields in the wrong order

  The transformed tuple stores the frequency first and the word second.

  Fix: Unpack as val, key and print the word together with its frequency.

• Assuming the top-ten slice always contains ten entries

  A short document may produce fewer than ten tuples.

  Fix: Use the slice directly; it returns all available tuples without error.

Practice the Pipeline

1. A correct solution should produce the frequency-first tuples (47, love), (31, hope), and (18, peace). They are already ordered from highest to lowest frequency, so selecting the first two returns (47, love) and (31, hope).

Key Takeaways

1. Use a dictionary with words as keys and frequencies as values to count occurrences.
2. For every word, initialize its count to 1 when it is new and increment the count when it already exists.
3. Convert each word-count pair into a frequency-word tuple so sorting compares the frequency first.
4. Sort the tuple list in reverse order to place the most frequent words at the beginning.
5. Slice the sorted list to select the top N words, including fewer than N when fewer results are available.