Stream Iterators

istream_iterator and ostream_iterator connect file streams to algorithms and containers, and are usually passed as arguments rather than used on their own.

Programming

Why Stream Iterators Exist

Stream iterators are rarely used on their own. They are normally handed to algorithms as arguments. The algorithm does the real work, while the stream iterator quietly moves values between a stream and a container. In this article you will follow one small program that reads words from a file, puts them in order, drops repeated words, and writes what remains to a second file.

What do you think happens?

Suppose the input file contains the words: pear apple pear. What do you think the output file contains, and in what order?

pear, apple, pear
apple, pear
pear, apple
apple, pear, pear

Reveal answer

Answer:

apple, pear

The words are gathered into a container, sorted into alphabetical order, and then copied with repeated values skipped. The sorted sequence is apple, pear, pear, and skipping the repeated pear leaves apple then pear, one word per line.

Keep one question in mind as you read: at which stage does the result stop matching what you expected? Most bugs in this style of program are not inside the algorithm itself. They sit at a boundary, such as where a value enters the program, where it is stored, or where it leaves.

Stream Iterators as Connectors

Definition

An istream_iterator reads values from an input stream. An ostream_iterator writes values to an output stream. Each one is an iterator, which means an algorithm can use it the same way it uses the iterators of a container.

The practical consequence is that the algorithm does not need to know that the other end of the iteration is a file. An algorithm sees a range of values to read from and a place to write to. Whether those values come from a vector, a set, or a file is hidden inside the iterator.

How does data get from an input file into an algorithm and back out to a stream, and where does each iterator sit along that path?

The program connects the two ends of a pipeline. On the input side, an ifstream is attached to a file, and an istream_iterator is built on that stream. On the output side, an ofstream is attached to another file, and an ostream_iterator is built on that stream. A vector or set sits between them as a buffer that holds the values while the algorithm works on them.

The algorithms never open a file. sort works on whatever range it is given, and unique_copy writes to whatever output iterator it receives. All file handling lives in the stream objects and their iterators. This keeps the algorithm calls short and identical in form to calls on in-memory containers.

The Full Program Step by Step

The first version of the program fills a vector from the input iterator, sorts it, and then copies it to the output with unique_copy. The three steps are separate, and each one hands its result to the next. Reading the program as a pipeline makes this clearer than reading it as a list of statements.

python

#include <algorithm>
#include <fstream>
#include <iterator>
#include <string>
#include <vector>
using namespace std;

int main() {
    ifstream in("words.txt");
    ofstream out("unique_words.txt");

    istream_iterator<string> eos;          // default-constructed: end marker
    istream_iterator<string> input(in);    // reads words from the file
    ostream_iterator<string> output(out, "\n");  // newline after each word

    vector<string> v;
    copy(input, eos, back_inserter(v));   // stage 1: fill the vector
    sort(v.begin(), v.end());              // stage 2: put words in order
    unique_copy(v.begin(), v.end(), output); // stage 3: write, skipping repeats

    return 0;
}

#include <algorithm>
#include <fstream>
#include <iterator>
#include <string>
#include <vector>
using namespace std;

int main() {
    ifstream in("words.txt");
    ofstream out("unique_words.txt");

    istream_iterator<string> eos;          // default-constructed: end marker
    istream_iterator<string> input(in);    // reads words from the file
    ostream_iterator<string> output(out, "\n");  // newline after each word

    vector<string> v;
    copy(input, eos, back_inserter(v));   // stage 1: fill the vector
    sort(v.begin(), v.end());              // stage 2: put words in order
    unique_copy(v.begin(), v.end(), output); // stage 3: write, skipping repeats

    return 0;
}

Output (expected)

Contents of unique_words.txt for input words.txt containing: pear apple pear banana apple
apple
banana
pear

Notes on the code

eos is a default-constructed istream_iterator and marks the end of the input
copy fills the vector from the input stream until eos is reached
sort reorders the vector, which unique_copy depends on
unique_copy writes to the output iterator, which sends each word to the file

Worked example

Tracing a Five-Word Input

The file words.txt contains: pear apple pear banana apple. Trace what each stage holds and what reaches unique_words.txt.

1
Stage 1: read
The input iterator reads the five words in file order. The vector now holds pear, apple, pear, banana, apple.
2
Stage 2: sort
sort rearranges the vector into alphabetical order: apple, apple, banana, pear, pear.
3
Stage 3: unique_copy
unique_copy walks the sorted vector and skips repeated values. The first apple is written, the second apple is skipped, banana is written, and the first pear is written while the second is skipped.
4
Stage 4: delimiter
The ostream_iterator writes a newline after each value, so each word lands on its own line.

Result

unique_words.txt contains three lines: apple, banana, pear.

What happens to each value in order as it moves from the input file into the vector and then out to the output file?

Notice that the vector holds the duplicates until the copy step. The sort is what places equal words next to each other, so that unique_copy can skip them as it writes. The file on disk is untouched until the last stage.

Reading the Declarations Closely

Two details in the declarations matter most. Both control what the program reads and what it writes, so both are common sources of surprises.

Vector Versus Set for Removing Duplicates

The vector version is longer than it needs to be. It reads the strings into a vector, sorts them, and then writes them out while skipping duplicates. A more direct approach is to never store duplicates in the first place. A set does this: it keeps its elements in order and does not keep duplicates.

python

#include <algorithm>
#include <fstream>
#include <iterator>
#include <set>
#include <string>
using namespace std;

int main() {
    ifstream in("words.txt");
    ofstream out("unique_words.txt");

    istream_iterator<string> eos;
    istream_iterator<string> input(in);
    ostream_iterator<string> output(out, "\n");

    set<string> s;
    copy(input, eos, inserter(s, s.begin()));  // fills the set; duplicates never stored
    copy(s.begin(), s.end(), output);           // writes the set in order

    return 0;
}

#include <algorithm>
#include <fstream>
#include <iterator>
#include <set>
#include <string>
using namespace std;

int main() {
    ifstream in("words.txt");
    ofstream out("unique_words.txt");

    istream_iterator<string> eos;
    istream_iterator<string> input(in);
    ostream_iterator<string> output(out, "\n");

    set<string> s;
    copy(input, eos, inserter(s, s.begin()));  // fills the set; duplicates never stored
    copy(s.begin(), s.end(), output);           // writes the set in order

    return 0;
}

Output (expected)

Contents of unique_words.txt for input words.txt containing: pear apple pear banana apple
apple
banana
pear

Notes on the code

The set replaces the vector, the sort, and unique_copy
The first copy fills the set from the input stream
The second copy writes the set's elements to the output stream

How do the two approaches differ in their intermediate steps, and what does the container hold at each stage?

Because the set already holds its elements in order without duplicates, the sort step disappears and unique_copy is replaced by a simpler copy. The vector version needs three operations, while the set version needs two lines: one that fills the set and one that copies it out. The output file is the same in both cases, but the set version has fewer stages where something could go wrong.

Finding Where the Output Goes Wrong

When the output does not match what you predicted, follow the pipeline in order instead of rereading the algorithm call. Each stage has a clear input and output, so the first stage that disagrees with your expectation is the place to look.

1
Confirm what the input iterator actually read from the file, word by word.
2
Check what the container holds after the fill, before any sort.
3
Check what the algorithm produced, such as the sorted vector or the set contents.
4
Check what reached the output stream, including how each value was delimited.

Common mistake
Looking for the bug inside the algorithm when the input is wrong
What it looks like
Output is missing a word, and the programmer rewrites the unique_copy call without checking the file.
Why it fails
The algorithm only sees what the input iterator delivered. If a value was never read, no algorithm can recover it.
Do this instead
Check the input stage first, then move forward through the pipeline.
Common mistake
Forgetting that the sort comes before unique_copy
What it looks like
The vector is filled and copied directly with unique_copy, so repeated words scattered through the file survive.
Why it fails
The three steps are a pipeline, and each one hands its result to the next. Skipping the sort changes the order the copy step sees.
Do this instead
Keep the fill, sort, and copy stages in that order, or use a set that stores values in order.
Common mistake
Using an ordinary iterator as the end marker
What it looks like
The end argument is built from the same file stream as the start argument, so it is not a default-constructed istream_iterator.
Why it fails
Reading stops when the iterator reaches the end marker, and the default-constructed eos is what represents the end of the input.
Do this instead
Declare a default-constructed istream_iterator as the end marker and pass it as the second argument.
Common mistake
Assuming the output is one word per line without a delimiter
What it looks like
The ostream_iterator is built without the newline argument, so all words run together in the output file.
Why it fails
The second argument to ostream_iterator is written after each value. Without it, nothing separates the words.
Do this instead
Pass the delimiter, such as a newline, when constructing the ostream_iterator.

Practice Tracing a Pipeline

MEDIUM

The input file contains: kiwi fig kiwi date fig kiwi. Trace the vector version of the program. What does the vector hold after the fill, after the sort, and what does unique_words.txt contain? Then say what a set version would write, and why.

Hints

Write the six words in file order first, then sort them alphabetically.
After sorting, equal words sit next to each other. unique_copy writes only the first of each run.
A set stores each word once in order, so compare its contents with the unique values from the sorted vector.

Your attempt (not saved)

Summary of the Pipeline

1
istream_iterator and ostream_iterator connect file streams to algorithms and containers, and are passed as arguments rather than used on their own.
2
The eos object, a default-constructed istream_iterator, marks the end of input, and the second argument to ostream_iterator is written after each value.
3
The vector version fills, sorts, and copies with unique_copy, while the set version stores ordered unique values and needs fewer steps.
4
When output is wrong, check each stage in order, starting at the input and ending at the output delimiter.

At a glance

istream_iterator reads values from a stream and ostream_iterator writes values to a stream, so algorithms can work with files the same way they work with containers.
The program fills a vector, sorts it, and writes it with unique_copy, treating the three steps as a pipeline.
A set keeps elements in order without duplicates, replacing the sort and unique_copy with a simpler copy.
To find a wrong output, trace the pipeline stage by stage from input to output delimiter and stop at the first stage that disagrees with your expectation.

Related concepts

More concepts starting with S