Writing Python Functions

A test case checks a function's output against an expected result for a given input, and in this style each assert statement represents one test case.

Programming

Why Reading Code Is Not Enough

Once you have written a function, you need some way to find out whether it does what it is supposed to do. Reading the code over once is rarely enough, because a function can look correct and still give a wrong result for some inputs. A more reliable approach is to choose some inputs, decide in advance what the output should be, run the function, and compare the two. This approach is called validating code with test cases.

What do you think happens?

If you have a function and a list of inputs with expected outputs, what would you need to write in Python so that the computer checks each result for you and tells you when one of them is wrong?

Print each result and read the printed values by eye
Write a check that compares each actual result with its expected value and stops at the first mismatch
Rewrite the function once for each input

Reveal answer

Answer:

Write a check that compares each actual result with its expected value and reports a mismatch automatically.

Python can do the comparing for you. Each test case compares the function's actual output with the expected output, and an assert statement reports the problem if they differ.

Test Cases as Input, Output, and Comparison

Definition

A test case checks a function's output against an expected result for a given input. In this style of testing, each assert statement represents one test case.

Every test case therefore has three parts. There is an input, which is the set of values you pass to the function. There is an expected result, which you decide before running anything. And there is a comparison, which checks whether the function's actual output equals that expected result. If the comparison holds, the test case passes. If it does not hold, the test case fails.

How does the input flow into the candidate function, and how does its output move into the comparison with the expected value?

The input goes into the function being tested, the function produces an actual output, and that output is compared with the expected value you wrote down. The check is a single true-or-false question. Only the comparison decides whether the test case passed.

The candidate Parameter and assert Calls

In the source material, test cases are written as assert statements inside a function named check. That function takes one parameter, named candidate. The parameter stands for the function being tested, so each assert calls candidate with a set of input values. Because candidate is a parameter, the same set of tests can be run against whichever function is passed in. This keeps the tests separate from the code they check.

The following example is written for this article. It defines a small function that adds two numbers, then defines a check function with three test cases. Read the code carefully and notice what each assert is comparing.

python

def add(a, b):
    return a + b

def check(candidate):
    assert candidate(1, 2) == 3
    assert candidate(0, 0) == 0
    assert candidate(-1, 5) == 4

check(add)

def add(a, b):
    return a + b

def check(candidate):
    assert candidate(1, 2) == 3
    assert candidate(0, 0) == 0
    assert candidate(-1, 5) == 4

check(add)

Output (expected)

No output. All three asserts are true, so check(add) returns without an error.

Notes on the code

add is the function under test, passed into check as candidate
Each assert calls candidate with one set of inputs and compares the result to the expected value
check(add) runs the three test cases from top to bottom

Output

No output appears when every assert is true. Silence from check(add) means every test case passed.

Tracing Which assert Fails

The check function runs its assert statements from top to bottom. For each one, Python first evaluates the comparison inside the assert. If the comparison is true, execution moves on to the next line. If it is false, the assert fails and the program stops at that line with an AssertionError. Lines below a failing assert do not run.

Worked example

Tracing a test case that does not match

Suppose the third assert in check is changed to expect 6 instead of 4, so the line reads assert candidate(-1, 5) == 6. What happens when check(add) runs?

1
First assert
candidate(1, 2) returns 3, and 3 == 3 is true, so execution continues.
2
Second assert
candidate(0, 0) returns 0, and 0 == 0 is true, so execution continues.
3
Third assert
candidate(-1, 5) returns 4, but the expected value is 6. The comparison 4 == 6 is false, so the assert fails.
4
Stop
The program stops at the third assert with an AssertionError. Any assert below it would not run.

Result

The failing line is the third assert. The output 4 differed from the expected 6 for the input (-1, 5).

When an assert's condition is False, what happens next, and how does that stop the remaining asserts from running?

What do you think happens?

With the third assert changed to expect 6, will the program print anything or stop? Which assert is the last one to run?

All three asserts run and the program finishes with no error
The first two asserts run, and the third one stops the program with an AssertionError
The program stops at the first assert

Reveal answer

Answer:

The first two asserts run and pass, and the third assert fails with an AssertionError.

The third comparison, 4 == 6, is false. Asserts run from top to bottom, so the failure stops execution at that line and nothing below it runs.

Mistakes That Make Tests Unreliable

A test is only useful if its expected value is right and its inputs exercise the behavior you care about. The mistakes below are common ways a test case can mislead you.

Common mistake
Writing the expected value without thinking it through
What it looks like
```
assert candidate(-1, 5) == 5 when -1 + 5 is 4
```
Why it fails
The expected value was decided carelessly, so the test fails even though the function is right.
Do this instead
Work out the expected output by hand before writing the assert, then decide whether a failure means the function or the expected value is at fault.
Common mistake
Assuming a failed assert means every test is broken
What it looks like
Seeing one AssertionError and rewriting the whole function
Why it fails
A failure identifies one input whose result differed from the expected value. The other test cases may still be correct.
Do this instead
Read the line the error points to, then check that single input and expected value.
Common mistake
Putting important checks below an assert that is already failing
What it looks like
A failing third assert hides the fourth assert from ever running
Why it fails
Asserts run from top to bottom, and a false one stops execution at that line.
Do this instead
Remember that a failure stops the remaining asserts, so fix the first failure and rerun to see the next result.
Common mistake
Choosing only inputs that the function handles easily
Why it fails
Test cases that avoid unusual inputs can pass while the function still does the wrong thing elsewhere.
Do this instead
Include a range of inputs, such as zero and negative values, so the test covers more than one case.
Common mistake
Writing a test case with no expected value decided in advance

Situation	Likely cause	What to check
The assert fails on a input you expected to work	The function may be wrong, or the expected value may be wrong	Recompute the expected output by hand, then inspect the function's logic
A test passes but the function is still wrong	The inputs did not reach the faulty behavior, or the expected value was set to match the output	Add inputs that cover edge cases and confirm each expected value independently
Later tests never run	An earlier assert is false and stops execution	Fix or examine the first failing assert, then rerun

Diagnosing a test case from its result

Practice Writing and Tracing Test Cases

MEDIUM

Write a function named multiply that takes two numbers and returns their product. Then write a check function that takes candidate and contains three assert statements, each calling candidate with different inputs and comparing the result to an expected value you have worked out by hand. Finally, change one expected value on purpose so that exactly one assert fails, and predict which line will report the AssertionError before you run it.

Hints

Use the same structure as the add example: one parameter named candidate, one assert per test case.
Choose one input that includes a zero and one that includes a negative number.
Predict the failing line by evaluating each comparison from top to bottom and stopping at the first false one.

Your attempt (not saved)

Key Takeaways

1
A test case checks a function's output against an expected result for a given input, and each assert statement represents one test case.
2
The check function receives the function under test as candidate, and each assert calls candidate with input values.
3
Asserts run from top to bottom; a false comparison stops the program at that line with an AssertionError.
4
A failing assert shows where the output diverged from the expected value, but you must still decide whether the function or the expected value is at fault.

At a glance

A test case checks a function's output against an expected result for a given input, and each assert statement represents one test case.
The check function takes the function under test as candidate and calls it with input values inside each assert.
Asserts run from top to bottom; a false comparison stops execution at that line with an AssertionError.
A failing assert shows where the output diverged from the expected value, but you must still decide whether the function or the expected value is wrong.

Related concepts

More concepts starting with W