Python Basics for Data Analysis

When you meet a new kind of data, the useful question is often whether a library already covers that task, rather than whether you must write the task yourself.

Programming

A Sensor Spreadsheet Raises Several Questions

Imagine you have a spreadsheet of sensor readings. You want to know whether the numbers drift over time, whether a few values look suspicious, and whether a simple model can predict the next reading. Each of those questions needs a different tool, and switching between many tools slows the work down. Python has become a common choice for this kind of work. The reason is a combination of two qualities: it is easy to use, and it comes with a very large collection of ready-made libraries.

This article looks at both qualities, at how Python's interactive style fits the way analysis actually unfolds, and at the kinds of tasks the library ecosystem covers. Along the way you will trace a short session line by line and predict what each line shows before you see the result.

Why Analysis Is a Repeated Cycle

The source describes data analysis and machine learning as fundamentally iterative processes, in which the data drives the analysis. Iteration means going around a cycle more than once. Each pass produces information, and that information changes what you try next. The first result is rarely the last, because you usually do not know in advance which summary, chart, or model will turn out to be useful.

The diagram shows the loop. Analysis moves from computing a result, to inspecting it, to choosing what to do next, and then back into another computation. Each trip around the loop is shaped by what the previous trip revealed. That is why the process is driven by the data rather than by a fixed plan written in advance.

What Makes Python Suitable for This Work

Definition

Python is widely used in data science because it combines the power of general-purpose programming with the ease of use of domain-specific scripting. Put simply, it is a full programming language that is also pleasant to write quick scripts in.

The source describes Python as having become the lingua franca for many data science applications. A lingua franca is a shared language that people with different native languages use to communicate. In that sense Python works as common ground: analysts, researchers, and engineers from different backgrounds can read and share the same code.

A major advantage, according to the source, is that you can interact directly with the code. You type a line, see its result immediately, and then type the next line based on what you saw. This can happen in a terminal, or in a tool such as the Jupyter Notebook, which the course covers in more detail later. Direct interaction matters because it lets you try an idea, look at the output, and decide the next step while the data is still open in front of you.

Libraries as a Ready-Made Toolbox

A library is a collection of prepared code that you can import and use instead of writing everything from scratch. Python offers libraries for data loading, visualization, statistics, natural language processing, image processing, and more. Together these form a large toolbox that gives data scientists both general-purpose and special-purpose functions.

The list in the source is not exhaustive. The phrase and more matters: the toolbox keeps growing beyond the examples it names. When you meet a new kind of data, the useful question is often whether a library already covers that task, rather than whether you must write the task yourself.

Tracing a Short Interactive Session

Before reading the example, decide what each line will leave behind. Pay attention to which values are stored along the way, because later lines depend on earlier ones. This example uses only built-in Python functions, so it runs without installing anything extra. Each line is typed at the prompt, and the result appears before you type the next line.

What do you think happens?

After typing the first line readings = [12, 15, 14, 18, 60], what does typing total = sum(readings) followed by total show?

Reveal answer

Answer:

119

sum adds every value in the list: 12 + 15 + 14 + 18 + 60 = 119. The list readings stays stored, so sum can use it on the next line.

Code walkthrough

readings = [12, 15, 14, 18, 60]
total = sum(readings)
average = total / len(readings)
average

1
readings = [12, 15, 14, 18, 60] creates a list and binds it to the name readings. Nothing is printed, because an assignment shows no result in the interactive prompt.
2
total = sum(readings) reads the stored list, adds its five values, and binds the result to total. Again, no output appears.
3
average = total / len(readings) uses total, and len(readings) counts five items. The division gives 119 / 5, which is 23.8, and that is stored in average.
4
Typing average alone shows the stored value 23.8. The name average carries over from line 3, which is the key idea of an interactive session: each line can depend on what earlier lines left behind.

Line 1 stores a list of five readings under the name readings
Line 2 adds the values in readings and stores the result in total
Line 3 divides total by the number of items to get an average
Line 4 shows the stored average on its own

Output

23.8

The diagram traces which names feed which. The list readings is used twice: once to compute the total and once to count the items. The value of total is then used to compute average. Because each name persists between lines, you can check any intermediate result at any time.

Changing One Value and Running Again

Suppose the average of 23.8 looks odd. The last reading, 60, is much larger than the others, so it may be a suspicious value. In the interactive style, you can change that one value and run the dependent lines again, without retyping the whole session from the top. The list is updated in place, and the later lines are repeated to see the new result.

Before

readings = [12, 15, 14, 18, 60]
total = sum(readings)
average = total / len(readings)
average

Output shows 23.8, with the suspicious value 60 in the list

After

readings[4] = 19
total = sum(readings)
average = total / len(readings)
average

Output shows 15.6, with the last value changed to 19 and the list otherwise unchanged

Notice the workflow this produces. You ran the import-free lines, checked the total, and only then decided whether to compute an average. When the average looked odd, you changed one value and ran the lines again. This short loop of change, run, and inspect is the kind of interaction the source has in mind, and it is the cycle from the earlier diagram played out at the scale of a single session.

Mistakes Beginners Make in Interactive Sessions

Common mistake
Treating the first result as the final answer
What it looks like
Computing one average from sensor data and reporting it without checking whether the odd value 60 is real.
Why it fails
The source describes analysis as iterative, so the first result is rarely the last. Each pass should inform the next question.
Do this instead
Inspect the output, decide what it suggests, and run another pass before you conclude anything.
Common mistake
Forgetting that names persist between lines
What it looks like
Typing total = sum(readings) in a fresh thought, but expecting readings to be empty because you assumed nothing was stored.
Why it fails
Each line can depend on earlier lines. The list readings stays stored until you change it.
Do this instead
Check the current value of a name by typing it alone before you rely on it.
Common mistake
Assuming an assignment prints a result
What it looks like
Typing readings = [12, 15, 14, 18, 60] and waiting for a list to appear on screen.
Why it fails
An assignment stores a value under a name; it does not display it. The result appears only when you type the name or an expression.
Do this instead
Type the name alone on its own line to see what was stored.
Common mistake
Believing the list of libraries is complete
What it looks like
Deciding that a task is impossible in Python because it is not among the examples in the source.
Why it fails
The source's list is not exhaustive, and the toolbox keeps growing beyond the examples it names.
Do this instead
Search for an existing library before writing a task from scratch.

Predicting a Session Before Running It

EASY

Read this session and predict what the last line shows before you check the answer. Lines: scores = [4, 8, 6]; count = len(scores); count; scores[0] = 10; sum(scores). What does the final sum show?

Hints

Work through the lines in order, and track what scores holds after each one.
len(scores) counts the items, so count stores 3, but changing scores[0] changes a value inside the list.
The final sum adds the updated list: 10, 8, and 6.

Your attempt (not saved)

What do you think happens?

What does the last line, sum(scores), show?