Concepts / Introduction to NumPy Libraries for Machine Learning

Introduction to NumPy Libraries for Machine Learning

NumPy provides more than arrays.

  • Programming

Predicting a Printed Array

Before any machine learning model can learn from data, that data has to be in a form the library can accept. In scikit-learn, that form is the NumPy array. This article works code first: you read a short program, predict what it prints, and then use the explanation to check your prediction. The gap between your prediction and the real output is where understanding forms, so commit to a guess before you read on.

What do you think happens?

The program calls np.array on a list whose two items are each a list of three numbers: [[1, 2, 3], [4, 5, 6]]. What does it print?

  • [1 2 3 4 5 6], one flat row of six numbers

  • [[1 2 3]
     [4 5 6]], two rows with three numbers each
  • [[1 4]
     [2 5]
     [3 6]], three rows with two numbers each
Reveal answer
Answer:
[[1 2 3]
 [4 5 6]], two rows with three numbers each

Each inner list becomes one row of the resulting two-dimensional array. The outer list supplies the two rows, and each row contributes three elements, so the printed grid keeps the same nesting as the input.

Building an Array from Nested Lists

The function np.array takes a Python list and builds a NumPy array from it. When the list is nested, each inner list becomes one row of the result. The program below uses the same two-row, three-column structure described in the source material, so the value 1 lands in the first row and first column, and the value 6 lands in the second row and third column.

python
Output (expected)
[[1 2 3]
 [4 5 6]]
Notes on the code
  • np.array converts the nested Python list into an ndarray object
  • Each inner list [1, 2, 3] and [4, 5, 6] becomes one row of the array
  • print shows the grid with rows in brackets and values separated by spaces
row 0, col 01row 1, col 04row 0, col 12row 1, col 15row 0, col 23row 1, col 26same rowsame rownext rowsame rowsame row
Which nested list element lands at which row and column of the array, and how does the printed grid reflect that nesting?

Read the diagram as a map from positions to values. Each cell is labeled with a zero-based row number and a zero-based column number, because Python indexing starts at zero. The first list supplies the entire top row, and the second list supplies the entire bottom row. The printed grid mirrors that structure: the first bracketed line holds the first inner list, and the second bracketed line holds the second.

What the ndarray Class Is

Definition

NumPy is a fundamental package for scientific computing in Python. Its core functionality is the ndarray class. The name is shorthand for an n-dimensional array, which is a multidimensional container of values. When np.array runs, it builds an object of this class.

NumPy provides more than arrays. The source describes three broad areas: functionality for multidimensional arrays, high-level mathematical functions, and pseudorandom number generators. This article focuses only on creating arrays, identifying what kind of object you have, and reasoning about it when something goes wrong. The other areas build on the same array object.

Key point

Objects of the ndarray class are called NumPy arrays, or simply arrays. You will see this convention throughout scikit-learn material, so the two names mean the same thing in this course.

Why Every Element Shares One Type

A Python list is happy to hold mixed types in a single container: an integer, a string, and another integer can sit side by side. A NumPy array does not work that way. All elements of a NumPy array must be of the same type. This rule is the most common source of confusing results, because it changes what the array holds when you pass in mixed input.

python
Output (expected)
[1, 'two', 3]
['1' 'two' '3']
Notes on the code
  • The original list keeps each item with its own type, so the integers print without quotes
  • np.array must choose one type for every element
  • The integers 1 and 3 come out of the array as text, so they print in quotes
1 as intoriginal item1 as textstored elementtwo as stroriginal itemtwo as textstored element3 as intoriginal item3 as textstored elementcoerced by np.arraycoerced by np.arraycoerced by np.array
What happens to each element when a list containing mixed types is passed to np.array, and why do the printed values change?

Tracing the Path into scikit-learn

scikit-learn takes in data in the form of NumPy arrays. This is a requirement rather than a stylistic preference. Any data you want to use with scikit-learn has to be converted to a NumPy array first, which makes the array the fundamental data structure of the library. The conversion step belongs in your workflow before you call any scikit-learn function.

Raw Python listsdata as listsnp.array callconversion stepndarraysame type per elementscikit-learnestimatoraccepts NumPy arrayspassed inreturnspassed to
At which step in the path from raw data to the scikit-learn estimator does the data stop being a list and become an ndarray, and where could a type or shape mismatch first appear?

In this path, the data stops being a list at the np.array call. Everything after that point is an ndarray. A mismatch can first appear at that conversion, because the list you handed in may contain mixed types or an uneven shape. It can also appear later, if the data was never converted and the estimator receives something other than an array.

Diagnosing Where a Mismatch Begins

When your predicted output and the real output diverge, the location of the divergence usually points to one of three places: the input list you handed to np.array, the call that created the array, or the step that printed or used it. Trace the program in that order, and compare what you expected at each line with what the code actually produces.

Worked example

Locating a divergence in a short program

You predicted that the program below prints the numbers 1 2 3 4 5 6 as a grid of two rows, but you suspect the output is different. Where does the divergence begin?

  1. 1
    Check the input list

    The list is [[1, 2, 3], [4, 'five', 6]]. It mixes integers with a string. This is the first possible source of a mismatch.

  2. 2
    Check the call that created the array

    np.array must convert every element to one common type. The string forces the integers in the array to become text, so the array no longer holds numbers you can compute with.

  3. 3
    Check the printed output

    The printed grid shows the integers in quotes, which confirms that the conversion happened at the array-creation step and not during printing.

Result

The divergence begins at the input list, because it mixes types. The np.array call then applies the one-type rule and converts the integers. Printing only reveals the result of that conversion.

Mistakes Beginners Make with np.array

  • Common mistake

    Assuming a mixed-type list stays mixed inside the array

    What it looks like

    np.array([1, "two", 3]) prints ['1' 'two' '3'] instead of the original values

    Why it fails

    An ndarray requires all of its elements to share one type, so NumPy converts the elements to a common type.

    Do this instead

    Make the input uniform before creating the array, and inspect the printed output to confirm the types you expect.

  • Common mistake

    Reading the printed grid as columns instead of rows

    What it looks like

    Expecting the value 4 to sit in the first column of np.array([[1, 2, 3], [4, 5, 6]])

    Why it fails

    Each inner list becomes one row, so 4 sits in the second row and first column.

    Do this instead

    Map each inner list to one printed line, then count positions from zero.

  • Common mistake

    Passing raw lists to scikit-learn without converting them

    Why it fails

    scikit-learn takes data in the form of NumPy arrays, so the conversion is a requirement.

    Do this instead

    Convert the data to a NumPy array with np.array before calling any scikit-learn function.

Practice: Predict Then Check

EASY

Without running the code, predict the printed output of np.array([[1, 2], [3, 4], [5, 6]]). Then state which element sits in row 2 and column 1 using zero-based indexing, and explain which of the three diagnosis steps you would check first if the printed grid did not match your prediction.

Hints
  • Each inner list becomes one row, so this input has three rows of two elements each.
  • Zero-based indexing means the first row is row 0, so row 2 is the third row.
  • Start with the input list, then the call that created the array, then the printed output.

Key Takeaways

  • np.array turns nested Python lists into a NumPy array, where each inner list forms one row of a two-dimensional grid.
  • Objects of the ndarray class are called NumPy arrays, and all of their elements must share one type, so mixed-type input can be silently converted.
  • scikit-learn takes in data as NumPy arrays, so any data must be converted with np.array before it is passed to a scikit-learn function.
  • When predicted and real output diverge, trace the input list, the array-creation call, and the printing step in that order to find where the mismatch begins.

Related concepts

More concepts starting with I