NumPy Arrays and Array Creation

The positions that held zero in the NumPy array are simply absent from the listing, because the sparse matrix does not store them.

Programming

A Diagonal Matrix Meets SciPy

This article starts with code rather than definitions. The program below builds a 4 by 4 NumPy array that has ones on its diagonal and zeros everywhere else, then converts that array into a SciPy sparse matrix. Run it in your head first, and pay attention to what the sparse matrix prints compared with the NumPy array printed just before it.

python

import numpy as np
from scipy import sparse

dense = np.eye(4)
print(dense)

csr = sparse.csr_matrix(dense)
print(csr)

import numpy as np
from scipy import sparse

dense = np.eye(4)
print(dense)

csr = sparse.csr_matrix(dense)
print(csr)

Output (expected)

  (0, 0)	1.0
  (1, 1)	1.0
  (2, 2)	1.0
  (3, 3)	1.0

Notes on the code

np.eye(4) builds the 4 by 4 array with ones on the diagonal
sparse.csr_matrix(dense) converts an existing NumPy array into a sparse matrix in CSR format
The output shown is the printed sparse matrix; the NumPy array printed first has sixteen positions, and only the four diagonal ones appear in the sparse listing

What do you think happens?

How many lines does the printed sparse matrix contain for this 4 by 4 diagonal array?

Reveal answer

Answer:

A sparse listing shows only positions that hold nonzero values. The 4 by 4 array has four ones on its diagonal and zeros in the other twelve positions, so four lines appear and the zero positions are absent.

SciPy is a collection of functions for scientific computing in Python. Within SciPy, the part most relevant to machine learning work is the scipy.sparse module, which provides sparse matrices. Those sparse matrices are one of the representations used for data in scikit-learn, the machine learning library this course relies on. When you see a sparse matrix in a scikit-learn workflow, it is coming from this module.

Reading the Printed Listing

Go back to the printed output from the first program. The sparse listing does not print every position of the 4 by 4 grid. Each line gives a position and a value, and only positions holding nonzero values appear. The positions that held zero in the NumPy array are simply absent from the listing, because the sparse matrix does not store them.

How do the coordinates and the value in one printed line map to a matrix entry?

Read each line as a single stored entry. The first part is the (row, column) position, and the value after it is the nonzero number held at that position. So the line for the position in row 2, column 2 with value 1.0 tells you that the matrix holds a 1.0 there. If you want to know what is at a position that has no line, the answer is zero, because the listing shows only nonzero positions. A missing line is a zero entry, not a missing row or column.

Why Sparse Storage Exists

A sparse matrix is the right tool whenever you want to store a two-dimensional array that contains mostly zeros. The phrase mostly zeros is the key condition. A dense array keeps a slot for every position, including the positions that hold zero. A sparse representation keeps track of the nonzero entries, and only those are stored.

How much data is stored for the 4 by 4 diagonal array in each representation?

Both representations hold the same mathematical matrix. The difference is what is stored. The dense form keeps all sixteen positions of the 4 by 4 grid, while the sparse form keeps only the four positions holding ones. When most positions are zero, the sparse form avoids storing a large number of values that carry no information.

Converting an Existing Array to CSR

The first creation route begins with a dense array that already exists in memory. The conversion takes that dense array and produces a sparse version of it. The conversion keeps the nonzero part and drops the zero positions from storage, so the work of describing the matrix was done by the dense array and the conversion only kept what mattered.

Which positions of the dense array remain as stored entries after conversion to CSR?

python

import numpy as np
from scipy import sparse

dense = np.array([
    [5, 0, 0],
    [0, 0, 0],
    [0, 8, 0]
])

csr = sparse.csr_matrix(dense)
print(csr)
print(csr.nnz)

import numpy as np
from scipy import sparse

dense = np.array([
    [5, 0, 0],
    [0, 0, 0],
    [0, 8, 0]
])

csr = sparse.csr_matrix(dense)
print(csr)
print(csr.nnz)

Output (expected)

  (0, 0)	5
  (2, 1)	8
2

Notes on the code

The dense array has three nonzero values and six zeros
csr_matrix(dense) keeps only the two nonzero entries
The attribute nnz reports the number of stored nonzero entries

Worked example

Predicting the listing after a conversion

A dense array has the values 5 at (0, 0), 8 at (2, 1), and zeros elsewhere. What lines appear when the array is converted to CSR and printed?

1
List the nonzero positions
Only (0, 0) with value 5 and (2, 1) with value 8 hold nonzero values.
2
Drop the zero positions
The seven positions holding zero have no entry in the sparse matrix, so they do not appear in the listing.
3
Print one line per stored entry
Each stored nonzero entry produces one line of the form (row, column) followed by its value.

Result

Two lines appear: (0, 0) with 5 and (2, 1) with 8. The missing positions are zeros.

Building a Matrix From Index Lists in COO

The source shows a second route for cases where the dense array would not fit in memory. Instead of starting from a full array, you build the sparse matrix directly from its contents. The COO format takes three pieces of information: a list of values, a list of row indices, and a list of column indices. Each position in those three lists describes one stored entry. The dense grid never exists at any point in this route.

Given the parallel data, row, and column arrays, which matrix position does each stored value occupy?

python

import numpy as np
from scipy import sparse

data = np.array([4, 9, 2])
row = np.array([0, 2, 3])
col = np.array([1, 0, 3])

coo = sparse.coo_matrix((data, (row, col)), shape=(4, 4))
print(coo)
print(coo.toarray())

import numpy as np
from scipy import sparse

data = np.array([4, 9, 2])
row = np.array([0, 2, 3])
col = np.array([1, 0, 3])

coo = sparse.coo_matrix((data, (row, col)), shape=(4, 4))
print(coo)
print(coo.toarray())

Output (expected)

  (0, 1)	4
  (2, 0)	9
  (3, 3)	2
[[0 4 0 0]
 [0 0 0 0]
 [9 0 0 0]
 [0 0 0 2]]

Notes on the code

data, row, and col are parallel lists: position i in each one describes the same stored entry
coo_matrix((data, (row, col)), shape=(4, 4)) builds the matrix without ever creating a full 4 by 4 array
toarray() expands the sparse matrix back into a full array so you can check every position

Both routes answer the same question: which entries are nonzero, and what are their values. The difference lies in the entry point. If your data starts as an existing array, use the CSR conversion. If your data starts as a list of nonzero entries, use COO, which is the route to choose when the dense form would be too large to hold.

Mistakes With Sparse Listings

When a sparse matrix result looks wrong, the faulty step is usually one of three: how the matrix was created, what the printed listing is showing, or what you expected it to contain. Work through these in order so you do not blame the wrong one.

Common mistake
Treating a missing line as a missing row or column
What it looks like
The listing has no line for (1, 1) and the reader concludes that row 1 was lost.
Why it fails
A sparse listing shows only nonzero positions. A missing line means the position holds zero.
Do this instead
Check the matrix shape, or expand with toarray(), to confirm that every position is still present and that the absent ones are zeros.
Common mistake
Expecting the dense array to appear in the sparse output
What it looks like
After converting a 4 by 4 array, the reader expects sixteen lines in the printout.
Why it fails
Only the nonzero positions are stored, so the sparse listing has one line per stored entry.
Do this instead
Count the nonzero values in the source array; that number should match the number of lines.
Common mistake
Assuming COO input is stored exactly as typed when positions repeat
What it looks like
Two COO entries share the position (1, 2) with values 5 and 3, and the reader expects two separate values.
Why it fails
When the COO matrix is converted to a full array, duplicate positions are combined into one total.
Do this instead
Remove or combine duplicate positions in the input lists before building the matrix if you need one value per position.
Common mistake
Mixing up which route fits the data: using CSR conversion when the dense array would not fit in memory

Tracing an Unexpected Result

Worked example

Locating a surprising value in a listing

A printed sparse listing shows the lines (0, 0) with 1.0 and (2, 2) with 1.0. A reader expects a value at position (1, 1) and asks where it went.

1
Identify the listed positions
The listing names two stored entries: (0, 0) and (2, 2).
2
Check for (1, 1)
No line starts with (1, 1), so that position is not stored.
3
Interpret the absence
The listing shows only nonzero positions, so the matrix holds zero at (1, 1). The position was not lost.
4
Check the source
If the source array really holds a nonzero value at (1, 1), the mistake is in the creation step, not the listing.

Result

The value at (1, 1) is zero. The listing is correct, and the expectation or the source array is what needs checking.

EASY

Build a COO matrix with shape (3, 3) using data [7, 5], row [0, 2], and col [0, 2]. Before you run the code, write down which positions you expect to appear in the printed listing and which positions you expect to be absent. Then run it and compare.

Hints

Position i in the row list and the col list describes the same entry as position i in the data list.
Only positions holding a stored value appear in the printed listing.
Use toarray() to check your expectation against every position of the 3 by 3 matrix.

Your attempt (not saved)

Key Takeaways

A sparse matrix stores only its nonzero entries, which is useful when a two-dimensional array contains mostly zeros.
SciPy's scipy.sparse module provides the sparse matrices used for data in scikit-learn.
CSR format is created by converting an existing NumPy array, which keeps the nonzero part of that array.
COO format is built from a list of values and two index lists, which is the route to use when the dense array would be too large to hold.
A printed sparse listing shows only nonzero positions, so a missing line means a zero entry rather than a missing row or column.

Related concepts

Dense NumPy Arrays and Indexing

More concepts starting with N

A Diagonal Matrix Meets SciPy

Reading the Printed Listing

Why Sparse Storage Exists

Converting an Existing Array to CSR

Building a Matrix From Index Lists in COO

Mistakes With Sparse Listings

Tracing an Unexpected Result

Key Takeaways

Related Topics

Related concepts