Creating and Inspecting pandas DataFrames
pd.DataFrame(data) builds a table from a dictionary: each key becomes a column and each list supplies that column's values row by row.
Why Start From a Dictionary
A table of people, with a name and an age for each person, is one of the most common shapes data takes. In pandas, that table is called a DataFrame. This article follows one short, complete program from start to finish. You will build a small table, look at it, and then pull out only the rows that meet a condition. The goal is not to memorize functions. The goal is to follow what the code does at each stage, so you can tell where a result matches your expectation and where it does not.
How Dictionary Keys Become Columns
A pandas DataFrame is created by calling pd.DataFrame() and passing it a Python dictionary. Each key in the dictionary becomes a column name. Each list stored under a key supplies the values for that column, read from top to bottom. So a key such as 'Name' becomes the heading of a column, and the list stored under 'Name' fills that column one row at a time.
pd.DataFrame(data) builds a table from a dictionary: each key becomes a column, and each list supplies that column's values row by row.
It helps to read the dictionary as a column-first picture. Each key is one column, and the list under it is the contents of that column. The table is built from the dictionary's columns. The keys are not read as rows. If you picture the dictionary as a set of vertical strips, one per key, then the DataFrame is those strips placed side by side.
Row Alignment Across Lists
Because each list supplies one column, the lists must line up. Position 0 in every list belongs to the same row, so the first row describes the first person across every column. Position 1 across every list forms the second row, and so on. This is why every list in the dictionary must be the same length: each position across the lists forms one row, and a missing position would leave a row incomplete.
Building the First DataFrame
The program below imports pandas, defines a dictionary with two keys, creates the DataFrame, and displays it. The display call comes from IPython.display, which allows DataFrames to be pretty printed in a Jupyter notebook. The data here is an illustrative example written for this article: the keys are Name and Age, and each list has four items.
Name Age
0 Alice 25
1 Bob 32
2 Carol 41
3 David 28- The key 'Name' becomes the first column header.
- The key 'Age' becomes the second column header.
- Position 0 across both lists forms row 0: Alice, 25.
- The index labels 0 to 3 are assigned automatically by pandas.
Check the display against the rules you just read. The column headers Name and Age come from the keys. Each row is one position across the two lists. Row 1 is Bob and 32 because position 1 in the Name list is Bob and position 1 in the Age list is 32.
Selecting Rows With a Condition
Before the next piece of code runs, look at the table you just built and decide what a filter should do. The next task is to keep only the rows where the Age column is greater than 30. Make your prediction first, then compare it with the answer.
What do you think happens?
If you keep only the rows where Age is greater than 30, which people remain?
Reveal answer
Bob and Carol
Bob is 32 and Carol is 41, so both are greater than 30. Alice is 25 and David is 28, so both are excluded.
Conditional selection is done with boolean indexing. The expression data_pandas.Age > 30 is evaluated for every row. It produces a series of True or False values, one per row. The result is a mask: a list of truth values that lines up with the rows of the table. When that series is placed inside square brackets after the DataFrame, pandas keeps only the rows where the value is True.
Filtering the Table Step by Step
The program below first builds the mask on its own, so you can inspect it, and then uses the mask to select rows. The comment states the intent: select all rows that have an age column greater than 30. The code expresses that intent, so when you check a filter, compare the code against the comment, then compare the displayed rows against your prediction.
0 False
1 True
2 True
3 False
Name: Age, dtype: bool
Name Age
1 Bob 32
2 Carol 41- The comparison runs once per row and produces one True or False for each.
- The brackets keep only the rows whose mask value is True.
- Index labels 1 and 2 are kept, so the selected rows still show their original positions.
display(data_pandas)Four rows: Alice, Bob, Carol, David
display(data_pandas[data_pandas.Age > 30])Two rows: Bob and Carol
The Original Table Stays Unchanged
The selection returns a new result. It does not change the original DataFrame. After the filter above runs, data_pandas still contains all four people. The smaller table exists only in the variable selected. If you want the variable data_pandas itself to hold only the filtered rows, you must assign the result back to it.
Name Age
1 Bob 32
2 Carol 41- The assignment replaces the original variable with the filtered table.
- Running this line again on the new table would filter the already filtered rows.
Mistakes When Creating and Filtering
Most errors in this kind of code come from a small set of habits. Each entry below names the mistake, shows what it looks like, and gives the fix.
Common mistake Building the dictionary with lists of unequal length
What it looks like{'Name': ['Alice', 'Bob'], 'Age': [25, 32, 41]}Why it failsEach position across the lists forms one row, so a list that is shorter or longer leaves rows incomplete.
Do this insteadCheck that every list has the same number of items before calling pd.DataFrame().
Common mistake Reading the dictionary keys as rows
What it looks likeExpecting the key 'Name' to form a row of text across the table.
Why it failsEach key is a column. The list under it supplies that column's values top to bottom.
Do this insteadRead the dictionary column-first: one key, one column header, one vertical list of values.
Common mistake Expecting the original DataFrame to shrink after a filter
What it looks likedata_pandas[data_pandas.Age > 30] is run, then data_pandas is displayed and still shows four rows.
Why it failsThe selection returns a new result and leaves the original unchanged.
Do this insteadAssign the result to a variable, or to the same variable name if you intend to replace the original table.
Common mistake Skipping the prediction and comparing only at the end
What it looks likeRunning the filter and accepting whatever rows appear.
Why it failsWithout a prediction there is no expectation to compare against, so a wrong result is hard to spot.
Do this insteadTrace the mask row by row first, write down which rows should remain, then compare with the output.
Predict Before You Run
Using the same DataFrame, predict which rows remain after the filter data_pandas[data_pandas.Age > 25]. Write your predicted list of names first. Then run the code and compare. Explain any row that surprised you by tracing its Age value through the comparison.
Hints
- Compare each Age value with 25 separately, one row at a time.
- Remember that the comparison is strictly greater than, so a value equal to 25 would not be kept.
- The original table is unchanged, so start again from the four-row table.
Key Takeaways
pd.DataFrame(data)builds a table from a dictionary. Each key becomes a column, and each list supplies that column's values row by row.- Position 0 across every list forms row 0, so all lists in the dictionary must have the same length.
- A condition such as
data_pandas.Age > 30 yields True or False for each row, and brackets keep only the True rows. - Conditional selection returns a new result and leaves the original
DataFrameunchanged unless you assign the result back to the variable. - Predicting which rows should remain before running the code, then comparing with the output, is the fastest way to locate where a result diverges from expectation.