Time Series Resampling in pandas
In this case study the resampled series is printed, and the printout shows the first five timestamps, all on August 1, 2015.
A Bike Station Outside One Home
Imagine a bike rental station on the street outside your home. Before you leave in the morning, you would like to know whether a bike will be available when you need one. This is the situation behind the case study in this article. Citi Bike operates a network of rental stations across New York and uses a subscription system. Its rental data is made public in an anonymized form, and people have analyzed it in many ways.
The question is simple to state: given a day and a time, how many people will rent a bike from a station in front of the house? If the answer is high, the person living there may find that no bikes are left when they need one. The source describes this kind of problem as closer to common sense than to specialized expertise, which is why it makes a good first forecasting exercise. The tool that does the work is a pipeline of pandas operations, and you will follow one station through it.
The pipeline has three main moves. You load one station's data into a table, reshape it into a coarser time series, and then split it into a training period and a test period. At each move the data looks different, so the article traces what the data looks like after every step. Keeping track of that shape is the habit that will later help you find a surprising result.
Loading One Station's August Rentals
The first step loads the data for August 2015 for one particular station as a pandas DataFrame. The source performs this loading with a helper from the mglearn package. The line that does it is a single assignment: the result of the function call is stored in a variable named citibike. Nothing else happens in that line, so the variable is the starting point for every later step.
Inspection comes next, and the most useful inspection is to read the output of each step rather than to trust it. In this case study the resampled series is printed, and the printout shows the first five timestamps, all on August 1, 2015. Each timestamp is paired with a number, which is the value stored for that three-hour window. The printout also names the series, which is one, and reports its dtype as float64.
Resampling Hourly Rentals into Three-Hour Bins
Resampling changes how often the data has a value. The source resamples the station's data into three-hour intervals so that the main trends for each day become visible. A series with one value per hour would have 24 values per day, and it would show more noise from hour to hour. The three-hour version compresses each day into a small number of points, so the daily shape is easier to read.
The arithmetic is direct. Each day has 24 hours, and each three-hour window covers 3 of them, so each day contains $\frac{24}{3} = 8$ three-hour values. Those 8 values are the points the rest of the case study works with. The source states this count explicitly, and it is the number that makes the later split calculations come out cleanly.
The diagram above is a generated illustration rather than a figure from the source. It assumes that each day's windows start at midnight, which is one common way to group hours. Under that assumption, the 24 hourly positions collapse into 8 windows, and each window carries one value that summarizes the three hours it covers. The source does not spell out how that summary is computed, so the article keeps to the count of values and the meaning of each window.
Counting Values Per Day Before and After Resampling
How many values does one day contain when the data is hourly, and how many when it is resampled into three-hour intervals?
- 1Hourly values
A day has 24 hours, and an hourly series has one value per hour, so there are 24 values per day.
- 2Hours per window
Each three-hour window spans 3 hours, so the day is cut into windows of 3 hours each.
- 3Windows per day
Dividing the 24 hours by 3 hours per window gives $\frac{24}{3} = 8$ windows.
- 4Result check
The resampled series has 8 values per day, one third of the hourly count of 24 divided by 3 applied to the hours. Each value stands for three hours of data.
The hourly series has 24 values per day; the three-hour series has 8 values per day.
Reading the Plot for Repeating Shapes
After loading and resampling, the source plots the rental frequencies for the whole month. The plot shows the number of bike rentals over time for the selected station. Its purpose is pattern recognition. A single printed row cannot show whether a day is busy or quiet, but a plot across thirty-one days can.
When you read a plot like this, look for repetition. Ask whether the three-hour pattern seems to recur from one day to the next, and whether certain times of day tend to have higher counts. Those repeating shapes are the main trends that the resampling step was designed to expose. The prediction task depends on whether such trends are stable enough to learn from, so the plot is a check on whether the rest of the pipeline is worth trusting.
Splitting the Series by Time
Finding Where an Unexpected Result Starts
When a result differs from what you expected, the useful question is where in the pipeline the difference first appears. The steps in this case study are loading, resampling, plotting, and splitting, with the model consuming the split at the end. Checking each intermediate output in turn usually locates the problem faster than inspecting the final model. If you only look at the final accuracy, you cannot tell whether the error came from the data, the resampling, or the split.
Start at the top of the pipeline. The Freq line in the resampled output shows the sampling frequency, and it is the first thing to check if a result looks wrong. Then compare the first timestamps and values with what you expect for the station, confirm that the plot shows the pattern you would predict, and verify the split counts. Each check narrows the search, so the stage that first disagrees is the one to investigate.
Common Mistakes When Reading Resampled Data
Common mistake Assuming resampling keeps the hourly count of values
What it looks likeExpecting 24 values per day after resampling into three-hour intervals.
Why it failsThree-hour windows each cover 3 hours, so a day holds 24 divided by 3, which is 8 values.
Do this insteadCount the windows per day with the window length, then use that count for every later calculation.
Common mistake Splitting the series at random
What it looks likeShuffling the 248 values and taking 184 at random for training.
Why it failsA random split puts later rentals into the training set, so the model is not tested the way it will be used to predict tomorrow.
Do this insteadSplit by time: the first 184 points train the model and the remaining 64 points test it.
Common mistake Reading float64 values as fractional rentals
What it looks likeConcluding that a bin with 2.0 rentals contains a partial rental.
Why it failsThe float64 dtype is a storage type; the values still represent counts of rentals in each window.
Do this insteadTreat the values as counts when you interpret them, and keep the dtype in mind if you change the data later.
Common mistake Inspecting only the final model output
Why it failsA poor final result can come from loading, resampling, or splitting, and the final output alone does not show which one.
Do this insteadCheck each intermediate output in order, starting with the Freq line of the resampled series.
Practice: Predict the Test Set Size
What do you think happens?
If the training set holds the first 184 three-hour values, how many values are left for the test set when the month has 31 days of 8 values each?
Reveal answer
64 values, covering 8 days (248 minus 184 is 64, and 64 divided by 8 is 8).
The month holds 31 times 8, which is 248 values. The 184 training values cover 23 days, so the remaining 64 values cover the last 8 days of August.
Suppose the resampling step is changed to four-hour intervals while the rest of the case study stays the same. Work out how many values each day would contain, and then how many training days the first 184 values would now cover. Check your answer by comparing it with the three-hour case.
Hints
- Divide 24 hours by the window length of 4 hours to get the values per day.
- Divide 184 by your new values-per-day count to find the number of training days.
Key Takeaways from the Case Study
The case study moves one station's rental history through three steps: load, reshape, and split. Resampling into three-hour intervals turns 24 hourly values per day into 8 values per day, which makes each day's trends easier to read. The split is made by time, with 184 points (23 days) for training and 64 points (8 days) for testing, and the boundaries follow from the 8-values-per-day count. When something looks wrong, check each stage in order, starting with the sampling frequency in the Freq line.
At a glance
- Loading one station's August 2015 rentals into a pandas
DataFrameis the starting point, and its output should be inspected rather than trusted. - Resampling into three-hour intervals gives 8 values per day instead of 24, which makes each day's main trends easier to see.
- The data is split by time: the first 184 points (23 days) train the model and the remaining 64 points (8 days) test it.
- When a result looks wrong, check the output of each pipeline stage in order, beginning with the Freq line, to find where the difference first appears.