Working with Time-Indexed Data in pandas

Andreas wants to know whether any bikes will be left at the rental station in front of his house. That is a practical question, and this case study answers it by working with real rental data rather than a formula. In New York, Citi Bike runs a network of bicycle rental stations spread across the city, operating on a subscription system. Because the bike rental data is made public in an anonymized form, it can be analyzed in many different ways.

The prediction task is precise: for a given time and day, estimate how many people will rent a bike at that station. Everyday knowledge helps here. Anyone who has watched a busy street knows that rentals rise and fall with the time of day. The rest of this article follows the code that prepares the data for that prediction, and it asks you to check the output of each step as you go, so that you can tell where a result would go wrong if it did.

The first step loads the August 2015 data for one particular station as a pandas DataFrame. A helper function from the mglearn library returns that data, and the result is assigned to a variable named citibike. At this point nothing about the prediction happens yet. The line only brings the data into memory, so it is worth reading it as a loading step and nothing more.

Once the data is loaded, the cheapest way to check it is to look at the first rows. The head() method shows those rows together with their time labels, so you can confirm that the data is the station you intended before going further. The diagram below shows the shape of the printed table for the first five intervals.

The printed table has two parts that matter. The index is labeled starttime, and each row carries one value. Each row covers one three-hour interval, and the value is the figure the data holds for that interval. The values are shown with a decimal point, such as 3.0 and 41.0. That tells you the column is stored as floating-point numbers, even though the quantities being counted are rentals.

Each time label is the start of its interval, and each value belongs to the label it is paired with. The pairing is the single most important thing to read correctly in the printed table. If you attach a value to the wrong label, you will reason about the wrong part of the day. For example, attaching 41.0 to the 06:00 label would make you think the station was busy at 06:00, when that value belongs to a different interval.

The source states that the data is resampled into three-hour intervals to obtain the main trends. Resampling means regrouping observations that are tied to time into new, regular time buckets, and then working with one value per bucket. Here the bucket width is three hours, which is the meaning of the Freq: 3H label in the printed output. A bucket that begins at midnight, then 03:00, then 06:00, and so on, gives one row per interval across each day.

The effect on the rows is a change in granularity. Many individual rental observations that fall inside one three-hour window are replaced by a single row for that window. The row's value is the count the data holds for that interval. The diagram below shows two adjacent buckets, with several raw rental observations feeding each bucket and one value coming out.

The reason for this choice is the trend, not the detail. A single bucket that covers three hours smooths out minute-to-minute noise and leaves the daily pattern visible. The source describes the goal as obtaining the main trends for each day. In other words, the resampled table answers how activity moves across a day, which is the kind of question Andreas needs answered.

After loading and resampling, the next move is visual. The source shows a plot of the rental frequencies for the whole month, with the number of bike rentals drawn over time for the selected station. Each point on the plot corresponds to one of the resampled three-hour rows, so the plot is a picture of the same table you printed earlier, extended across August 2015. Peaks and quiet stretches become visible as a pattern across days, which a list of printed numbers hides.

The order of these stages is part of the lesson. Each stage consumes the output of the stage before it. The loaded DataFrame feeds the resampling step, and the resampled table feeds the plot. The diagram below traces that chain from left to right so you can see what each stage hands to the next.

When a result diverges from what you expected, the efficient move is to narrow the search instead of rereading all the code at once. Work through the stages in order, using the printed table as the cheapest checkpoint.