0:0017:08

I know where you wanna go next summer - Demand Forecasting at FlixBus

17:08 2021

A forecast nobody trusts creates no value, however good the model

OpenStreetMap covers small German towns in detail but leaves much of the US thin, and in a 2020 survey almost 90% of its contributors identified as male: gaps that flow straight into any forecast built on that data. Miriam, a former data scientist who now leads network planning products at FlixBus, forecasts demand across 30 million possible connections. What does it take for planners to stake a whole network on those forecasts?

Learn three hard-won lessons for turning machine learning forecasts into decisions people trust

Watch on YouTube

In short

Miriam, Domain Product Officer at FlixBus, explains how her teams forecast passenger demand between cities across a network of 30 million possible connections to plan bus supply a year ahead. She argues that successful applied machine learning depends on solid data groundwork, an experimental and iterative development process, and early involvement of the people who must trust and use the forecasts.

“Um, but basically like in general, if you feed like garbage data into your model, no matter how good that model is, you will get just a garbage forecast.”
“So it's not that you build one model, but you build hundreds of candidate models.”

What you will learn

  • Frame demand forecasting as one input to a larger human decision-making system rather than an end in itself.
  • Identify the external factors, such as population, distance and tourist attractions, that drive demand between locations.
  • Spot non-random gaps and bias in crowdsourced data before they produce biased forecasts.
  • Plan for an experimental process with many candidate models, dead ends and structural breaks such as the pandemic.
  • Build user trust by involving domain experts early, running feedback rounds and using explainable machine learning.

Details

Level
Intermediate
Format
Talk
Technical depth
Introductory technical
For
Mid-career, Senior individual contributor, People manager
Length
17:08
Published
June 2021

Questions this talk answers

Why does FlixBus need automated demand forecasting?

As the network grew to around 30 million possible connections with a limited number of buses, manual planning no longer scaled. Planners need a forecast in space and time to decide which connections to serve and how many buses to supply.

What factors influence demand between two cities?

The population of the two cities and the distance between them are core factors: the bigger and closer the cities, the more demand. Tourist attractions and many other factors add on top of that.

Why is data groundwork so important in machine learning projects?

Garbage data produces a garbage forecast no matter how good the model is. Crowdsourced sources like OpenStreetMap have non-random gaps, such as less detail in the US than in Germany, which must be handled to avoid biased forecasts.

How did the pandemic affect FlixBus's forecasting models?

The lockdown created a structural break in the travel data. The team has to account for that special period when training and evaluating models on historic data.

How do you get planners to trust and use a forecasting model?

Involve them early as part of the development process through feedback rounds, interviews and testing phases. Explainable machine learning methods also help show why a forecast has a certain value.

Topics

Related talks

Browse the library