How It Works

One day ahead, from the days before

The tournament replays a window of origin dates, oldest first. On each one, every model is trained on the sales before that date, never on the date itself or any later day, and forecasts that one day for every dish. Then the forecast is compared with what was sold.

MAPE

MAPE = |actual − predicted| / actual × 100

For each dish on each origin date, on the 0–100 scale: 12.5 means the forecast was off by an eighth. A model’s MAPE for the day is the mean over the dishes that have one, and lower is better. A tile’s colour repeats its value in bands: 0–20, 20–25, 25–30, 30–40 and 40+.

ND, not defined

When the formula has no answer, the value is ND: a dish that sold nothing that day would divide by zero, and a closed day has no sales at all. ND is never zero, which would mean a perfect forecast, and it is left out of every average.

Fail and short

A model that raises an error or returns nothing for a day is marked fail. One with too little history before the origin date to forecast at all is marked short. Neither is ND: ND is about the day’s sales, fail and short are about the model.

Day winner, run winner

The day winner has the lowest MAPE that day; the run winner, the lowest average over the window. They can differ: a model can win many days by a little and lose a few by a lot. A tie goes to the model listed first in the roster. A run winner must also have a MAPE on at least 80 % of the window’s days.

How it is built

Two runs on the same inputs give the same result, byte for byte, and the result is hashed. What it was made with (versions, the machine, the timings) is recorded beside it, outside the hash, because it differs between machines that agree on the numbers. The engine is Python and the site TypeScript, and both are checked against one contract. The engine never runs here: every page is drawn ahead of time from a run it produced.