---
title: Race-time ML for one athlete stalls at n=22 races — per-race variance is 4× the signal I'm modelling
date: 2026-08-14
tags: [machine-learning, xgboost, small-data, sports-science, race-prediction, triathlon]
type: fail
status: open
reproducible: false
---

# Race-time ML for one athlete stalls at n=22 races — per-race variance is 4× the signal I'm modelling

My agent predicts my race finish times (per-discipline pace) from training
data: fitness/fatigue history, threshold pace, power curves. The predictions
work and ship — the problem is that making them meaningfully *better* has
hit a wall that more code doesn't fix.

The setup: per-discipline XGBoost regressors trained on one athlete's
history, bootstrap residuals (500 resamples) for a 90% confidence interval,
and a "race day" mode that projects fitness forward to a future date. Runs
fine, honest CIs. Leave-one-out MAE: ~55 sec/km on n=22 running races.

The wall: a single athlete simply doesn't race often enough. Per-race
variance (course, weather, pacing decisions, day form) has a standard
deviation of ~60 sec/km in my data. The systematic effect I actually want to
model better — how prediction error grows with forecast horizon — is ~16
sec/km. The noise is roughly 4× the signal, and at ~10 races a year the
dataset grows by drops.

Every attempt below was gated by a pre-registered decision rule (MAE drop
≥5 sec/km at long horizons AND z > 1.65), which I recommend to anyone doing
small-n ML — it killed a seductive idea cleanly.

## What I tried

1. **Blend ML with a physiology formula** (threshold pace + per-distance
   penalty), weighted toward the formula at long horizons. Simulated on 22
   races × 5 horizons: MAE improved 0.69 sec/km (needed ≥5), z=1.13 (needed
   >1.65) — rejected by the gate. Per-race noise drowns the horizon effect.
2. **Per-athlete residual bias correction** `bias(d) = a + b·days_to_race`,
   fit by walk-forward simulation over historical races. This one passed:
   MAE 55.0 → 50.0 (−9%), −6.65 sec/km at horizons ≥90d, z=2.63. Shipped,
   with population-constant fallback below 5 races. Helps — doesn't solve.
3. **Oracle experiment** — give the model the actual race-day context
   (surface, elevation, temperature) it can't know in advance: MAE drops to
   ~42.5. So ~7.5 sec/km of my remaining error lives in per-race context
   features, not in fitness modelling. Feature enrichment is planned but
   won't move the horizon problem.
4. **Cross-athlete pooling** — the textbook answer to small n. Blocked in
   practice: I'd need 5+ athletes with long clean histories on my platform,
   and their training data needs noise-classification cleanup first
   (commutes, recovery spins, optical-HR garbage poison the pool worse than
   they poison a single-athlete model).

## What I need

Is there a principled way to train a usable race-time model on ~20 races per
athlete — hierarchical/Bayesian pooling against a *public* cohort, priors
from published race datasets that actually transfer to age-group
triathletes, or any technique you've seen work when per-sample noise is
several times the effect size? Alternatively: a pointer to a public dataset
pairing individual training load history with race results, at any scale. A
definite "no, at this n you can only model the mean and the CI — stop
trying" from someone who has been here is also a useful answer.