Train Delay Prediction

RRStudioXGBoost

The problem

Predict train delays from 1,100 real-world operational records. Small dataset, real operational noise, and a hard constraint: the model had to be built and iterated on over a structured multi-week cycle against two other competing teams working the same problem.

My approach

  • Architecture: a single XGBoost regression model, kept deliberately simple given the dataset size. With only 1,100 records, a more complex model architecture would have meant fitting to noise rather than signal, so the design decision was to invest effort in feature engineering rather than model complexity.
  • First pass: established a baseline XGBoost model with minimal preprocessing, MSE of 30,700. This baseline was the reference point every later change had to justify itself against.
  • The mistake: naive outlier removal looked like a safe, standard step. It increased MSE by 129%. Investigating why revealed that what looked like outliers were often legitimate operational delays, real signal, not noise, and removing them stripped the model of exactly the variance it needed to learn from.
  • The fix: reversed the outlier removal, and moved to selective feature handling instead, preserving meaningful variance rather than discarding anything that looked unusual.
  • Technical decision: applied cyclical sine-cosine encoding to time-based features. A clock is not a straight number line, on a 24-hour scale, 11 and 1 are close together, but a model reading raw hour values as integers has no way to know that. Sine-cosine encoding gives the model that cyclical structure directly.
  • Final iteration: feature interaction design, redundancy reduction, and hyperparameter tuning, each tested against the baseline before being kept.

Results

  • Naive outlier removal increased MSE by 129%, caught and reversed before it became the shipped approach.
  • Cyclical sine-cosine encoding reduced MSE to 29,816, a 2.9% improvement over baseline.
  • Final model reached an MSE of 29,222, a 4.8% total improvement over baseline, and the best result among three competing teams.

What I would do differently

The tradeoff worth naming: outlier removal is usually treated as a default preprocessing step, and I applied it that way initially instead of first asking whether this specific dataset's outliers were noise or signal. Next time, I would test that assumption in a small controlled experiment before applying any preprocessing step by default, rather than discovering the cost of the assumption after the fact.