Vincenzo Timmel

Machine Learning Engineer

My biggest Numerai gains came after fixing one subtle preprocessing mismatch between backtests and live inference. Once training and deployment paths matched exactly, validation drift dropped and live MMC became far more stable.

Key Takeaways

  • Process parity beats optimistic offline metrics.
  • Era-wise training design matters for realistic Numerai behavior.
  • Positive MMC requires differentiated signal, not just decent raw accuracy.
Numerai tournament banner

Numerai tournament context for the 2025 season recap.

Numerai 2025

The Results

I finished Season 2025 ranked #24 overall and #4 in MMC (Meta Model Contribution), earning the title of Numerai Master.

You can track my live performance and the current standings here:

The Breakthrough: Catching the Ghost in the Code

Early in the year, I was hitting a wall. My models showed incredible cross-validation scores, but their live performance was inconsistent.

The culprit? A very subtle bug in the data preprocessing. I only caught it by painstakingly comparing my backtest results against the live results, round-by-round. This mismatch was the turning point. It forced me to stop trusting “good metrics” and start trusting process parity.

My Solution: Mirroring Reality

This realization led to my most successful approach: mirror the training as closely as possible to the way the model is used in live deployment.

In the Numerai tournament, the model only ever sees one “era” (a snapshot of the market) at a time during live inference. Most “out-of-the-box” solutions fail because they allow the model to see too much context during training that simply won’t exist in production.

I restructured my pipeline to:

  • Treat each era as an independent unit.
  • Apply feature neutralization and ensembling at training time exactly as they occur at inference time.
  • Use loss functions specifically tailored to rank-based correlation.

By closing the gap between the “lab” and the “market,” I reduced the drift between validation and live performance, leading to much more stable returns and higher MMC.

A Word of Advice: The MMC Trap

If you want to try Numerai, don’t be scared-it’s one of the most fascinating ML challenges out there. However, keep in mind that most easy, out-of-the-box solutions are already heavily utilized.

This is dangerous because of MMC (Meta Model Contribution), which is the primary reward metric. MMC calculates how much your model actually improves the Numerai meta-model.

If your solution is “good” but mirrors what everyone else is doing, you aren’t adding value. Even worse: if your solution is decent but generally less accurate than the existing meta-model, and your contribution effectively makes the overall ensemble worse, your MMC will be negative. To win here, you don’t just need a “good” model; you need a model that knows something the rest of the world doesn’t.

Updated: