~/learn/ML System Design · Monitoring & feedback loops
Retraining strategy
When and how often to retrain — scheduled, triggered or continuous.
why this matters
'How often would you retrain?' needs a measured answer, not a guess.
Scheduled retraining (daily, weekly) is the simplest and most common: retrain on a sliding window of recent data and redeploy if it beats the current model.
Triggered retraining fires when monitoring detects drift or a metric drop. Continual (online) learning updates the model incrementally as new data streams in — used for ads and news feeds where trends move within hours.
Strategies
scheduled every N days simple, predictable triggered on drift / drop efficient, needs good monitoring continual every few minutes freshest, hardest to operate safely
How often? Measure it: train a model on old data and evaluate it on each following week. If quality drops sharply after 3 days, retrain more often than that.
Interview trap
Automatic retraining without automatic validation. A bad data day can produce a broken model that ships itself. Always gate deploys on evaluation checks.
Pro tip
Warm-start from the previous model's weights to make frequent retraining cheaper.
key takeaways
- Scheduled, triggered or continual.
- Measure decay to choose frequency.
- Gate every auto-deploy on evaluation.
check your understanding
How should you choose retraining frequency?