~/learn/ML System Design · Data & feature pipelines
Batch vs streaming features
How fresh does each feature need to be — and what does freshness cost?
why this matters
Choosing feature freshness is a core cost-versus-quality tradeoff interviewers probe.
Batch features are computed on a schedule (hourly or daily) with tools like Spark or SQL in a warehouse: 'user's purchases in the last 30 days', 'item's average rating'. Cheap and reliable, but up to a day stale.
Streaming features are updated within seconds from an event stream (Kafka + Flink): 'clicks in this session', 'transactions from this card in the last 5 minutes'. Essential for fraud and session-aware recommendations, but harder to build and operate.
Request-time features come with the request itself: device, time of day, search query, location. They're free and always fresh.
Freshness tiers
request-time query, device, time → always fresh, free streaming session clicks, 5-min → seconds stale, costly batch 30-day history, stats → hours stale, cheap
Interview trap
Don't make every feature real-time. Ask: 'does this value change meaningfully within the hour?' Most long-term user preferences don't.
Pro tip
For fraud, velocity features ('number of transactions in the last 10 minutes') are often the single strongest signal — that's the classic case for streaming.
key takeaways
- Batch: cheap, hours stale.
- Streaming: seconds fresh, costly.
- Only stream features that change fast.
check your understanding
Which feature most needs streaming computation?