~/learn/Machine Learning · Feature engineering
Missing data & class imbalance
Imputation tradeoffs, missingness as a signal, class weights, resampling and SMOTE.
why this matters
Messy, imbalanced data is the norm. Interviewers want a plan: metric first, then weights, resampling and threshold tuning.
Missing data is MCAR (completely random), MAR (explained by other features) or MNAR (depends on the missing value itself, e.g. high earners hiding income). Mean/median imputation is quick but shrinks variance; model-based imputation is better under MAR. Add a 'was missing' indicator — missingness is often predictive.
For imbalanced labels, first pick the right metric (PR-AUC, recall at precision). Then: class weights in the loss, undersampling the majority, oversampling the minority, or SMOTE (synthesizing minority points between neighbours). Finally tune the decision threshold.
Interview trap
Resample only the training fold. Oversampling before splitting puts copies of the same rows in train and validation.
Probabilities shift
Resampling distorts predicted probabilities. Recalibrate if you need real probabilities.
key takeaways
- Add missing-value indicators — missingness can be a signal.
- Handle imbalance with weights, resampling or SMOTE.
- Resample only inside the training fold.
check your understanding
When should you apply SMOTE?