~/learn/Machine Learning · Feature engineering
Scaling & transformations
Standardization vs min-max scaling, log transforms, and which models care.
why this matters
Forgetting to scale (or scaling with test data) is a quick way to fail a take-home — interviewers check for it.
standardize: z = (x − μ) / σ mean 0, std 1 min-max: x' = (x − min) / (max − min) range [0, 1] log: x' = log(1 + x) tames right-skew
Distance- and gradient-based models (k-NN, k-means, SVM, linear models with regularization, neural nets) are sensitive to feature scale. Tree-based models are not. Min-max is fragile to outliers; robust scaling uses median and IQR instead. Log transforms help skewed values like income or counts.
Interview trap
Compute μ, σ, min and max on the training set only, then apply them to validation and test data. Fitting on all data is leakage.
key takeaways
- Distance and gradient models need scaling; trees don't.
- Use log transforms for skewed values.
- Fit scalers on training data only.
check your understanding
Which model is least affected by unscaled features?