~/learn/Machine Learning · Evaluation metrics & tradeoffs
Classification metrics
Accuracy, precision, recall, F1 and the confusion matrix — and which error costs more.
why this matters
Choosing between precision and recall is really a business question — interviewers want you to tie metrics to the cost of mistakes.
Every binary prediction lands in one of four cells: true positive, false positive, true negative, false negative. All classification metrics are ratios of these cells.
Precision = TP / (TP + FP) of what I flagged, how much was right? Recall = TP / (TP + FN) of what was really there, how much did I catch? F1 = 2 · P · R / (P + R) harmonic mean of the two
Confusion matrix
predicted + predicted −
actual + TP FN ← recall row
actual − FP TN
↑ precision columnChoose by cost: fraud detection and cancer screening favour recall (missing a case is expensive); spam filtering favours precision (blocking a real email is expensive). The decision threshold moves you along this tradeoff.
Interview trap
With 1% positives, predicting 'negative' for everything scores 99% accuracy. Accuracy is misleading under class imbalance.
key takeaways
- Precision: how many flags were right.
- Recall: how many real cases were caught.
- Accuracy misleads with imbalanced classes.
check your understanding
A cancer screening test should miss as few real cases as possible. Optimize for…