~/learn/Pandas · Performance & scale
Vectorisation vs apply
Let pandas do the loop in C instead of in Python.
why this matters
A loop over a million rows takes minutes; the vectorised version takes a fraction of a second. This is the single biggest pandas speed lesson.
Column arithmetic runs on whole arrays at once. An apply with a lambda runs your Python function per row and is typically an order of magnitude slower.
Try it: Compare a vectorised np.where against an apply with an if/else.
Conditional logic vectorises too: np.where for two branches, np.select for several, and boolean masks with .loc for targeted assignment.
Note
Rule of thumb: if you are writing a for loop over rows, or apply(axis=1), there is nearly always a vectorised equivalent.
Fast vs slow
df['a'] * df['b'] ← whole column at once, in C df.apply(lambda r: …, axis=1) ← a Python loop in disguise
Pro tip
Use np.where(condition, a, b) for if/else across a column instead of applying a function row by row.
Common gotcha
apply(axis=1) is the usual culprit when pandas code feels slow. Look for a column-level operation first.
key takeaways
- Column arithmetic runs in compiled code.
- apply with axis=1 loops in Python and is far slower.
- np.where handles conditional logic vectorised.
check your understanding
Which is fastest for revenue = units × price over a million rows?
free practice
Build confidence with a question you can solve without Pro.