~/learn/Pandas · Performance & scale
Dtypes and memory
Make a frame several times smaller without losing data.
why this matters
The same data can take 10× more memory than it needs. Fixing dtypes is often the difference between a frame that fits and one that crashes.
memory_usage(deep=True) shows where the bytes go. Object (string) columns are usually the worst offenders.
Converting a repeated-value text column to 'category' stores each distinct value once, and downcasting numbers to the narrowest type that fits does the rest.
Try it: Convert the product column to category as well and re-measure.
Note
When a file is too big to load, read it in chunks with pd.read_csv(..., chunksize=...), aggregate each chunk, then combine the small results.
Pro tip
A text column with few distinct values — country, status, category — shrinks dramatically as 'category' dtype.
Common gotcha
object dtype means Python strings stored one by one. It is the usual reason a frame is far bigger than the file it came from.
key takeaways
- df.info(memory_usage='deep') shows the real size.
- Downcast numbers to int32/float32 where precision allows.
- Repeated text belongs in the category dtype.
check your understanding
A country column has 12 distinct values in 2 million rows. Best dtype?
free practice
Build confidence with a question you can solve without Pro.