Dtypes and memory

memory_usage(deep=True) shows where the bytes go. Object (string) columns are usually the worst offenders.

Converting a repeated-value text column to 'category' stores each distinct value once, and downcasting numbers to the narrowest type that fits does the rest.

When a file is too big to load, read it in chunks with pd.read_csv(..., chunksize=...), aggregate each chunk, then combine the small results.