Data Types & Conversion
Python · Pandas Basics
Theory
Every column has a dtype: int64, float64, bool, object (usually text), datetime64[ns] or category. Wrong types are common after loading - amounts read as text because of a stray comma, dates read as strings - and they break maths, sorting and joins. df.dtypes shows them and df['col'].astype('int64') converts when every value is valid.
For messy data use the forgiving converters. pd.to_numeric(s, errors='coerce') turns anything that is not a number into NaN instead of raising an error, so you can find and fix the bad rows. pd.to_datetime(s, format='%d-%m-%Y') parses dates (errors='coerce' gives NaT for bad ones). astype('category') stores repeated labels like city names once, saving memory and allowing a custom sort order. Integer columns that contain NaN become float64; use the nullable 'Int64' dtype to keep whole numbers with missing values.
Why we use it
Correct types make calculations right and fast: you cannot sum text, sort dates stored as strings correctly or join an int key to a text key. Converting early prevents subtle bugs later.
When to use it
Check dtypes right after loading. Use astype when data is clean, to_numeric/to_datetime with errors='coerce' when it may not be, and category for low-cardinality text columns. Join keys must have the same dtype on both sides.
How it works
- Check df.dtypes after loading
- Is the column already the right type? Yes: Leave it. No: Pick a converter.
- Are all values valid? Yes: Use astype(...). No: Use to_numeric / to_datetime with errors='coerce'.
- Inspect rows that became NaN or NaT and fix the source
- Correctly typed DataFrame
Syntax
df['col'].astype('int64' | 'float64' | 'string' | 'category')
pd.to_numeric(s, errors='coerce')
pd.to_datetime(s, format='%Y-%m-%d', errors='coerce')Example
df['amount'] = pd.to_numeric(df['amount'], errors='coerce')
df['order_date'] = pd.to_datetime(df['order_date'])Key points
- object dtype usually means text; numbers stored as text will not sum correctly
- errors='coerce' turns bad values into NaN / NaT so you can inspect them
- Give pd.to_datetime a format= when dates are day-first like 05-01-2024
- 'Int64' (capital I) is a nullable integer; plain int64 cannot hold NaN
- SQL equivalent: CAST(amount AS INT), TRY_CAST(amount AS NUMBER), TO_DATE(...)