Inspecting Data
Python · Pandas Basics
Theory
Before cleaning or analysing, look at what you loaded. df.head(n) and df.tail(n) show the first and last rows, df.shape gives (rows, columns), df.dtypes lists each column's type and df.info() combines types, non-null counts and memory use in one report. df.describe() summarises numeric columns (count, mean, std, min, quartiles, max); describe(include='object') does the same for text columns.
For categories, df['city'].value_counts() counts how often each value appears, sorted from most to least common; normalize=True gives proportions and dropna=False includes missing values. df['city'].nunique() counts distinct values and df['city'].unique() lists them. df.isna().sum() counts missing values per column. Together these quickly reveal wrong types, typos, unexpected nulls and outliers.
Why we use it
Most data bugs are visible in the first minute of inspection: a numeric column read as text, a city spelled two ways, a column that is half empty. Inspecting first saves you from analysing broken data.
When to use it
Run these right after loading any dataset and again after each major transformation. value_counts and nunique are also the quickest way to check if a column is a sensible key or category before a join or groupby.
How it works
- New DataFrame loaded
- head() / tail(): eyeball sample rows
- shape, dtypes, info(): size, types, non-null counts
- describe(): ranges and averages of numeric columns
- value_counts() / nunique(): check categories and keys
- Anything odd (wrong type, nulls, typos)? Yes: Note it for cleaning. No: Move on to analysis.
- Know your data
Syntax
df.head(); df.tail(); df.shape; df.dtypes; df.info()
df.describe()
df['col'].value_counts(); df['col'].nunique()Example
print(df.shape)
df.info()
print(df['city'].value_counts())Key points
- info() prints directly - do not wrap it in print()
- describe() covers numeric columns; use include='all' for every column
- value_counts() sorts by count descending and skips NaN unless dropna=False
- nunique() counts distinct values; unique() returns them as an array
- SQL equivalent of value_counts: SELECT city, COUNT(*) FROM t GROUP BY city ORDER BY 2 DESC