String Methods
Python · Data Cleaning
Theory
Text columns have a .str accessor that applies string operations to every value at once and skips NaN safely. Cleaning methods include str.strip() (remove spaces at both ends), str.lower(), str.upper(), str.title() and str.replace('-', ' '). Testing methods return booleans for filters: str.contains('pur'), str.startswith('R'), str.endswith('.in'); pass case=False to ignore case and na=False to treat missing as False.
str.len() gives lengths and str[:3] slices. str.split('@') returns lists, and .str[1] takes an element from each list, so df['email'].str.split('@').str[1] gives the domain; expand=True returns separate columns instead. str.extract(r'(\d+)') pulls a regex group into a new column, for example the number from 'Order-1023'. str.cat() or plain + joins strings. Chaining is common: df['name'].str.strip().str.title() turns ' asha ' into 'Asha'.
Why we use it
Free-text fields are messy: extra spaces, mixed case, codes buried inside longer strings. The .str methods clean and parse a whole column in one expression, so keys match and categories group correctly.
When to use it
Use .str methods for any per-value text work on a column, especially before joins and groupbys. For heavy pattern logic, use str.extract with a regex; avoid apply with Python string functions, which is slower and trips on NaN.
How it works
- Text column (Series of strings)
- Clean: str.strip(), str.lower() / title(), str.replace()
- Need part of the text? Yes: str.split(...).str[i] or str.extract(regex). No: Keep the whole value.
- Filtering by text? Yes: str.contains / startswith with na=False. No: Use the cleaned column.
- Clean, parsed text columns
Syntax
s.str.strip() / .lower() / .title() / .replace(a, b)
s.str.contains(pat, case=False, na=False)
s.str.split(sep, expand=False).str[i]
s.str.extract(r'(regex)')
s.str.len()Example
df['name'] = df['name'].str.strip().str.title()
df['domain'] = df['email'].str.split('@').str[1]Key points
- .str methods skip NaN; use na=False in contains() to filter safely
- split(...).str[i] picks a part; expand=True gives separate columns
- str.extract(r'(group)') pulls regex matches into a column
- Chain cleaning steps: s.str.strip().str.lower()
- SQL equivalent: TRIM, LOWER, UPPER, LIKE '%pur%', SPLIT_PART, REGEXP_SUBSTR