apply vs Vectorised Operations
Python · Reshaping & Workflow
Theory
Vectorised operations work on whole columns at once in fast compiled code: df['amount'] * 1.18, df['a'] + df['b'], np.where(cond, x, y), .str and .dt methods. apply() calls a Python function once per value (Series.apply or Series.map) or once per row (df.apply(func, axis=1)), so it is flexible but runs at Python speed - often 100 times slower or worse on large data, and row-wise apply also loses column dtypes.
The rule: try the vectorised form first. A tax column is df['amount'] * 0.18, not apply(lambda x: x * 0.18). An if/else is np.where or np.select, not apply with if. A lookup is map(dict) or merge. Keep apply for logic with no vectorised equivalent, such as calling a parsing function from a library. For element-wise functions on a whole DataFrame, DataFrame.map(func) replaced applymap(), which is deprecated since pandas 2.1. groupby().apply is powerful but slow; prefer agg/transform where they fit.
Why we use it
Most slow pandas code comes from apply and loops where a vectorised expression exists. Knowing the vectorised alternatives keeps code short, fast and dtype-safe, while apply remains available for the truly custom cases.
When to use it
Use vectorised arithmetic, np.where/np.select, .str, .dt, map and merge first. Use Series.apply or map for a custom per-value function, df.apply(axis=1) only when the logic genuinely needs several columns and no vectorised form exists. Never loop with iterrows for calculations.
How it works
- Need a new value per row
- Plain arithmetic or comparison? Yes: Write it on columns: df['a'] * df['b']. No: Look further.
- if/else, text, date or lookup? Yes: np.where / .str / .dt / map / merge. No: Custom logic.
- Use Series.apply or map for a custom per-value function
- Use df.apply(axis=1) only if several columns are truly needed
- Fast, readable column
Syntax
df['c'] = df['a'] * df['b'] # vectorised
df['c'] = np.where(df['a'] > 0, 'pos', 'neg')
s.map(func_or_dict); s.apply(func)
df.apply(func, axis=1) # row-wise, slow
df.map(func) # element-wise (was applymap)Example
df['gst'] = df['amount'] * 0.18 # fast
df['tier'] = np.where(df['amount'] > 1000, 'big', 'small')Key points
- Vectorised column maths runs in compiled code; apply runs Python per value or row
- if/else per row -> np.where / np.select; lookups -> map(dict) or merge
- df.apply(axis=1) is the slowest option and loses column dtypes
- DataFrame.map replaces the deprecated applymap
- SQL thinking helps: write expressions on columns, not loops over rows