Method Chaining & pipe
Python · Reshaping & Workflow
Theory
Method chaining writes a sequence of steps as one expression, each method working on the result of the previous one, wrapped in parentheses so it can span lines: (df.query('amount > 0').assign(gst=lambda d: d['amount'] * 0.18).groupby('city', as_index=False)['gst'].sum().sort_values('gst', ascending=False)). It reads top to bottom like a SQL query and avoids a trail of temporary variables such as df2, df3 and df_final.
Inside a chain, use lambda d: ... in assign or loc so the step sees the current intermediate DataFrame rather than the original df. pipe(func, *args) lets your own functions join the chain: def add_tax(d, rate): return d.assign(tax=d['amount'] * rate), then df.pipe(add_tax, rate=0.18). This makes cleaning steps reusable and testable. Chains are easiest to debug when each step is on its own line; comment out the tail to inspect an intermediate result.
Why we use it
Long scripts that overwrite df again and again are hard to read and easy to break. A chain makes the transformation one clear pipeline, and pipe turns repeated steps into named, reusable functions.
When to use it
Use chaining for multi-step transformations that produce one result, and pipe for custom steps you reuse across datasets. Break the chain into named variables when an intermediate result is reused or needs inspection.
How it works
- Raw DataFrame
- Clean with .pipe(clean_names)
- Filter with .query(...)
- Derive with .assign(col=lambda d: ...)
- Summarise with .groupby(...).agg(...)
- Order with .sort_values(...)
- Final report from one expression
Syntax
(df
.query('...')
.assign(new=lambda d: ...)
.groupby('key', as_index=False)
.agg(...)
.sort_values('col')
.pipe(my_func, arg=1))Example
report = (df.query('amount > 0')
.assign(gst=lambda d: d['amount'] * 0.18)
.groupby('city', as_index=False)['gst'].sum())Key points
- Wrap the chain in parentheses and put one step per line
- Use lambda d: ... inside assign/loc so it sees the current intermediate frame
- pipe(func, *args) adds your own functions to a chain
- Comment out trailing lines to inspect an intermediate result
- Reads like SQL: query = WHERE, groupby/agg = GROUP BY, sort_values = ORDER BY