Reading & Writing Data
Python · Pandas Basics
Theory
pandas reads files straight into a DataFrame. pd.read_csv() takes a file path, a URL or any file-like object, splits each line on the separator, uses the first line as column names and infers a dtype for every column. Useful options: sep for other delimiters, usecols to load only some columns, dtype to force types, parse_dates to turn text into datetimes, na_values for extra missing markers and nrows to read a sample. For practice, io.StringIO(text) lets you read CSV text held in a string.
Writing is the mirror image: df.to_csv('out.csv', index=False) saves a file; leave out index=False and the row index is written as an extra unnamed column. JSON uses read_json()/to_json(orient='records'). read_excel()/to_excel() need the openpyxl package, read_parquet() needs pyarrow, and read_sql() takes a database connection plus a query.
Why we use it
Data almost always arrives as a file or a query result. read_* functions turn it into a typed DataFrame in one line, and to_* functions hand your results back to Excel, a dashboard or another pipeline.
When to use it
Use read_csv for CSV/TSV exports, read_json for API output, read_excel for spreadsheets and read_sql for databases. For very large files read in chunks (chunksize=) or with usecols, or use a columnar format like Parquet.
How it works
- pd.read_csv(source)
- Read the header line as column names
- Split each line on sep into fields
- Infer a dtype per column; mark NA values as NaN
- parse_dates or dtype given? Yes: Convert those columns as asked. No: Keep inferred types.
- DataFrame ready; later df.to_csv(..., index=False)
Syntax
pd.read_csv(path_or_buffer, sep=',', usecols=[...], parse_dates=[...])
df.to_csv(path, index=False)
pd.read_json(buffer) / df.to_json(orient='records')Example
df = pd.read_csv('orders.csv', parse_dates=['order_date'])
df.to_csv('clean_orders.csv', index=False)Key points
- read_csv accepts a path, URL or file-like object such as io.StringIO
- Pass index=False to to_csv unless you really want the index saved
- parse_dates=['col'] gives real datetimes instead of text
- 'NA', 'NaN' and empty cells become NaN; add more markers with na_values
- read_excel needs openpyxl; read_parquet needs pyarrow
- SQL equivalent: read_sql('SELECT ...', conn) runs a query into a DataFrame