PyTorch
Python · Advanced Python & AI
Theory
PyTorch is an open-source deep learning framework created at Meta and, since 2022, governed by the vendor-neutral PyTorch Foundation (part of the Linux Foundation); it is very popular in research. Its core object is the tensor, which behaves much like a NumPy array but can run on a GPU and track gradients. Models are built from modules in torch.nn: in the example, nn.Sequential chains nn.Linear(10, 16) (10 inputs to 16 neurons), an nn.ReLU activation and nn.Linear(16, 1) for a single output.
PyTorch is 'define-by-run': the computation graph is built as the code executes, so ordinary Python if-statements and loops work inside models and debugging feels like normal Python. Training is an explicit loop you write: run the forward pass, compute the loss (nn.MSELoss), call loss.backward() so autograd computes gradients, call optimizer.step() to update weights, and optimizer.zero_grad() to clear gradients before the next batch.
Why we use it
It gives full, transparent control over how models are built and trained while staying Pythonic. That flexibility makes it easy to experiment with new architectures, which is why much modern AI research uses it.
When to use it
Use PyTorch for deep learning when you want flexibility, custom architectures or access to research models. For a quick standard network Keras can be shorter; for ordinary tabular problems scikit-learn is usually enough.
How it works
- Get a batch of input tensors and targets
- Forward pass: predictions = model(x)
- Compute loss = loss_fn(predictions, y)
- loss.backward() computes gradients
- optimizer.step() updates weights; optimizer.zero_grad() clears gradients
- More batches or epochs? Yes: Repeat the loop. No: Stop training.
- Evaluate with torch.no_grad()
Syntax
import torch
import torch.nn as nn
model = nn.Sequential(nn.Linear(10, 1))Example
import torch.nn as nn
model = nn.Sequential(
nn.Linear(10, 16), nn.ReLU(), nn.Linear(16, 1)
)Key points
- Created at Meta, now run by the PyTorch Foundation; very popular in ML research and NLP/vision
- Tensors work much like NumPy arrays but support GPU acceleration
- Define-by-run: normal Python control flow works in models
- You write the training loop: forward, loss, backward, step
- Call optimizer.zero_grad() each step or gradients accumulate