PCA
Python · Advanced Python & AI
Theory
Principal Component Analysis (PCA) reduces many correlated features to a few new ones called principal components. Each component is a weighted combination of the original features. The first component points in the direction where the data varies the most, the second captures the most remaining variation while being uncorrelated with the first, and so on.
Keeping only the first few components gives a smaller dataset that still holds most of the information. PCA(n_components=2).fit_transform(X_scaled) turns, say, 10 features into 2, which you can plot on a scatter chart. explained_variance_ratio_ reports the share of total variance each component keeps; you can also pass a fraction, PCA(n_components=0.95), to keep enough components for 95% of the variance. Scale features first, because PCA is driven by variance and large-unit columns would dominate.
Why we use it
High-dimensional data is hard to visualise, slow to train on and often redundant. PCA compresses it, removes correlation between features and can reduce noise, while keeping most of the signal.
When to use it
Use PCA to visualise high-dimensional data in 2-D/3-D, or to shrink many correlated numeric features before modelling. Avoid it when you need to explain results in terms of the original features, since components are mixtures.
How it works
- Numeric feature matrix X
- Scale features with StandardScaler
- PCA finds directions of maximum variance
- Enough variance explained? Yes: Use the chosen components. No: Increase n_components.
- Project the data onto the components
- Reduced features for plotting or modelling
Syntax
from sklearn.decomposition import PCA
X_reduced = PCA(n_components=2).fit_transform(X)Example
pca = PCA(n_components=2)
X_reduced = pca.fit_transform(X_scaled)Key points
- Commonly used to visualize high-dimensional data in 2D/3D
- explained_variance_ratio_ shows how much information each component keeps
- Always scale features before PCA
- Components are uncorrelated combinations of original features
- Fit PCA on training data only, like a scaler