Missing Values (NaN)
Python · NumPy Library
Theory
NumPy marks a missing number with np.nan (Not a Number), which is a special float value. An integer array cannot hold nan, so adding one turns the whole array into float64. nan spreads through maths: anything plus nan is nan, so np.mean of an array with one gap returns nan. It is also not equal to anything, even itself, so arr == np.nan is always False.
Use np.isnan(arr) to get a mask of the gaps; np.isnan(arr).sum() counts them and arr[~np.isnan(arr)] drops them. The nan-aware functions np.nanmean, np.nansum, np.nanmin, np.nanmax and np.nanstd ignore the gaps. To fill gaps, np.where(np.isnan(arr), fill, arr) replaces them with a value such as the mean, and np.nan_to_num(arr, nan=0) replaces them with 0.
Why we use it
Real data from sensors, forms and files has gaps. If you do not handle nan, one missing value silently turns totals and averages into nan, which then breaks later steps such as model training.
When to use it
Check np.isnan(arr).sum() right after loading numeric data. Use nan-aware functions for summaries, fill with the mean or median when a model needs complete data, and drop rows when gaps are few. For tables, pandas isna, fillna and dropna are easier.
How it works
- Numeric array loaded
- Count gaps: np.isnan(a).sum()
- Any gaps? Yes: Decide how to handle them. No: Use normal functions.
- Just need a summary? Yes: Use np.nanmean / np.nansum. No: Fill with np.where or drop with a[~np.isnan(a)].
- Results that are not silently nan
Syntax
np.nan
np.isnan(a) np.isnan(a).sum()
a[~np.isnan(a)]
np.nanmean(a) np.nansum(a) np.nanmax(a)
np.where(np.isnan(a), fill, a)Example
temps = np.array([31.5, np.nan, 29.0])
print(np.mean(temps)) # nan
print(np.nanmean(temps)) # 30.25Key points
- np.nan is a float, so arrays with gaps become float64
- Any normal maths with nan returns nan
- Never test with == np.nan; use np.isnan
- nanmean, nansum, nanmax, nanstd skip the gaps
- np.where(np.isnan(a), fill, a) fills gaps without a loop