The Statistics Behind Machine Learning: Mean, Variance, and Distributions

The statistics behind machine learning are the small set of tools — mean, variance, standard deviation, and probability distributions — used to describe and summarize data numerically before and during model building.

Why It's Used

Every machine learning model is, underneath all the code, built on statistical ideas. When a model "learns," what it's really doing is estimating patterns in numbers — and the language for describing those numbers, their center, their spread, and their shape, is statistics.

You don't need to become a statistician to do machine learning. But you do need a working grip on a handful of core ideas, because they show up constantly: in your EDA, in your evaluation metrics, in how algorithms like linear regression and Naïve Bayes are literally built. Skipping this foundation means later articles will keep referencing ideas you never quite locked in.

Intuition

Imagine you're told the average height in a classroom is 150 cm. That single number — the mean — gives you a sense of "typical," but it hides a lot. Are all the kids close to 150 cm? Or is it a room with a few very short kids and a few very tall ones that just happen to average out to 150? The mean alone can't tell you.

That's where variance and standard deviation come in — they describe the spread. A low spread means everyone in the room is close to that average height. A high spread means there's a huge range, even though the average is the same.

And a distribution is the full picture — not just the center and the spread, but the actual shape of how values are arranged. Some data clusters symmetrically around the middle (like adult heights, forming the classic "bell curve"). Some data is lopsided, with a long tail stretching to one side (like income, where most people earn a moderate amount but a few earn enormous amounts, dragging the tail far to the right).

Knowing the shape of your data isn't academic trivia — it directly affects which models and techniques will work well, and which assumptions are safe to make.

The Math

Mean

$$\bar{x} = \frac{1}{n}\sum_{i=1}^{n} x_i$$

Add every value together, divide by how many values there are ($n$). This gives you the "center of mass" of your data.

Variance

$$\sigma^2 = \frac{1}{n}\sum_{i=1}^{n}(x_i - \bar{x})^2$$

For every value, measure how far it is from the mean ($x_i - \bar{x}$), square that distance, then average all those squared distances. Squaring matters for two reasons: it makes every distance positive (so distances above and below the mean don't cancel out), and it punishes larger distances more heavily than small ones.

Standard Deviation

$$\sigma = \sqrt{\sigma^2}$$

Variance is in "squared units" (like squared centimetres, which doesn't mean much intuitively), so we take the square root to bring it back into the original units. This is why standard deviation, not variance, is usually the number people actually report and interpret.

The Normal Distribution

A huge number of real-world quantities (heights, measurement errors, test scores) roughly follow a specific bell-shaped pattern called the normal distribution (or Gaussian distribution), described by this formula:

$$f(x) = \frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{(x-\bar{x})^2}{2\sigma^2}}$$

You don't need to memorize this to use machine learning — but it's worth knowing what it says: given a mean $\bar{x}$ and standard deviation $\sigma$, this formula tells you how likely any particular value $x$ is to occur. Values near the mean are common (the tall middle of the bell); values far from the mean are rare (the thin tails on either side).

A Tiny Worked Example

Take the values: $2, 4, 4, 4, 5, 5, 7, 9$.

Mean:

$$\bar{x} = \frac{2+4+4+4+5+5+7+9}{8} = \frac{40}{8} = 5$$

Variance:

$$\sigma^2 = \frac{(2-5)^2+(4-5)^2+(4-5)^2+(4-5)^2+(5-5)^2+(5-5)^2+(7-5)^2+(9-5)^2}{8} = \frac{9+1+1+1+0+0+4+16}{8} = \frac{32}{8} = 4$$

Standard deviation:

$$\sigma = \sqrt{4} = 2$$

So this data has a mean of 5 and typically strays about 2 units away from it.

Code

From Scratch (NumPy)

import numpy as np

data = np.array([2, 4, 4, 4, 5, 5, 7, 9])

# Mean — the center of the data
mean = np.sum(data) / len(data)
print(f"Mean: {mean}")   # 5.0

# Variance — average squared distance from the mean
variance = np.sum((data - mean) ** 2) / len(data)
print(f"Variance: {variance}")   # 4.0

# Standard deviation — square root of variance, back in original units
std_dev = np.sqrt(variance)
print(f"Standard Deviation: {std_dev}")   # 2.0

Library Version (NumPy + SciPy)

import numpy as np
from scipy import stats

data = np.array([2, 4, 4, 4, 5, 5, 7, 9])

print(np.mean(data))   # 5.0 — same as our from-scratch version
print(np.var(data))    # 4.0
print(np.std(data))    # 2.0

# Generating and inspecting a normal distribution
normal_data = np.random.normal(loc=5, scale=2, size=1000)  # loc=mean, scale=std
print(f"Generated mean: {normal_data.mean():.2f}")
print(f"Generated std: {normal_data.std():.2f}")

# Checking how "normal" (bell-curve-like) a dataset actually is
stat, p_value = stats.shapiro(normal_data)
print(f"Shapiro-Wilk p-value: {p_value:.3f}")
# A p-value above 0.05 is a rough sign the data looks close to normal

Visualizing the Distribution

import matplotlib.pyplot as plt
import seaborn as sns

sns.histplot(normal_data, kde=True, bins=30)
plt.axvline(normal_data.mean(), color="red", linestyle="--", label="Mean")
plt.title("Generated Normal Distribution")
plt.legend()
plt.show()

Example / Output

Running the from-scratch code prints:

Mean: 5.0
Variance: 4.0
Standard Deviation: 2.0

Matching exactly what we worked out by hand earlier — a good sign the code is doing exactly what the formulas describe, nothing more mysterious than that.

The generated normal distribution might print something like:

Generated mean: 5.03
Generated std: 1.98

Very close to the loc=5, scale=2 we asked for — small differences are expected since we're drawing random samples, not recreating the distribution exactly.

Advantages and Disadvantages

Common Mistakes / Misunderstandings

• Reporting the mean without the standard deviation. A mean by itself tells you almost nothing about how consistent or spread out the data actually is.
• Assuming all data is normally distributed. Many real datasets (income, website traffic, wait times) are skewed, not symmetric — using techniques that assume normality on this kind of data can quietly produce misleading results.
• Confusing variance and standard deviation. They measure the same idea, but variance is in squared units and standard deviation is in the original units — mixing them up in a report or comparison is an easy slip.
• Treating the 3σ outlier rule as always reliable. It works best on roughly normal data — on skewed data, it can flag too many, or too few, unusual points.

Try This Yourself

Generate two datasets with np.random.normal() — one with scale=1 and one with scale=10, both with loc=50. Plot both as histograms side by side. Watching the same "center" produce a tight, narrow bell versus a wide, spread-out one is the clearest way to feel what standard deviation actually controls.

Want to go deeper? You can learn all of this properly, step by step, on our website.

Summary

• Mean tells you the center of your data; variance and standard deviation tell you how spread out it is.
• Standard deviation is just the square root of variance, expressed back in the original units for easier interpretation.
• The normal distribution is a common bell-shaped pattern that a lot of real-world data roughly follows, though not always.
• These statistics aren't just descriptive — they quietly power many ML algorithms and evaluation metrics you'll meet later.
• Understanding spread and shape, not just the average, is essential before trusting any conclusion drawn from data.