---
title: The Statistics Behind Machine Learning: Mean, Variance, and Distributions
source: https://app.sythra.ai/learn/machine-learning/statistics-behind-machine-learning
topic: Machine Learning
updated: 2026-08-28
publisher: Sythra (https://app.sythra.ai)
---

# The Statistics Behind Machine Learning: Mean, Variance, and Distributions

The statistics behind machine learning are the small set of tools — mean, variance, standard deviation, and probability distributions — used to describe and summarize data numerically before and during model building.

_Source: [https://app.sythra.ai/learn/machine-learning/statistics-behind-machine-learning](https://app.sythra.ai/learn/machine-learning/statistics-behind-machine-learning) — free to read on Sythra._

## Key points

- Mean is the center of your data; variance and standard deviation measure how spread out it is.
- Standard deviation is variance taken back to the original units via a square root.
- The normal distribution describes the bell-shaped pattern many real-world measurements follow.
- These statistical concepts underpin evaluation metrics and algorithms across all of machine learning.

The statistics behind machine learning are the small set of tools — mean, variance, standard deviation, and probability distributions — used to describe and summarize data numerically before and during model building.

## Why It's Used

Every machine learning model is, underneath all the code, built on statistical ideas. When a model "learns," what it's really doing is estimating patterns in numbers — and the language for describing those numbers, their center, their spread, and their shape, is statistics.

You don't need to become a statistician to do machine learning. But you do need a working grip on a handful of core ideas, because they show up constantly: in your EDA, in your evaluation metrics, in how algorithms like linear regression and Naïve Bayes are literally built. Skipping this foundation means later articles will keep referencing ideas you never quite locked in.

## Intuition

Imagine you're told the average height in a classroom is 150 cm. That single number — the **mean** — gives you a sense of "typical," but it hides a lot. Are all the kids close to 150 cm? Or is it a room with a few very short kids and a few very tall ones that just happen to average out to 150? The mean alone can't tell you.

That's where **variance** and **standard deviation** come in — they describe the _spread_. A low spread means everyone in the room is close to that average height. A high spread means there's a huge range, even though the average is the same.

And a **distribution** is the full picture — not just the center and the spread, but the actual _shape_ of how values are arranged. Some data clusters symmetrically around the middle (like adult heights, forming the classic "bell curve"). Some data is lopsided, with a long tail stretching to one side (like income, where most people earn a moderate amount but a few earn enormous amounts, dragging the tail far to the right).

Knowing the shape of your data isn't academic trivia — it directly affects which models and techniques will work well, and which assumptions are safe to make.

## The Math

### Mean

$$\bar{x} = \frac{1}{n}\sum_{i=1}^{n} x_i$$

Add every value together, divide by how many values there are ($n$). This gives you the "center of mass" of your data.

### Variance

$$\sigma^2 = \frac{1}{n}\sum_{i=1}^{n}(x_i - \bar{x})^2$$

For every value, measure how far it is from the mean ($x_i - \bar{x}$), square that distance, then average all those squared distances. Squaring matters for two reasons: it makes every distance positive (so distances above and below the mean don't cancel out), and it punishes larger distances more heavily than small ones.

### Standard Deviation

$$\sigma = \sqrt{\sigma^2}$$

Variance is in "squared units" (like squared centimetres, which doesn't mean much intuitively), so we take the square root to bring it back into the original units. This is why standard deviation, not variance, is usually the number people actually report and interpret.

### The Normal Distribution

A huge number of real-world quantities (heights, measurement errors, test scores) roughly follow a specific bell-shaped pattern called the **normal distribution** (or Gaussian distribution), described by this formula:

$$f(x) = \frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{(x-\bar{x})^2}{2\sigma^2}}$$

You don't need to memorize this to use machine learning — but it's worth knowing what it says: given a mean $\bar{x}$ and standard deviation $\sigma$, this formula tells you how likely any particular value $x$ is to occur. Values near the mean are common (the tall middle of the bell); values far from the mean are rare (the thin tails on either side).

### A Tiny Worked Example

Take the values: $2, 4, 4, 4, 5, 5, 7, 9$.

**Mean:**

$$\bar{x} = \frac{2+4+4+4+5+5+7+9}{8} = \frac{40}{8} = 5$$

**Variance:**

$$\sigma^2 = \frac{(2-5)^2+(4-5)^2+(4-5)^2+(4-5)^2+(5-5)^2+(5-5)^2+(7-5)^2+(9-5)^2}{8} = \frac{9+1+1+1+0+0+4+16}{8} = \frac{32}{8} = 4$$

**Standard deviation:**

$$\sigma = \sqrt{4} = 2$$

So this data has a mean of 5 and typically strays about 2 units away from it.

## Code

### From Scratch (NumPy)

```python
import numpy as np

data = np.array([2, 4, 4, 4, 5, 5, 7, 9])

# Mean — the center of the data
mean = np.sum(data) / len(data)
print(f"Mean: {mean}")   # 5.0

# Variance — average squared distance from the mean
variance = np.sum((data - mean) ** 2) / len(data)
print(f"Variance: {variance}")   # 4.0

# Standard deviation — square root of variance, back in original units
std_dev = np.sqrt(variance)
print(f"Standard Deviation: {std_dev}")   # 2.0
```

### Library Version (NumPy + SciPy)

```python
import numpy as np
from scipy import stats

data = np.array([2, 4, 4, 4, 5, 5, 7, 9])

print(np.mean(data))   # 5.0 — same as our from-scratch version
print(np.var(data))    # 4.0
print(np.std(data))    # 2.0

# Generating and inspecting a normal distribution
normal_data = np.random.normal(loc=5, scale=2, size=1000)  # loc=mean, scale=std
print(f"Generated mean: {normal_data.mean():.2f}")
print(f"Generated std: {normal_data.std():.2f}")

# Checking how "normal" (bell-curve-like) a dataset actually is
stat, p_value = stats.shapiro(normal_data)
print(f"Shapiro-Wilk p-value: {p_value:.3f}")
# A p-value above 0.05 is a rough sign the data looks close to normal
```

### Visualizing the Distribution

```python
import matplotlib.pyplot as plt
import seaborn as sns

sns.histplot(normal_data, kde=True, bins=30)
plt.axvline(normal_data.mean(), color="red", linestyle="--", label="Mean")
plt.title("Generated Normal Distribution")
plt.legend()
plt.show()
```

## Example / Output

Running the from-scratch code prints:

```plaintext
Mean: 5.0
Variance: 4.0
Standard Deviation: 2.0
```

Matching exactly what we worked out by hand earlier — a good sign the code is doing exactly what the formulas describe, nothing more mysterious than that.

The generated normal distribution might print something like:

```plaintext
Generated mean: 5.03
Generated std: 1.98
```

Very close to the `loc=5, scale=2` we asked for — small differences are expected since we're drawing random samples, not recreating the distribution exactly.

## Advantages and Disadvantages

| Advantages | Disadvantages |
| --- | --- |
| Gives you a compact, precise way to describe data instead of eyeballing it | A single number (like the mean) can hide important detail if used alone |
| Forms the mathematical backbone of many ML algorithms and metrics | Assuming data is "normal" when it isn't can lead to poor modeling choices |
| Makes it possible to detect outliers and unusual patterns objectively | Real-world data is often messier than the clean distributions used to teach these ideas |

## Common Mistakes / Misunderstandings

- **Reporting the mean without the standard deviation.** A mean by itself tells you almost nothing about how consistent or spread out the data actually is.
- **Assuming all data is normally distributed.** Many real datasets (income, website traffic, wait times) are skewed, not symmetric — using techniques that assume normality on this kind of data can quietly produce misleading results.
- **Confusing variance and standard deviation.** They measure the same idea, but variance is in squared units and standard deviation is in the original units — mixing them up in a report or comparison is an easy slip.
- **Treating the 3σ outlier rule as always reliable.** It works best on roughly normal data — on skewed data, it can flag too many, or too few, unusual points.

## Try This Yourself

Generate two datasets with `np.random.normal()` — one with `scale=1` and one with `scale=10`, both with `loc=50`. Plot both as histograms side by side. Watching the same "center" produce a tight, narrow bell versus a wide, spread-out one is the clearest way to _feel_ what standard deviation actually controls.

Want to go deeper? You can learn all of this properly, step by step, on our website.

## Summary

- Mean tells you the center of your data; variance and standard deviation tell you how spread out it is.
- Standard deviation is just the square root of variance, expressed back in the original units for easier interpretation.
- The normal distribution is a common bell-shaped pattern that a lot of real-world data roughly follows, though not always.
- These statistics aren't just descriptive — they quietly power many ML algorithms and evaluation metrics you'll meet later.
- Understanding spread and shape, not just the average, is essential before trusting any conclusion drawn from data.

## FAQ

### Why do I need statistics for machine learning?

Machine learning models are built on statistical ideas. Mean, variance, and distributions appear constantly — in EDA, evaluation metrics, and algorithm internals like linear regression and Naive Bayes. A working understanding of these concepts is essential for interpreting what your model is doing.

### What is the difference between variance and standard deviation?

Both measure the spread of data. Variance is the average of squared distances from the mean, so it's in squared units. Standard deviation is the square root of variance, bringing the measurement back into the original units and making it easier to interpret alongside the mean.

---

Written by Sythra — Learn machine learning by building. Practice this topic with Sythra's AI tutor: https://app.sythra.ai/pricing