---
title: ROC Curve and AUC in Python: How They Are Calculated Step by Step
source: https://app.sythra.ai/learn/machine-learning/roc-curve-auc-score-python-explained
topic: Machine Learning
updated: 2026-08-30
publisher: Sythra (https://app.sythra.ai)
---

# ROC Curve and AUC in Python: How They Are Calculated Step by Step

An ROC Curve plots the True Positive Rate against the False Positive Rate across all decision thresholds, with the AUC (Area Under the Curve) summarizing the classifier's overall discriminative power.

_Source: [https://app.sythra.ai/learn/machine-learning/roc-curve-auc-score-python-explained](https://app.sythra.ai/learn/machine-learning/roc-curve-auc-score-python-explained) — free to read on Sythra._

## Key points

- Explains threshold sweeps and how ROC curves trace sensitivity trade-offs.
- Derives AUC using the Trapezoidal Rule and explains the Mann-Whitney U ranking connection.
- Compares ROC-AUC against Precision-Recall AUC (PR-AUC) for imbalanced data.
- Provides from-scratch NumPy implementations and Scikit-Learn Youden's J threshold optimization.

An **ROC Curve (Receiver Operating Characteristic)** is a graphical plot illustrating the diagnostic ability of a binary classifier across all possible classification thresholds. The **AUC (Area Under the Curve)** reduces this entire curve into a single scalar value ($0.0$ to $1.0$) summarizing the model's overall class discrimination ability independent of any single threshold.

Think about an airport security guard with a sensitivity dial ranging from _'Flag Literally Everything'_ (threshold $0.0$) down to _'Flag Almost Nothing'_ (threshold $1.0$). At each dial setting, the guard achieves a different **True Positive Rate (Recall)** while generating a different number of **False Positive Rates (False Alarms)**. The ROC curve maps out the complete trade-off frontier as that dial is turned.

## 1. Why ROC and AUC Are Essential

- **Threshold Independence:** Evaluates the model's intrinsic ranking ability before committing to a specific production decision boundary.
- **Fair Model Comparison:** Compares multiple competing classifiers across the entire spectrum of operating sensitivities.
- **Clear Visual Diagnostics:** Curves bulging toward the top-left corner identify superior sensitivity-to-false-alarm trade-offs.

## 2. ROC-AUC vs. Precision-Recall AUC (PR-AUC)

| Dimension | ROC Curve & AUC | Precision-Recall Curve (PR-AUC) |
| --- | --- | --- |
| **Plotted Axes** | $\text{FPR}$ ($x$-axis) vs. $\text{TPR}$ ($y$-axis) | $\text{Recall}$ ($x$-axis) vs. $\text{Precision}$ ($y$-axis) |
| **Includes True Negatives ($TN$)?** | **Yes:** $\text{FPR} = \frac{FP}{FP + TN}$ | **No:** Focuses strictly on the positive class. |
| **Class Imbalance Sensitivity** | Can appear overly optimistic if $TN$ is massive. | **Highly sensitive:** Accurately reflects performance on rare positives. |
| **Random Guess Baseline** | Diagonal Line ($\text{AUC} = 0.50$) | Horizontal Line at Positive Class Prevalence ($P / (P + N)$) |
| **Optimal Use Case** | Balanced binary classification tasks. | Heavy class imbalance (fraud, medical diagnosis, ad click prediction). |

## 3. The Mathematical Formulations

### 1. The Two Axes: TPR and FPR

$$\text{TPR} = \frac{TP}{TP + FN} \quad (\text{Recall / Sensitivity})$$

$$\text{FPR} = \frac{FP}{FP + TN} \quad (1 - \text{Specificity})$$

### 2. AUC via the Trapezoidal Rule

Sorting the threshold points by increasing $\text{FPR}$, total AUC is computed by summing the trapezoidal slices between consecutive points:

$$\text{AUC} = \sum_{k=1}^{N-1} \frac{(\text{TPR}_k + \text{TPR}_{k+1})}{2} \times (\text{FPR}_{k+1} - \text{FPR}_k)$$

### 3. The Probabilistic Interpretation (Mann-Whitney U)

Mathematically, AUC equals the exact probability that a randomly chosen positive instance is assigned a higher predicted score than a randomly chosen negative instance:

$$\text{AUC} = P(\hat{y}_{\text{pos}} > \hat{y}_{\text{neg}})$$
An AUC of $0.85$ means that $85\%$ of the time, the model correctly ranks a true positive above a true negative.

## 4. Worked Numerical Example (By Hand)

Consider $5$ samples ($3$ actual positives: A, B, E; $2$ actual negatives: C, D) with predicted probabilities:

| Sample | Actual Class ($y$) | Predicted Probability |
| --- | --- | --- |
| A | 1 | 0.90 |
| B | 1 | 0.60 |
| C | 0 | 0.55 |
| D | 0 | 0.30 |
| E | 1 | 0.20 |

- **Threshold $0.70$:** Flagged: A $\implies TP=1, FN=2, FP=0, TN=2 \implies (\text{FPR}=0.0, \text{TPR}=0.333)$.
- **Threshold $0.50$:** Flagged: A, B, C $\implies TP=2, FN=1, FP=1, TN=1 \implies (\text{FPR}=0.5, \text{TPR}=0.667)$.
- **Threshold $0.10$:** Flagged: All $\implies TP=3, FN=0, FP=2, TN=0 \implies (\text{FPR}=1.0, \text{TPR}=1.0)$.
- **Trapezoidal Sum:** Slices $(0, 0) \rightarrow (0, 0.333) \rightarrow (0.5, 0.667) \rightarrow (1.0, 1.0)$ yield:
$$\text{AUC} = 0 + \left(\frac{0.333+0.667}{2} \times 0.5\right) + \left(\frac{0.667+1.0}{2} \times 0.5\right) = 0.250 + 0.417 = 0.667$$

## 5. Code: Python Implementation From Scratch & Scikit-Learn

### 1. NumPy ROC Sweep & Trapezoidal AUC Engine

```python
import numpy as np

y_actual = np.array([1, 1, 0, 0, 1])
y_probs  = np.array([0.90, 0.60, 0.55, 0.30, 0.20])

def compute_roc_points(y_true, y_score):
    thresholds = np.sort(np.unique(y_score))[::-1]
    thresholds = np.concatenate(([1.1], thresholds, [0.0]))
    points = []

    for t in thresholds:
        y_pred = (y_score >= t).astype(int)
        tp = np.sum((y_pred == 1) & (y_true == 1))
        fn = np.sum((y_pred == 0) & (y_true == 1))
        fp = np.sum((y_pred == 1) & (y_true == 0))
        tn = np.sum((y_pred == 0) & (y_true == 0))

        tpr = tp / (tp + fn) if (tp + fn) > 0 else 0.0
        fpr = fp / (fp + tn) if (fp + tn) > 0 else 0.0
        points.append((fpr, tpr))

    return sorted(points)

def compute_auc(points):
    auc = 0.0
    for i in range(1, len(points)):
        fpr_prev, tpr_prev = points[i - 1]
        fpr_curr, tpr_curr = points[i]
        width = fpr_curr - fpr_prev
        avg_height = (tpr_curr + tpr_prev) / 2.0
        auc += width * avg_height
    return auc

points = compute_roc_points(y_actual, y_probs)
auc_val = compute_auc(points)

print("ROC Points (FPR, TPR):", [(round(f, 3), round(t, 3)) for f, t in points])
print(f"Calculated AUC: {auc_val:.3f}")
```

### 2. Production Scikit-Learn & Optimal Threshold (Youden's J)

```python
from sklearn.metrics import roc_curve, roc_auc_score

fpr, tpr, thresholds = roc_curve(y_actual, y_probs)
auc_score = roc_auc_score(y_actual, y_probs)

# Youden's J statistic finds the optimal threshold: max(TPR - FPR)
j_scores = tpr - fpr
best_idx = np.argmax(j_scores)
best_threshold = thresholds[best_idx]

print(f"Scikit-Learn AUC Score: {auc_score:.3f}")
print(f"Optimal Threshold (Youden's J): {best_threshold:.2f} (TPR={tpr[best_idx]:.2f}, FPR={fpr[best_idx]:.2f})")
```

## Summary

- The **ROC Curve** plots True Positive Rate vs. False Positive Rate across all possible probability thresholds.
- **AUC (Area Under the Curve)** measures overall class separation ($1.0$ is perfect, $0.5$ is random guessing).
- AUC corresponds to the probability that a randomly chosen positive example ranks higher than a randomly chosen negative example.
- On heavily imbalanced datasets, **Precision-Recall AUC (PR-AUC)** provides a more reliable metric than ROC-AUC.
- **Youden's J Statistic ($J = \text{TPR} - \text{FPR}$)** helps identify the single most balanced operational threshold.

## FAQ

### What does an AUC of 0.5 mean?

An AUC of 0.5 means the classifier has no discriminative ability and performs identically to random coin flipping. An AUC of 1.0 represents perfect separation, while an AUC below 0.5 indicates inverse predictions.

### Why use ROC-AUC instead of accuracy?

Accuracy depends on an arbitrary single threshold (usually 0.5) and fails on imbalanced data. ROC-AUC evaluates the classifier's overall ability to rank positive cases higher than negative cases across all possible thresholds.

### When should I use Precision-Recall AUC (PR-AUC) instead of ROC-AUC?

Use PR-AUC when working with severely imbalanced datasets (e.g. fraud detection or rare medical diagnosis). ROC-AUC incorporates True Negatives, which can make a model look deceptively optimistic when negatives heavily outnumber positives.

### What is Youden's J statistic?

Youden's J statistic (J = TPR - FPR) identifies the optimal single operating threshold on an ROC curve by finding the point that maximizes the difference between the True Positive Rate and False Positive Rate.

---

Written by Sythra — Learn machine learning by building. Practice this topic with Sythra's AI tutor: https://app.sythra.ai/pricing