--- title: RMSE, MAE, and R-Squared in Python: Regression Evaluation Metrics Explained source: https://app.sythra.ai/learn/machine-learning/rmse-mae-r-squared-regression-metrics-python topic: Machine Learning updated: 2026-08-28 publisher: Sythra (https://app.sythra.ai) --- # RMSE, MAE, and R-Squared in Python: Regression Evaluation Metrics Explained RMSE, MAE, and R-Squared evaluate regression models: MAE measures average absolute error, RMSE squares errors to heavily penalize large outliers, and R-Squared measures the percentage of target variance explained relative to a baseline mean. _Source: [https://app.sythra.ai/learn/machine-learning/rmse-mae-r-squared-regression-metrics-python](https://app.sythra.ai/learn/machine-learning/rmse-mae-r-squared-regression-metrics-python) — free to read on Sythra._ ## Key points - Covers MAE, RMSE, R-Squared, and Adjusted R-Squared formulas in detail. - Explains why RMSE is always greater than or equal to MAE due to squared penalties. - Provides worked hand calculations and pure NumPy implementations. - Demonstrates Scikit-Learn root_mean_squared_error and outlier stress tests. **RMSE (Root Mean Squared Error)**, **MAE (Mean Absolute Error)**, and **$R^2$ (R-squared / Coefficient of Determination)** are the three primary statistical metrics used to evaluate the accuracy and predictive power of regression models. Imagine you are a daily temperature forecaster. At the end of the season, your station manager asks: _"How accurate were your forecasts overall?"_ You cannot hand over a 90-row spreadsheet — you need a single, rigorous score to summarize performance. **MAE** tells you the raw average error in degrees, **RMSE** penalizes the days you were wildly off, and **$R^2$** measures how much better your model is compared to just guessing the seasonal average every single day. ## 1. The Regression Evaluation Decision Matrix | Metric | Output Units | Outlier Sensitivity | Scale Dependent? | Best Used For | | --- | --- | --- | --- | --- | | **MAE (Mean Absolute Error)** | Same units as target ($y$) | **Robust:** Treats small and large errors linearly. | Yes | Reporting average real-world dollar/unit errors to business stakeholders. | | **RMSE (Root Mean Squared Error)** | Same units as target ($y$) | **High:** Squares errors, heavily penalizing large misses. | Yes | When extreme errors have catastrophic financial, medical, or safety costs. | | **$R^2$ (Coefficient of Determination)** | Dimensionless ($-\infty$ to $1.0$) | Sensitive to large residual variance. | **No (Scale-Free)** | Comparing goodness-of-fit across models trained on completely different datasets. | | **Adjusted $R^2$** | Dimensionless ($-\infty$ to $1.0$) | Penalizes the addition of useless feature columns. | **No (Scale-Free)** | Selecting the best subset of features in multiple linear regression. | ## 2. Mathematical Formulations & Derivations ### 1. Mean Absolute Error (MAE) Calculates the average absolute magnitude of the residual errors across all $m$ samples: $$\text{MAE} = \frac{1}{m} \sum_{i=1}^{m} | \hat{y}^{(i)} - y^{(i)} |$$ ### 2. Root Mean Squared Error (RMSE) Squares residuals before averaging, then takes the square root to return to the target's original units: $$\text{RMSE} = \sqrt{\frac{1}{m} \sum_{i=1}^{m} (\hat{y}^{(i)} - y^{(i)})^2}$$ **Why $\text{RMSE} \ge \text{MAE}$ always:** Because squaring magnifies larger numbers disproportionately ($20^2 = 400$ vs $10^2 = 100$), RMSE is always greater than or equal to MAE. A large gap between RMSE and MAE indicates that your model occasionally produces extreme outlier errors. ### 3. R-Squared ($R^2$) Formulation Compares the model's Sum of Squared Residuals ($\text{SSR}$) against the Total Sum of Squares ($\text{SST}$) of a naive baseline predicting the sample mean $\bar{y}$: $$R^2 = 1 - \frac{\text{SSR}}{\text{SST}} = 1 - \frac{\sum_{i=1}^{m} (y^{(i)} - \hat{y}^{(i)})^2}{\sum_{i=1}^{m} (y^{(i)} - \bar{y})^2}$$ - **$R^2 = 1.0$:** Perfect prediction ($\text{SSR} = 0$). - **$R^2 = 0.0$:** The model performs no better than guessing the constant mean $\bar{y}$. - **$R^2 < 0.0$:** The model performs _worse_ than guessing the baseline average (common with severe overfitting or improper constraints). ### 4. Adjusted $R^2$ (Penalizing Feature Inflation) Standard $R^2$ has a flaw: adding more feature columns will _never_ decrease $R^2$, even if the features are random noise. Adjusted $R^2$ penalizes adding useless predictors $p$: $$R^2_{\text{adj}} = 1 - \left[ \frac{(1 - R^2)(m - 1)}{m - p - 1} \right]$$ ## 3. Worked Numerical Example (By Hand) Let's evaluate a 4-sample house price dataset (in Lakhs ₹): | Actual ($y$) | Predicted ($\hat{y}$) | Error ($\hat{y} - y$) | \|Error\| | Error$^2$ | | --- | --- | --- | --- | --- | | 50 | 48 | -2 | 2 | 4 | | 60 | 65 | +5 | 5 | 25 | | 55 | 50 | -5 | 5 | 25 | | 70 | 72 | +2 | 2 | 4 | - **$\text{MAE}$:** $\frac{2 + 5 + 5 + 2}{4} = \frac{14}{4} = 3.50$ (Average error is ₹3.5 Lakh). - **$\text{RMSE}$:** $\sqrt{\frac{4 + 25 + 25 + 4}{4}} = \sqrt{14.5} \approx 3.81$ Lakh. - **Mean $\bar{y}$:** $\frac{50 + 60 + 55 + 70}{4} = 58.75$. - **$\text{SST}$:** $(50-58.75)^2 + (60-58.75)^2 + (55-58.75)^2 + (70-58.75)^2 = 218.75$. - **$R^2$:** $1 - \frac{58}{218.75} = 1 - 0.265 = 0.735$ (Model explains 73.5% of target variance). ## 4. Python Implementation: From Scratch & Scikit-Learn ### 1. NumPy Implementation From Scratch ```python import numpy as np y_actual = np.array([50.0, 60.0, 55.0, 70.0]) y_pred = np.array([48.0, 65.0, 50.0, 72.0]) def calculate_mae(y_true, y_pred): return np.mean(np.abs(y_true - y_pred)) def calculate_rmse(y_true, y_pred): return np.sqrt(np.mean((y_true - y_pred) ** 2)) def calculate_r2(y_true, y_pred): ssr = np.sum((y_true - y_pred) ** 2) sst = np.sum((y_true - np.mean(y_true)) ** 2) return 1 - (ssr / sst) def calculate_adjusted_r2(y_true, y_pred, n_features): r2 = calculate_r2(y_true, y_pred) m = len(y_true) return 1 - ((1 - r2) * (m - 1) / (m - n_features - 1)) print(f"MAE: {calculate_mae(y_actual, y_pred):.3f}") print(f"RMSE: {calculate_rmse(y_actual, y_pred):.3f}") print(f"R²: {calculate_r2(y_actual, y_pred):.3f}") print(f"Adjusted R²: {calculate_adjusted_r2(y_actual, y_pred, n_features=1):.3f}") ``` ### 2. Production Scikit-Learn Metrics ```python from sklearn.metrics import mean_absolute_error, root_mean_squared_error, r2_score mae = mean_absolute_error(y_actual, y_pred) rmse = root_mean_squared_error(y_actual, y_pred) r2 = r2_score(y_actual, y_pred) print(f"Scikit-Learn MAE: {mae:.3f}") print(f"Scikit-Learn RMSE: {rmse:.3f}") print(f"Scikit-Learn R²: {r2:.3f}") ``` ### 3. The Outlier Stress Test (MAE vs RMSE) ```python # Introducing one extreme prediction mistake (150 instead of 65) y_pred_outlier = np.array([48.0, 150.0, 50.0, 72.0]) print("--- Outlier Impact Comparison ---") print(f"MAE shifted from 3.50 -> {mean_absolute_error(y_actual, y_pred_outlier):.2f}") print(f"RMSE shifted from 3.81 -> {root_mean_squared_error(y_actual, y_pred_outlier):.2f} (Exploded!)") print(f"R² shifted from 0.735 -> {r2_score(y_actual, y_pred_outlier):.3f} (Collapsed Negative!)") ``` ## 5. Common Pitfalls When Evaluating Models - **Comparing RMSE Across Different Datasets:** Because RMSE has units of the target variable, an RMSE of 5 on housing (lakhs) is fundamentally different from 5 on weather (°C). Use $R^2$ for cross-dataset comparisons. - **Relying on $R^2$ Alone:** An $R^2$ of 0.92 sounds impressive, but if target variance is massive, your predictions could still be off by thousands of dollars. Always report $R^2$ alongside MAE or RMSE. - **Ignoring Negative $R^2$:** A negative $R^2$ is not a bug; it proves your model performs worse than a horizontal line predicting the sample mean. ## Summary - **MAE** averages absolute residuals — intuitive, robust to outliers, and directly interpretable. - **RMSE** squares residuals before taking the root — heavily penalizes large errors. - **$R^2$** measures percentage of variance explained relative to a mean baseline ($0–1$, can be negative). - **Adjusted $R^2$** prevents feature inflation by penalizing the addition of irrelevant predictor columns. ## FAQ ### What is the difference between MAE and RMSE? MAE treats all errors linearly by averaging their absolute values, making it robust to outliers. RMSE squares errors before averaging, penalizing large mistakes much more heavily than small ones. ### Can R-Squared be negative? Yes. R-Squared can be negative when a model's predictions have larger squared error than simply predicting the mean of the target variable for every sample (often seen in severely overfitted or unconstrained models). ### Why is Adjusted R-Squared better than standard R-Squared for multiple regression? Standard R-Squared will always increase or stay the same when you add new features, even if they are pure noise. Adjusted R-Squared penalizes the addition of irrelevant features by accounting for the number of predictors (p). ### Why is RMSE always greater than or equal to MAE? Because squaring individual residual errors gives disproportionate weight to larger deviations before taking the square root, RMSE will always be greater than or equal to MAE for the same predictions. --- Written by Sythra — Learn machine learning by building. Practice this topic with Sythra's AI tutor: https://app.sythra.ai/pricing