---
title: Feature Engineering in Python: 6 Essential Techniques From Scratch
source: https://app.sythra.ai/learn/machine-learning/feature-engineering-python-techniques-from-scratch
topic: Machine Learning
updated: 2026-08-28
publisher: Sythra (https://app.sythra.ai)
---

# Feature Engineering in Python: 6 Essential Techniques From Scratch

Feature engineering is the process of transforming, selecting, and combining raw data columns into model-ready numerical inputs that allow machine learning algorithms to uncover patterns and make accurate predictions.

_Source: [https://app.sythra.ai/learn/machine-learning/feature-engineering-python-techniques-from-scratch](https://app.sythra.ai/learn/machine-learning/feature-engineering-python-techniques-from-scratch) — free to read on Sythra._

## Key points

- Structured across 4 core pillars: scaling, distribution fixes, encoding, and feature creation.
- Provides both from-scratch Python logic and production Scikit-Learn implementations.
- Includes a model compatibility guide explaining which algorithms require feature scaling.
- Emphasizes avoiding data leakage by fitting exclusively on training sets.

**Feature engineering** is the process of creating, transforming, and selecting input variables (called _features_) from raw data columns to help machine learning models discover meaningful patterns and make significantly more accurate predictions.

Think of it like a professional chef preparing a meal: throwing raw, unwashed, whole vegetables directly into a cooking pot produces a rough, inedible dish. Feature engineering is the essential chopping, peeling, and seasoning step — reshaping raw data into clean, digestible ingredients your model can learn from.

## Why It Is Used: Features Beat Complex Algorithms

A fundamental reality in applied machine learning is that **the quality of your features matters far more than the complexity of your algorithm**. A simple linear model trained on smart, engineered features routinely outperforms a deep neural network fed raw, noisy columns.

Raw datasets are rarely model-ready. A raw date of birth string like `1990-04-12` has zero mathematical meaning to an algorithm, whereas deriving `age`, `day_of_week`, or `is_weekend` provides clear, predictive signal.

## The 4 Core Pillars of Feature Engineering

| Pillar | Objective | Standard Techniques | Practical Example |
| --- | --- | --- | --- |
| **1. Feature Scaling** | Brings numeric variables to a common scale so large units do not dominate small units. | Standardization ($z$-score), Min-Max Normalization $[0, 1]$. | Rescaling Annual Income ($150,000) and Age (28) to equal footing. |
| **2. Distribution Fixes** | Stabilizes variance and pulls skewed long tails toward a bell curve. | Log Transformation ($log(x+1)$), Power Transforms. | Unskewing real estate prices where a few mansions distort the mean. |
| **3. Categorical Encoding** | Converts text categories into numerical format without imposing fake ordinal relationships. | One-Hot Encoding, Ordinal Encoding, Target Encoding. | Encoding cities (Delhi, Mumbai, Chennai) into binary 0/1 indicator columns. |
| **4. Feature Creation** | Extracts domain-driven signals and interaction ratios. | Datetime decomposition, interaction terms, Binning. | Calculating Income per Family Member by dividing Total Income by Household Size. |

## Which Models Actually Need Feature Scaling?

Not all algorithms require feature scaling. Use this quick reference guide to avoid unnecessary preprocessing steps:

| Algorithm Family | Requires Scaling? | Why / Technical Explanation |
| --- | --- | --- |
| **Distance-Based** (KNN, K-Means, SVM) | **YES (Mandatory)** | Calculates Euclidean distance ($d = \sqrt{\sum(x_i - y_i)^2}$). Unscaled features dominate distances. |
| **Gradient-Based** (Linear/Logistic Reg, Neural Networks) | **YES (Recommended)** | Enables gradient descent to step smoothly without oscillating across asymmetric loss surfaces. |
| **Tree-Based** (Decision Trees, Random Forests, XGBoost) | **NO** | Trees make binary split decisions ($x \ge 50$) on single features independently of scale. |

## The Mathematical Foundations

### 1. Standardization (Z-Score Scaling)

Rescales a column so it has a mean ($mu$) of 0 and a standard deviation ($sigma$) of 1. It is resilient against moderate outliers:

$z = \frac{x - \mu}{\sigma}$

### 2. Min-Max Normalization

Compresses all data points into a strict bounded interval, typically between 0 and 1:

$x' = \frac{x - x_{\text{min}}}{x_{\text{max}} - x_{\text{min}}}$

### 3. Log Transformation

Pulls extreme right-skewed tails inward toward a normal distribution. The $+1$ prevents undefined $log(0)$ errors:

$x' = \log(x + 1)$

## Code: Step-by-Step Python Implementation

### 1. Setting Up the Sample Dataset

```python
import pandas as pd
import numpy as np

# Sample realistic customer profile dataset
data = {
    "age": [22, 35, 58, 41, 29],
    "income": [25000, 60000, 120000, 85000, 40000],
    "city": ["Delhi", "Mumbai", "Delhi", "Chennai", "Mumbai"],
    "date_of_birth": pd.to_datetime(
        ["2002-03-15", "1989-07-22", "1966-01-10", "1983-11-05", "1995-09-30"]
    )
}

df = pd.DataFrame(data)
print(df)
```

### 2. Standardization (Z-Score Scaling)

**From Scratch:**

```python
def standardize_from_scratch(col):
    mean = col.mean()
    std = col.std(ddof=0)
    return (col - mean) / std

df["income_std_manual"] = standardize_from_scratch(df["income"])
print(df[["income", "income_std_manual"]])
```

**Scikit-Learn Version:**

```python
from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
df["income_standardized"] = scaler.fit_transform(df[["income"]])
print(df[["income", "income_standardized"]])
```

### 3. Min-Max Normalization

**From Scratch:**

```python
def minmax_from_scratch(col):
    return (col - col.min()) / (col.max() - col.min())

df["age_norm_manual"] = minmax_from_scratch(df["age"])
print(df[["age", "age_norm_manual"]])
```

**Scikit-Learn Version:**

```python
from sklearn.preprocessing import MinMaxScaler

minmax = MinMaxScaler()
df["age_normalized"] = minmax.fit_transform(df[["age"]])
print(df[["age", "age_normalized"]])
```

### 4. One-Hot Encoding

```python
# Option A: Quick Pandas get_dummies
city_dummies = pd.get_dummies(df["city"], prefix="city")

# Option B: Scikit-Learn OneHotEncoder (Best for Pipelines)
from sklearn.preprocessing import OneHotEncoder

encoder = OneHotEncoder(sparse_output=False)
city_encoded = encoder.fit_transform(df[["city"]])
city_df = pd.DataFrame(city_encoded, columns=encoder.get_feature_names_out(["city"]))
print(city_df)
```

### 5. Log Transformation (Fixing Skew)

```python
# Use np.log1p which safely computes log(x + 1)
df["income_log"] = np.log1p(df["income"])
print(df[["income", "income_log"]])
```

### 6. Discretization / Binning

```python
df["age_group"] = pd.cut(
    df["age"],
    bins=[0, 19, 35, 60, 200],
    labels=["Teen", "Young Adult", "Adult", "Senior"]
)
print(df[["age", "age_group"]])
```

### 7. Feature Extraction From Datetime

```python
today = pd.Timestamp("2026-08-28")
df["computed_age"] = ((today - df["date_of_birth"]).dt.days / 365.25).astype(int)
print(df[["date_of_birth", "computed_age"]])
```

## Common Pitfalls & How to Avoid Data Leakage

- **Fitting Scalers on Full Dataset:** Never call `scaler.fit(X)` before train-test split. Always run `scaler.fit(X_train)` and then `scaler.transform(X_test)`.
- **High-Cardinality One-Hot Explosion:** One-hot encoding columns with thousands of unique categories (like zip codes) creates massive sparse matrices. Use frequency or target encoding instead.
- **Target Leakage:** Avoid creating features from variables that will not be available at inference time (e.g. including future payment status in a loan default model).

## Summary

- Feature engineering transforms raw columns into high-signal inputs that boost model performance.
- Distance-based and gradient-based models require scaling; tree-based models do not.
- Log transforms stabilize variance in skewed data; one-hot encoding converts text to binary flags.
- Always fit all encoders and scalers strictly on training data.

## FAQ

### What is the difference between Standardization and Min-Max Normalization?

Standardization rescales data to have a mean of 0 and standard deviation of 1 (resilient to outliers), while Min-Max Normalization bounds all values between 0 and 1.

### Do Decision Trees and Random Forests need feature scaling?

No. Tree-based models evaluate split thresholds on individual features independently, making them immune to monotonic scale transformations.

### Why must scalers only be fit on training data?

Fitting a scaler on both train and test data leaks information (the test set mean and variance) into the training process, producing overly optimistic validation scores that fail in production.

---

Written by Sythra — Learn machine learning by building. Practice this topic with Sythra's AI tutor: https://app.sythra.ai/pricing