Feature Engineering in Python: 6 Essential Techniques From Scratch

Feature engineering is the process of creating, transforming, and selecting input variables (called features) from raw data columns to help machine learning models discover meaningful patterns and make significantly more accurate predictions.

Think of it like a professional chef preparing a meal: throwing raw, unwashed, whole vegetables directly into a cooking pot produces a rough, inedible dish. Feature engineering is the essential chopping, peeling, and seasoning step — reshaping raw data into clean, digestible ingredients your model can learn from.

Why It Is Used: Features Beat Complex Algorithms

A fundamental reality in applied machine learning is that the quality of your features matters far more than the complexity of your algorithm. A simple linear model trained on smart, engineered features routinely outperforms a deep neural network fed raw, noisy columns.

Raw datasets are rarely model-ready. A raw date of birth string like 1990-04-12 has zero mathematical meaning to an algorithm, whereas deriving age, day_of_week, or is_weekend provides clear, predictive signal.

The 4 Core Pillars of Feature Engineering

Which Models Actually Need Feature Scaling?

Not all algorithms require feature scaling. Use this quick reference guide to avoid unnecessary preprocessing steps:

The Mathematical Foundations

1. Standardization (Z-Score Scaling)

Rescales a column so it has a mean ($mu$) of 0 and a standard deviation ($sigma$) of 1. It is resilient against moderate outliers:

$z = \frac{x - \mu}{\sigma}$

2. Min-Max Normalization

Compresses all data points into a strict bounded interval, typically between 0 and 1:

$x' = \frac{x - x_{\text{min}}}{x_{\text{max}} - x_{\text{min}}}$

3. Log Transformation

Pulls extreme right-skewed tails inward toward a normal distribution. The $+1$ prevents undefined $log(0)$ errors:

$x' = \log(x + 1)$

Code: Step-by-Step Python Implementation

1. Setting Up the Sample Dataset

import pandas as pd
import numpy as np

# Sample realistic customer profile dataset
data = {
    "age": [22, 35, 58, 41, 29],
    "income": [25000, 60000, 120000, 85000, 40000],
    "city": ["Delhi", "Mumbai", "Delhi", "Chennai", "Mumbai"],
    "date_of_birth": pd.to_datetime(
        ["2002-03-15", "1989-07-22", "1966-01-10", "1983-11-05", "1995-09-30"]
    )
}

df = pd.DataFrame(data)
print(df)

2. Standardization (Z-Score Scaling)

From Scratch:

def standardize_from_scratch(col):
    mean = col.mean()
    std = col.std(ddof=0)
    return (col - mean) / std

df["income_std_manual"] = standardize_from_scratch(df["income"])
print(df[["income", "income_std_manual"]])

Scikit-Learn Version:

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
df["income_standardized"] = scaler.fit_transform(df[["income"]])
print(df[["income", "income_standardized"]])

3. Min-Max Normalization

From Scratch:

def minmax_from_scratch(col):
    return (col - col.min()) / (col.max() - col.min())

df["age_norm_manual"] = minmax_from_scratch(df["age"])
print(df[["age", "age_norm_manual"]])

Scikit-Learn Version:

from sklearn.preprocessing import MinMaxScaler

minmax = MinMaxScaler()
df["age_normalized"] = minmax.fit_transform(df[["age"]])
print(df[["age", "age_normalized"]])

4. One-Hot Encoding

# Option A: Quick Pandas get_dummies
city_dummies = pd.get_dummies(df["city"], prefix="city")

# Option B: Scikit-Learn OneHotEncoder (Best for Pipelines)
from sklearn.preprocessing import OneHotEncoder

encoder = OneHotEncoder(sparse_output=False)
city_encoded = encoder.fit_transform(df[["city"]])
city_df = pd.DataFrame(city_encoded, columns=encoder.get_feature_names_out(["city"]))
print(city_df)

5. Log Transformation (Fixing Skew)

# Use np.log1p which safely computes log(x + 1)
df["income_log"] = np.log1p(df["income"])
print(df[["income", "income_log"]])

6. Discretization / Binning

df["age_group"] = pd.cut(
    df["age"],
    bins=[0, 19, 35, 60, 200],
    labels=["Teen", "Young Adult", "Adult", "Senior"]
)
print(df[["age", "age_group"]])

7. Feature Extraction From Datetime

today = pd.Timestamp("2026-08-28")
df["computed_age"] = ((today - df["date_of_birth"]).dt.days / 365.25).astype(int)
print(df[["date_of_birth", "computed_age"]])

Common Pitfalls & How to Avoid Data Leakage

• Fitting Scalers on Full Dataset: Never call scaler.fit(X) before train-test split. Always run scaler.fit(X_train) and then scaler.transform(X_test).
• High-Cardinality One-Hot Explosion: One-hot encoding columns with thousands of unique categories (like zip codes) creates massive sparse matrices. Use frequency or target encoding instead.
• Target Leakage: Avoid creating features from variables that will not be available at inference time (e.g. including future payment status in a loan default model).

Summary

• Feature engineering transforms raw columns into high-signal inputs that boost model performance.
• Distance-based and gradient-based models require scaling; tree-based models do not.
• Log transforms stabilize variance in skewed data; one-hot encoding converts text to binary flags.
• Always fit all encoders and scalers strictly on training data.