One-Hot Encoding vs Label Encoding in Python: When to Use Which

One-Hot Encoding and Label Encoding are the two foundational techniques used in data science to convert categorical (text-based) data into machine-readable numeric formats — One-Hot Encoding creates a distinct binary ($0$ or $1$) indicator column for each category, while Label Encoding assigns each category a single integer.

Think of it like sorting fruits for a computer that cannot read words: Label Encoding is like slapping a numbered sticker on each fruit (Apple = 1, Banana = 2, Mango = 3). One-Hot Encoding is like giving each fruit its own dedicated tray with a checkbox ("Is it an Apple?", "Is it a Banana?", "Is it a Mango?"). Both achieve the goal of numerical conversion, but picking the wrong one can mislead your algorithm into finding fake mathematical patterns that do not exist.

Why Categorical Encoding Is Necessary in Python

Machine learning algorithms are mathematical engines — they compute matrix multiplications, Euclidean distances, and gradient slopes. They cannot perform arithmetic on string values like "Delhi", "Mumbai", or "Small", "Large".

Attempting to pass raw text columns into Scikit-Learn models will immediately trigger a ValueError: could not convert string to float. Converting text to numbers is mandatory, but how you convert it determines whether your model learns real signals or spurious correlations.

The Intuition: Nominal vs. Ordinal Data

To pick the right encoding technique, you must determine whether your categories possess a natural, meaningful hierarchy:

The Fake Ranking Trap in Label Encoding

Suppose you have a City column with Delhi, Mumbai, and Chennai. If you encode them with Label Encoding as $0, 1, 2$, a linear regression model calculates a single slope weight $\beta \cdot \text{City}$. The model will mathematically assume that Chennai ($2$) is twice as much as Mumbai ($1$), and that Chennai is "greater than" Delhi ($0$).

Because cities have no numerical ranking, this introduces artificial bias into linear models, support vector machines, and distance-based algorithms like KNN. One-Hot Encoding solves this by ensuring all categories are equidistant in geometric space.

The Mathematical Foundations

1. Label / Ordinal Encoding Formulation

Maps each category to an integer index from $0$ to $n-1$:

$$f(\text{category}_i) = i, \quad i \in \{0, 1, 2, \dots, n-1\}$$

2. One-Hot Encoding Formulation

For $n$ unique categories, creates an $n$-dimensional binary vector where exactly one element equals $1$ and the rest equal $0$:

$$\text{col}_j = \begin{cases} 1 & \text{if } j = k \\ 0 & \text{if } j \neq k \end{cases} \quad \text{for } j = 1, 2, \dots, n$$

3. The Dummy Variable Trap (Multicollinearity)

When One-Hot Encoding $n$ categories, the sum of all $n$ columns always equals $1$ ($c_1 + c_2 + \dots + c_n = 1$). This creates perfect multicollinearity in linear models, making matrix inversion ($(\mathbf{X}^T\mathbf{X})^{-1}$) unstable. Dropping the first column (using $n-1$ dummy variables with drop='first') resolves this while preserving 100% of the information.

Model Reaction Matrix: How Algorithms Behave

Code: Step-by-Step Python Implementation

1. Setting Up the Sample Dataset

import pandas as pd
import numpy as np

# Sample realistic e-commerce transactions dataset
data = {
    "city": ["Delhi", "Mumbai", "Chennai", "Delhi", "Mumbai"],
    "size": ["Small", "Large", "Medium", "Small", "Large"],
    "price": [200, 450, 300, 210, 470]
}

df = pd.DataFrame(data)
print(df)

2. Label / Ordinal Encoding — From Scratch

def ordinal_encode_scratch(column, order_list=None):
    if order_list is not None:
        mapping = {category: idx for idx, category in enumerate(order_list)}
    else:
        unique_cats = column.unique()
        mapping = {category: idx for idx, category in enumerate(unique_cats)}
    return column.map(mapping), mapping

# Ordinal encoding with a defined order: Small (0) < Medium (1) < Large (2)
df["size_encoded_scratch"], size_mapping = ordinal_encode_scratch(
    df["size"], order_list=["Small", "Medium", "Large"]
)
print(df[["size", "size_encoded_scratch"]])
print("Mapping Dictionary:", size_mapping)

3. Ordinal Encoding with Scikit-Learn (Production Version)

Pro Tip: Always use OrdinalEncoder for input feature matrix $X$ rather than LabelEncoder (which is reserved exclusively for the 1D target label $y$):

from sklearn.preprocessing import OrdinalEncoder

# Explicitly declare category order
ordinal_enc = OrdinalEncoder(
    categories=[["Small", "Medium", "Large"]],
    handle_unknown="use_encoded_value",
    unknown_value=-1
)
df["size_ordinal_sklearn"] = ordinal_enc.fit_transform(df[["size"]])
print(df[["size", "size_ordinal_sklearn"]])

4. One-Hot Encoding — From Scratch

def one_hot_encode_scratch(df, column_name):
    categories = df[column_name].unique()
    encoded_df = pd.DataFrame(index=df.index)
    for cat in categories:
        encoded_df[f"{column_name}_{cat}"] = (df[column_name] == cat).astype(int)
    return encoded_df

city_onehot_scratch = one_hot_encode_scratch(df, "city")
print(city_onehot_scratch)

5. One-Hot Encoding with Pandas & Scikit-Learn

# Option A: Quick Pandas get_dummies (ideal for EDA)
df_dummies = pd.get_dummies(df["city"], prefix="city", drop_first=True)
print("--- Pandas get_dummies (drop_first=True) ---")
print(df_dummies)

# Option B: Scikit-Learn OneHotEncoder (ideal for ML Pipelines)
from sklearn.preprocessing import OneHotEncoder

ohe = OneHotEncoder(sparse_output=False, drop="first", handle_unknown="ignore")
city_encoded = ohe.fit_transform(df[["city"]])
city_df = pd.DataFrame(city_encoded, columns=ohe.get_feature_names_out(["city"]))
print("
--- Scikit-Learn OneHotEncoder ---")
print(city_df)

Side-by-Side Comparison of Output

Common Pitfalls & Best Practices

• Using LabelEncoder on Features: LabelEncoder is designed strictly for the 1D target column $y$. For 2D feature matrix $X$, use OrdinalEncoder or OneHotEncoder to maintain pipeline compatibility.
• The High Cardinality Explosion: One-Hot Encoding a column with 10,000 unique postal codes creates 10,000 sparse columns, consuming gigabytes of memory. Use Frequency Encoding or Target Encoding instead.
• Data Leakage at Test Time: Always fit your encoder strictly on X_train (encoder.fit(X_train)) and only transform on X_test (encoder.transform(X_test)). Set handle_unknown='ignore' so unexpected test categories do not crash your pipeline.

Summary: The Quick Decision Checklist

• Use One-Hot Encoding for nominal (unordered) categories like cities, colors, and brands.
• Use Ordinal Encoding for ordinal (ranked) categories like ratings, sizes, and education levels.
• Use drop='first' in One-Hot Encoding when training linear models to avoid multicollinearity.
• Tree-based algorithms (Random Forests, XGBoost) handle integer encoding well, while distance-based models (KNN, SVM) require One-Hot Encoding.
• Always fit encoders exclusively on training sets to prevent data leakage.