Business Analytics Course [2026] | Master Business Strategy & Data-Driven Decisions

🌳 Lesson 6.3 — Decision Trees & Random Forests

Decision Trees and Random Forests are intuitive, powerful supervised learning methods for classification and regression. This lesson explains how they work, business use-cases, strengths & weaknesses, and practical tips for deployment.

🌿 What is a Decision Tree?

A Decision Tree is a flowchart-like model of decisions: internal nodes test features, branches represent outcomes, and leaves show predictions. Trees split data recursively using features that best separate the target (classification) or reduce variance (regression).

✔ Easy to visualize and interpret
✔ Handles numerical and categorical features
✔ Common splitting criteria: Gini impurity, Entropy (information gain), MSE (regression)

🛠️ How Decision Trees Work (Intuition)

Choose best split: Find feature & threshold that best separates classes or reduces error.
Create branches: Split dataset into child nodes and repeat recursively.
Stop & assign leaf: Stop when node is pure, min samples reached, or depth limit hit — assign majority class or mean value.
Prune (optional): Remove weak branches to avoid overfitting and improve generalization.

Quick Example

A bank decision tree might split on “credit score > 700?” then “income > $50k?” to predict loan default (Yes/No).

⚠️ Limitations of Single Decision Trees

✔ Prone to overfitting if grown deep
✔ High variance — small data changes can yield different trees
✔ May not capture complex relationships as smoothly as ensemble or parametric models

🌲 What is a Random Forest?

A Random Forest is an ensemble of decision trees. It builds many trees on random subsets of data and features, then aggregates their predictions (majority vote for classification, average for regression). This reduces variance and improves robustness.

✔ Lower overfitting than single trees
✔ Handles high-dimensional data and nonlinear relationships
✔ Provides feature importance scores to help explain drivers

🔁 How Random Forests Work (Key Ideas)

Bootstrap sampling: Build each tree on a different random sample (with replacement) of the training data.
Feature randomness: At each split, consider a random subset of features to reduce correlation between trees.
Aggregate predictions: Combine tree outputs (voting/averaging) to get the final prediction.

Why randomness helps

Randomness decorrelates trees — averaging many de-correlated estimators reduces variance and yields more stable predictions.

🏢 Business Applications

Fraud Detection

Random Forests detect complex fraud patterns using many transaction features while remaining robust to noisy data.

Customer Churn

Decision trees offer intuitive rules for why customers churn; forests usually produce more accurate predictions for retention targeting.

Credit Scoring

Banks use ensembles to evaluate credit risk using many borrower attributes and transaction histories.

Marketing Targeting

Segment high-value prospects and predict response likelihood with tree-based models.

✅ Pros & ⚠️ Cons

Pros

Interpretable (single trees)
Works well with mixed data types
Robust to outliers and irrelevant features (forests)
Require little feature scaling

Cons

Single trees overfit easily
Random forests are less interpretable than single trees
Forests can be computationally heavy on very large datasets
Feature importance can be biased towards numeric or high-cardinality features

🧭 Practical Tips for Model Building

✔ Start with a shallow tree to get interpretable rules; grow deeper only if needed.
✔ Use cross-validation to estimate performance and tune hyperparameters (max_depth, min_samples_leaf, n_estimators).
✔ Balance classes or use class weights for imbalanced targets (fraud, churn).
✔ Use feature engineering (interaction terms, domain transforms) to improve predictive power.
✔ Use SHAP or partial dependence plots to explain forest predictions for stakeholders.

🔍 Mini Exercise

Using a customer dataset (features: age, tenure, monthly_charges, support_calls), design a workflow to predict churn using Random Forests. Outline steps from data prep to model explanation.

Data cleaning and handling missing values
Feature selection/engineering (e.g., tenure buckets)
Train/test split and class balancing
Train Random Forest and tune n_estimators & max_depth
Evaluate (ROC-AUC, precision-recall) and explain with SHAP

✅ Hints & Starter Code

# scikit-learn starter (Python)
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split, GridSearchCV

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y)
rf = RandomForestClassifier(class_weight='balanced', random_state=42)
params = {'n_estimators':[100,200], 'max_depth':[5,10,None]}
cv = GridSearchCV(rf, params, cv=3, scoring='roc_auc')
cv.fit(X_train, y_train)
print(cv.best_params_)

📌 Key Takeaways

Decision trees are intuitive and interpretable but can overfit.
Random forests reduce variance by averaging many randomized trees, improving accuracy and robustness.
Both methods are versatile for classification and regression in business contexts.
Explainability tools (SHAP, PDP) help translate model outputs to business actions.

➜