Polynomial Regression in Python: A Step-by-Step Guide [2025]

📊 Sample Dataset (curved relationship)

We model a smooth curve: Y = 4 + 0.8X + 0.15X². Below are evenly spaced points (no noise) we’ll use in code and chart.

(We’ll fit degree 1 and degree 2 models; degree 2 should match this curve almost perfectly.)

1) Import libraries & load the dataset

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.metrics import r2_score

# Deterministic curved data (matches the chart)
X = np.arange(0, 21).reshape(-1, 1)           # 0,1,2,...,20
y = (4 + 0.8*X.flatten() + 0.15*(X.flatten()**2)).astype(float)

df = pd.DataFrame({"X": X.flatten(), "Y": y})
print(df.head())
    

What’s happening? We generate a clean curved dataset using the formula above so your Python results match the JS chart exactly.

2) Fit Linear (deg=1) vs Polynomial (deg=2, 3)

def fit_poly_degree(X, y, degree):
    poly = PolynomialFeatures(degree=degree, include_bias=False)
    X_poly = poly.fit_transform(X)         # [X, X^2, X^3, ...]
    model = LinearRegression().fit(X_poly, y)
    y_hat = model.predict(X_poly)
    r2 = r2_score(y, y_hat)
    return model, poly, y_hat, r2

models = {}
for d in [1, 2, 3]:
    mdl, fe, yhat, r2 = fit_poly_degree(X, y, d)
    models[d] = {"model": mdl, "fe": fe, "yhat": yhat, "r2": r2}
    print(f"Degree {d} → R^2: {r2:.4f}")
    

Why this step? We learn three models and compute R² for each. For this dataset, degree 2 is the true generating curve, so it should yield R²≈1.00.

3) Visualize (Matplotlib): data vs fitted curves

plt.figure(figsize=(8,6))
plt.scatter(X, y, label="Data", edgecolor="black", color="gold")
for d, sty in zip([1,2,3], ["--", "-", ":"]):
    plt.plot(X, models[d]["yhat"], sty, linewidth=2, label=f"Degree {d} (R²={models[d]['r2']:.3f})")
plt.title("Polynomial Regression: Linear vs Quadratic vs Cubic")
plt.xlabel("X"); plt.ylabel("Y"); plt.legend(); plt.grid(True, ls="--", alpha=.4)
plt.show()
    

How to read this: Gold points are actual data; dashed/solid/dotted lines are deg 1/2/3 fits. The deg 2 line should pass right through the points.

4) Predict with your chosen degree

# Choose degree (2 is the correct curve here)
degree = 2
fe = models[degree]["fe"]
mdl = models[degree]["model"]

# Predict for any X values
X_new = np.array([[5], [12], [20]])
X_new_poly = fe.transform(X_new)
y_pred = mdl.predict(X_new_poly)
for xval, yval in zip(X_new.flatten(), y_pred):
    print(f"Predicted Y at X={xval}: {yval:.2f}")
    

What you’ll see: Predictions that match the formula very closely. For example: X=5 → Y= 4 + 0.8*5 + 0.15*25 = 11.75, X=12 → Y=67.60, X=20 → Y=174.00.

✅ Outcome (what to expect)

• Degree 1 (Linear): Underfits the curve → R² notably below 1 (e.g., ~0.95–0.98).
• Degree 2 (Quadratic): Exact generating form → R² ≈ 1.0000, line passes through all points.
• Degree 3 (Cubic): Also reaches R² ≈ 1 but adds an unnecessary term (β₃ close to 0).
• Predictions: Using the degree-2 model will reproduce the formula values (e.g., X=12 → 67.60).

📈 Live Visualization (JS): Data vs Quadratic Fit

Gold = data (Y = 4 + 0.8X + 0.15X²), Orange = degree 2 fit (same curve).