RegressionMedium10 min25 pts

Noisy Quadratic

The target follows a noisy quadratic in one feature. Explain the variance on held-out data; a straight-line fit only reaches R-squared 0.26.

Target

R² ↑ 0.880

Baseline

0.260

Time

10 min

Points

Higher is better. Scored on 15 held-out rows.

Dataset

x: feature 1 · y: label

Each dot is one training row: the single feature on x, its label on y.

Training split · 30 rows · 1 features. Scored by R² on 15 held-out rows; target ≥ 0.88, baseline 0.26.

What this lab teaches

Add x-squared as a feature and the linear least-squares machinery fits a curve; read R-squared against the noise floor instead of chasing 1.0.

The shape and the floor

The single feature x runs across roughly [-3, 3] over 30 training rows and 15 hidden rows, and the target follows a quadratic plus additive noise. A straight-line fit explains only R-squared 0.26 on the hidden rows - the recorded baseline - while the training-mean starter sits at R-squared 0.00 by construction.

Two features, one trick

Stack [1, x, x-squared] into each row and solve the same 3x3 normal equations used for a line. The fit comes out near (1.04, -0.62, 0.79) and reaches R-squared 0.939 on the hidden rows, above the 0.88 target. The model is still linear in its coefficients, which is why the identical solver works.

Why R-squared cannot hit 1

The label carries noise, so even the correct quadratic leaves residual error - its training MSE bottoms out near 0.24. That is the irreducible floor of this dataset. If a model reports R-squared much above the noise-ceiling estimate, it is fitting individual noisy points, and the hidden rows will disagree.

Common pitfalls

Squaring x in the design matrix but predicting with c0 + c1*x + c2*x. The exponent has to appear in both the fit and the evaluation, or the result is still a straight line.
Centering x for conditioning but forgetting to use the same centering when squaring and when predicting; the cross terms no longer cancel and the curve shifts.
Adding x-cubed or x-fourth terms to squeeze out the last few points of training fit. With 30 rows they absorb noise, raising training R-squared while hidden R-squared falls.

Rules of the run

Define predict(train_X, train_y, test_X) and return exactly one value per test row.
Pure Python or numpy both run; no external data or network access.
Return a numeric prediction for every test row.

Hint

Engineer x-squared as a second feature, then fit least squares on [1, x, x-squared].

Reference solution

Revealing doesn't affect your record — but try the hint first.

Timed run

10 min limit

25 points

not started · 10:00

The clock starts when you press start. Your predict(train_X, train_y, test_X) is then scored on 15 held-out rows within 10 minutes.

Starter codePython

def predict(train_X, train_y, test_X):
    # Baseline: predict the mean training target.
    # Replace with a quadratic fit, then return one value per test row.
    mean = sum(train_y) / len(train_y)
    return [mean for _ in test_X]

Tab inserts 4 spaces · ⌘+Enter runs your code

More ML Fundamentals problems Lab trails

Quadratic Features

ml-085 · ML Fundamentals

Easy

Quadratic Fit via Sums

ml-162 · ML Fundamentals

Hard

Polynomial Design Matrix

ml-051 · ML Fundamentals

Medium