ClassificationEasy5 min10 pts

Logistic Regression, From Scratch

Two continuous features, one binary label. Fit a classifier and score accuracy on held-out rows; a majority-class guess scores 0.60.

Target

Accuracy ↑ 0.850

Baseline

0.600

Time

5 min

Points

Higher is better. Scored on 15 held-out rows.

Dataset

class 0
class 1

Each dot is one training row: feature 1 against feature 2, coloured by class.

Training split · 30 rows · 2 features. Scored by accuracy on 15 held-out rows; target ≥ 0.85, baseline 0.6.

What this lab teaches

Fit a sigmoid of a weighted sum with batch gradient descent, then let the 0.5 threshold turn those probabilities into held-out accuracy.

The split in numbers

There are 30 training rows with two features and 15 hidden rows. The training labels are exactly balanced (15 ones), so the starter's majority rule - label 1 only when the positive rate is greater than 0.5 - falls through to predicting 0 for every row and scores 9/15 = 0.60 on the hidden split. The target is 0.85, which needs at least 13 of 15 rows correct; 12/15 = 0.80 misses.

The gradient you need

The model is p = sigmoid(w1*x1 + w2*x2 + b). Log-loss has the cleanest gradient in machine learning: for each row the bias sees (p - y) and each weight sees (p - y) times its feature, all averaged over the 30 rows. A few hundred steps at a sensible learning rate is plenty; an internal run reaches 14/15 = 0.933 on the hidden rows. Standardize both features first, because the first column has the wider spread and uneven inputs make a single step size behave differently per weight.

What the harness accepts

predict() may return probabilities or hard labels; the scorer reads any value >= 0.5 as class 1 and anything below as class 0, and checks the list length matches the 15 test rows. That means raw logits are a trap: a logit of 0 is a probability of 0.5, but the harness reads 0 as class 0, so return sigmoid outputs or already-thresholded 0/1 values.

Common pitfalls

Leaving in the starter's majority vote. It is a constant function, so no amount of studying the rows changes the 0.60 score.
Dropping the bias term, which pins the decision boundary through the origin even though these two classes are not centred there.
Using a learning rate so large that the loss bounces around instead of descending, or so small that a few hundred steps barely move the weights off zero.

Rules of the run

Define predict(train_X, train_y, test_X) and return exactly one value per test row.
Pure Python or numpy both run; no external data or network access.
Return probabilities or 0/1 labels; values are thresholded at 0.5.

Hint

Standardize both features, then run batch gradient descent on the logistic loss for a few thousand iterations.

Reference solution

Revealing doesn't affect your record — but try the hint first.

Timed run

5 min limit

10 points

not started · 05:00

The clock starts when you press start. Your predict(train_X, train_y, test_X) is then scored on 15 held-out rows within 5 minutes.

Starter codePython

def predict(train_X, train_y, test_X):
    # Baseline: predict the majority class.
    # Replace with logistic regression, then return one value per test row.
    rate = sum(train_y) / len(train_y)
    label = 1 if rate > 0.5 else 0
    return [label for _ in test_X]

Tab inserts 4 spaces · ⌘+Enter runs your code

More ML Fundamentals problems Lab trails

Logistic Sigmoid

ml-002 · ML Fundamentals

Easy

Logistic Regression Gradient

ml-103 · ML Fundamentals

Hard

Binary Cross-Entropy Loss

ml-014 · ML Fundamentals

Medium