ClassificationMedium10 min25 pts

Credit Default Detector

Only about a third of applicants default. F1 is measured for the default class, and predicting everyone as safe scores 0.00.

Target

F1 ↑ 0.850

Baseline

0.000

Time

10 min

Points

Higher is better. Scored on 20 held-out rows.

Dataset

class 0
class 1

Each dot is one training row: feature 1 against feature 2, coloured by class.

Training split · 40 rows · 4 features. Scored by F1 on 20 held-out rows; target ≥ 0.85, baseline 0.

What this lab teaches

Imbalanced classification where F1 on the default class - not accuracy - is the whole score, so the threshold and the rare positives decide the run.

About a third positives

The training split has 40 rows with 14 defaults (35%) and the hidden split has 20 rows with 7. The starter's majority rule finds a positive rate of 0.35, so it predicts safe for everyone: 65% accurate, F1 0.00, which is the recorded baseline. The target is 0.85.

What the four features say

Correlations with the default label run from -0.40 on the first feature to +0.64, +0.45, and +0.58 on the others, so the signal is spread across columns rather than sitting in one. Standardized logistic regression reaches hidden F1 about 0.923 - six of seven defaults caught with no false alarms - and the largest weights land on the two middle features after standardization.

The F1 budget on seven positives

With 7 hidden defaults, 6 TP + 0 FP scores 0.923, 6 TP + 1 FP still scores 0.857, but 5 TP + 1 FP is 0.769 and 5 TP + 0 FP is 0.833 - both miss. In other words, one mistake is allowed and two are fatal. That is why returning sensible probabilities and choosing the threshold deliberately matters more here than any extra optimizer tuning.

Common pitfalls

Choosing the threshold on accuracy. The majority class dominates accuracy while F1 stays at zero, so the two metrics point in different directions.
Standardizing with statistics computed on the hidden rows. The transform must come from the training split alone, or the score is not reproducible.
Treating the 0.5 threshold as sacred. With rare positives the best F1 often sits below 0.5, and the harness accepts any monotone score you return.

Rules of the run

Define predict(train_X, train_y, test_X) and return exactly one value per test row.
Pure Python or numpy both run; no external data or network access.
Return a default probability or 0/1 label; values are thresholded at 0.5.
F1 is computed for the default class (label 1).

Hint

Standardize the four features and fit logistic regression. Raising the decision threshold trades precision for recall.

Reference solution

Revealing doesn't affect your record — but try the hint first.

Timed run

10 min limit

25 points

not started · 10:00

The clock starts when you press start. Your predict(train_X, train_y, test_X) is then scored on 20 held-out rows within 10 minutes.

Starter codePython

def predict(train_X, train_y, test_X):
    # Baseline: predict the majority class.
    # Replace with logistic regression, then return one value per test row.
    rate = sum(train_y) / len(train_y)
    label = 1 if rate > 0.5 else 0
    return [label for _ in test_X]

Tab inserts 4 spaces · ⌘+Enter runs your code

More ML Fundamentals problems Lab trails

Precision, Recall, F1

ml-009 · ML Fundamentals

Medium

Threshold Sweep Best F1

ml-239 · ML Fundamentals

Hard

Inverse-Frequency Class Weights

ml-048 · ML Fundamentals

Medium