Imbalanced Signal Hunt
Twenty features, mostly noise, and rare positives. Beat the all-negative baseline by finding the sparse signal.
Your goal
Beat the always negative baseline — score higher F1 than 0.0000 on the hidden 30 test rows. The first winning run records 30 points in this browser.
- Train
- 60 × 20
- Hidden test
- 30 rows
- Metric
- F1 · higher
- Baseline score
- 0.0000
Data
20 features drawn uniformly from [0, 1]. The label is 1 when 4·f3 - 4·f11 + 3·f17 - 3 plus uniform noise in [-0.6, 0.6] is positive, which makes roughly a fifth of the rows positive. 60 train rows and 30 hidden test rows from one LCG stream (seed 33), 21 draws per row.
accent: |r| ≥ 0.30 · muted: weaker · y axis: Pearson r from −1 to +1
Pearson correlation between each of the 20 feature columns and the label. Accent bars mark |r| ≥ 0.30.
Method & theory
read before you submitTwenty columns, three of them real, and roughly a fifth of the rows positive. The baseline predicts nothing and scores zero F1 - the job is to find sparse signal without drowning in noise dimensions.
Exactly how the labels were made
Each of the 20 features is uniform on [0, 1]. The label is 1 when 4*f3 - 4*f11 + 3*f17 - 3 plus uniform noise in [-0.6, 0.6] is positive, where the columns are indexed from zero. That places 12 of the 60 training rows in the positive class (20%) and 8 of the 30 hidden rows (26.7%). The signal is a weighted sum of three columns with a threshold of 3, so the features alone span [-4, 7] before noise.
Why F1 refuses to reward the do-nothing model
F1 for the positive class is 2*TP / (2*TP + FP + FN). Predicting all zeros gives TP = 0, so F1 = 0 no matter how respectable the accuracy looks - the always-negative baseline would be 80% accurate on train and 73.3% on the hidden rows. Flipping to all-positive scores 2*8 / (2*8 + 22) = 0.421 on the hidden split. Any F1 above 1e-9 counts as beating the zero baseline.
Which columns carry the signal
Pearson correlation against the label ranks the columns immediately: f3 is about +0.51, f17 about +0.42, and f11 about -0.26, while the 17 noise columns cluster near zero with a median absolute r of about 0.095 and a maximum near 0.20. A bar chart of correlations is a free feature-selection pass before any model is fit.
Standardize, fit, then respect the rounding
Standardize all 20 columns with training statistics and run logistic regression on log-loss: the per-row gradient is (p - y) times the standardized row. Signal weights grow while noise weights hover near zero. The harness rounds each prediction with Python's round before scoring, so returning probabilities is fine, but a value that rounds to 0 can never be a true positive - when recall matters, a small threshold shift is worth more than another hundred iterations.
Pitfalls
- Reporting accuracy instead of F1. Dropping all positives is 73.3% accurate on the hidden rows and still scores zero, because the metric is F1 for the positive class.
- Training on raw [0, 1] columns with a single learning rate. The irrelevant wide columns contribute gradients of the same magnitude as the three signal columns, so the optimizer spends steps fitting noise before the signal stands out.
- Letting a small positive class drive the intercept far negative and then thresholding at 0.5. Predictions concentrate in a narrow band, and Python's round turns borderline values into 0s, quietly destroying recall.
Takeaway. With rare positives and many noise features, the metric and the scaling are as important as the model itself - correlation ranking plus standardized logistic regression is enough to clear a zero baseline.
Hint
Most of the 20 columns are irrelevant. Standardize every feature and run logistic regression with gradient descent — the weights on f3, f11, and f17 will pull away from zero while the noise columns stay near it.