Skip to content
DeepForgeby svx
Intermediatefeature-engineeringdata-pipelinesdata-qualitystreaming

Data Pipelines & Feature Engineering

30 problems·~10h·5 stages

Turn raw tables and event streams into trustworthy model inputs. You will construct interactions, bins, and crossings, build lag and rolling features, split honestly without leakage, detect drift and duplicates, and sketch streams with HyperLogLog, Space-Saving, t-digest, and reservoir sampling. Take it after ML From Scratch and Statistics Mastery.

Your progress

0/30 · 0%

Path readiness 0%Easy 0% · Medium 0% · Hard 0%

Continue path · Feature Interaction Product

What you will learn

  • Construct interaction, polynomial, binning, log, and crossing features
  • Build lag, rolling, and expanding-window features without leakage
  • Split datasets honestly and validate with walk-forward windows
  • Detect drift and duplicates, and sketch streams under bounded memory

Before you start

1Feature Construction

0/6

Encode what the data means: feature-interaction products, quadratic features, binning continuous variables, log transforms, feature crossings, and feature-bagging indices.

Stage checkpoint

0/2 to pass

A checkpoint is a short retrieval test drawn from this stage: solve 2 of 3 from memory to mark the stage complete. The stage stays open either way.

Tried this set and did not pass? Record it — the next set swaps in fresh problems from the same stage.
LabRecommended next stepLogistic Regression, From ScratchEasy

2Lag, Window and Rolling Features

0/6

Time-aware features for any model: the lag operator, rolling means, medians, and maxima, rolling skewness, and a compact rolling-stats feature block.

Stage checkpoint

0/2 to pass

A checkpoint is a short retrieval test drawn from this stage: solve 2 of 3 from memory to mark the stage complete. The stage stays open either way.

Tried this set and did not pass? Record it — the next set swaps in fresh problems from the same stage.

3Splits, Leakage and Validation

0/6

Protect the future from the past: expanding-window splits and mean forecasts, walk-forward window counts, stratified and shuffled train/test splits, and one-hot encoding.

Stage checkpoint

0/2 to pass

A checkpoint is a short retrieval test drawn from this stage: solve 2 of 3 from memory to mark the stage complete. The stage stays open either way.

Tried this set and did not pass? Record it — the next set swaps in fresh problems from the same stage.
LabRecommended next stepLogistic Regression, From ScratchEasy

4Drift, Dedup and Data Quality

0/6

Keep the pipeline honest: dedup threshold decisions and n-gram overlap, one-sample KS and chi-square drift tests, population stability index, and feature-drift KS.

Stage checkpoint

0/2 to pass

A checkpoint is a short retrieval test drawn from this stage: solve 2 of 3 from memory to mark the stage complete. The stage stays open either way.

Tried this set and did not pass? Record it — the next set swaps in fresh problems from the same stage.
LabRecommended next stepLogistic Regression, From ScratchEasy

5Sketching the Data

0/6

Summarize streams without storing them: HyperLogLog register indices, Space-Saving stream summaries, t-digest centroid compression, reservoir sampling, and tumbling and sliding window aggregates.

Stage checkpoint

0/2 to pass

A checkpoint is a short retrieval test drawn from this stage: solve 2 of 3 from memory to mark the stage complete. The stage stays open either way.

Tried this set and did not pass? Record it — the next set swaps in fresh problems from the same stage.
SimulationRecommended next stepDijkstra Step-Through

Capstone

Lab

Close the path with one real artifact from the catalog.

Robust Sensor Calibration

Real sensor data with outliers: build the pipeline and the model that survives them.