Skip to content
DeepForgeby svx
Intermediatedeep-learningneural-networksbackpropagationattention

Deep Learning Essentials

27 problems·~10h·4 stages

Activations, forward passes, and backprop, built one function at a time. You will go from ReLU and dense layers up through batch norm, attention, and a full transformer encoder block, with a little tokenization to feed the models. Perfect before you touch a framework, because you will recognize what every layer is doing.

Your progress

0/27 · 0%

Path readiness 0%Easy 0% · Medium 0% · Hard 0%

Continue path · ReLU

What you will learn

  • Implement activation functions, dense layers, and a full forward pass
  • Backpropagate through softmax cross-entropy and train with SGD
  • Tokenize text and turn tokens into embeddings with positional information
  • Assemble scaled dot-product attention and a transformer encoder block

Before you start

1Neurons and Forward Passes

0/7

The building blocks: ReLU, sigmoid, and tanh activations, a dense layer forward pass, and the initialization and update rules that keep training stable.

Stage checkpoint

0/2 to pass

A checkpoint is a short retrieval test drawn from this stage: solve 2 of 3 from memory to mark the stage complete. The stage stays open either way.

Tried this set and did not pass? Record it — the next set swaps in fresh problems from the same stage.
LabRecommended next stepXOR BoundaryHard

2Text to Embeddings

0/7

Models read numbers, not words. Tokenize, count, weight with TF-IDF, learn BPE merges, and look up embeddings with sinusoidal positions.

Stage checkpoint

0/2 to pass

A checkpoint is a short retrieval test drawn from this stage: solve 2 of 3 from memory to mark the stage complete. The stage stays open either way.

Tried this set and did not pass? Record it — the next set swaps in fresh problems from the same stage.
LabRecommended next stepSpam FilterMedium

3Training and Backprop

0/7

Make the network learn: stable softmax, MLP and convolution forward passes, normalization layers, and gradient computation through the loss.

Stage checkpoint

0/2 to pass

A checkpoint is a short retrieval test drawn from this stage: solve 2 of 3 from memory to mark the stage complete. The stage stays open either way.

Tried this set and did not pass? Record it — the next set swaps in fresh problems from the same stage.
LabRecommended next stepXOR BoundaryHard

4Attention and Transformers

0/6

Attention lets every token look at every other. Build the scores, softmax weights, and weighted sum, then add an LSTM gate, beam search, and a full encoder block.

Stage checkpoint

0/2 to pass

A checkpoint is a short retrieval test drawn from this stage: solve 2 of 3 from memory to mark the stage complete. The stage stays open either way.

Tried this set and did not pass? Record it — the next set swaps in fresh problems from the same stage.
LabRecommended next stepXOR BoundaryHard

Capstone

Project

Close the path with one real artifact from the catalog.

Build a Neural Network Framework

Turn the forward passes and gradients from this path into a working autograd engine.