Intermediatereinforcement-learningmdpq-learningpolicy-gradients

Reinforcement Learning Intro

27 problems·~12h·4 stages

Agents that learn by acting: MDPs, value iteration, Q-learning, and policy gradients. You will begin with policy evaluation and bandits, then implement TD updates, SARSA, and REINFORCE from scratch. A gentle on-ramp before tackling deep RL.

Your progress

0/27 · 0%

Path readiness 0%Easy 0% · Medium 0% · Hard 0%

Continue path · Policy Evaluation Sweep

What you will learn

Evaluate policies and extract actions from value functions
Implement value iteration, TD updates, and Q-learning
Balance exploration with epsilon-greedy and SARSA variants
Train policies directly with REINFORCE and actor-critic updates

Before you start

1MDP Foundations

0/8

The vocabulary of RL: policy-evaluation sweeps, policy extraction, state values from action values, discounted returns, Monte Carlo returns, incremental means, action values, and advantages.

Policy Evaluation Sweep
rl-001Reinforcement Learning
Easy
Policy Extraction From Q
rl-004Reinforcement Learning
Easy
State-Value From Action Values
rl-008Reinforcement Learning
Easy
Expected Return With Discount
rl-010Reinforcement Learning
Easy
Monte Carlo Return
rl-012Reinforcement Learning
Easy
Advantage Function
rl-028Reinforcement Learning
Easy
Incremental Mean Update
rl-026Reinforcement Learning
Easy
Bandit Sample Average Update
rl-025Reinforcement Learning
Easy

Stage checkpoint

0/2 to pass

A checkpoint is a short retrieval test drawn from this stage: solve 2 of 3 from memory to mark the stage complete. The stage stays open either way.

Tried this set and did not pass? Record it — the next set swaps in fresh problems from the same stage.

2Value-Based Methods

0/6

Plan and learn from experience: value-iteration sweeps, policy improvement, Q-values from transitions, TD(0), n-step returns, and generalized advantage estimation.

Value Iteration One Sweep
rl-002Reinforcement Learning
Medium
Policy Improvement Step
rl-005Reinforcement Learning
Medium
Q-Value From Transitions
rl-007Reinforcement Learning
Medium
TD(0) Update
rl-013Reinforcement Learning
Medium
N-Step Return
rl-014Reinforcement Learning
Medium
GAE One Step
rl-029Reinforcement Learning
Medium

Stage checkpoint

0/2 to pass

A checkpoint is a short retrieval test drawn from this stage: solve 2 of 3 from memory to mark the stage complete. The stage stays open either way.

Tried this set and did not pass? Record it — the next set swaps in fresh problems from the same stage.

3Control and Exploration

0/7

Choose actions and improve them: SARSA, Q-learning, expected SARSA, epsilon-greedy selection and probabilities, double Q-learning, and value iteration to convergence.

SARSA Update
rl-016Reinforcement Learning
Medium
Q-Learning Update
rl-017Reinforcement Learning
Medium
Expected SARSA Update
rl-019Reinforcement Learning
Medium
Epsilon-Greedy Action
rl-020Reinforcement Learning
Medium
Epsilon-Greedy Action Probabilities
rl-021Reinforcement Learning
Medium
Double Q-Learning Update
rl-018Reinforcement Learning
Hard
Value Iteration To Convergence
rl-003Reinforcement Learning
Hard

Stage checkpoint

0/2 to pass

A checkpoint is a short retrieval test drawn from this stage: solve 2 of 3 from memory to mark the stage complete. The stage stays open either way.

Epsilon-Greedy Action ProbabilitiesMedium
Double Q-Learning UpdateHard
Value Iteration To ConvergenceHard

Tried this set and did not pass? Record it — the next set swaps in fresh problems from the same stage.

4Policy Gradients

0/6

Optimize the policy directly: Bellman residuals, TD(lambda), policy iteration, softmax policy probabilities, an actor-critic update, and the REINFORCE log-probability gradient.

Softmax Policy Probabilities
rl-022Reinforcement Learning
Medium
Bellman Residual
rl-009Reinforcement Learning
Hard
TD(Lambda) Forward View
rl-015Reinforcement Learning
Hard
Policy Iteration One Round
rl-006Reinforcement Learning
Hard
Actor-Critic Update
rl-044Reinforcement Learning
Hard
REINFORCE Log-Prob Gradient
rl-040Reinforcement Learning
Hard

Stage checkpoint

0/2 to pass

A checkpoint is a short retrieval test drawn from this stage: solve 2 of 3 from memory to mark the stage complete. The stage stays open either way.

Tried this set and did not pass? Record it — the next set swaps in fresh problems from the same stage.

Capstone

Collection

Close the path with one real artifact from the catalog.

Reinforcement Learning: Foundations

Returns, TD, SARSA, and REINFORCE in one run over Markov decisions.