Skip to content
DeepForgeby svx
Advancedproduction-mlinferencequantizationservingmonitoring

Production ML: Serving, Quantization and Monitoring

30 problems·~12h·5 stages

Everything between a trained model and a live endpoint: training-system budgets, quantization, batching, KV-cache capacity, and drift monitoring. You will estimate throughput and memory, quantize weights and caches, size continuous batching, and decide when to retrain. Take it after the ML engineer track or the deep-learning essentials.

Your progress

0/30 · 0%

Path readiness 0%Easy 0% · Medium 0% · Hard 0%

Continue path · Micro-Batch Count

What you will learn

  • Budget training memory, throughput, and sharded parameter storage
  • Quantize weights and KV caches and measure latency and memory savings
  • Serve with dynamic batching, prefix caching, and latency SLA budgets
  • Monitor drift and time retraining against its cost

Before you start

1Training Systems and Cost

0/6

The compute bill before anything ships: micro-batch counts, throughput and latency estimates, gradient-checkpointing memory, parameter shard ranges, and FSDP shard sizes.

Stage checkpoint

0/2 to pass

A checkpoint is a short retrieval test drawn from this stage: solve 2 of 3 from memory to mark the stage complete. The stage stays open either way.

Tried this set and did not pass? Record it — the next set swaps in fresh problems from the same stage.
LabRecommended next stepXOR BoundaryHard

2Quantization and Compression

0/6

Smaller and faster models: quantized inference latency, MXFP4 block-quantization error, step sizes and error variance, memory-savings ratios, and quantized KV-cache size.

Stage checkpoint

0/2 to pass

A checkpoint is a short retrieval test drawn from this stage: solve 2 of 3 from memory to mark the stage complete. The stage stays open either way.

Tried this set and did not pass? Record it — the next set swaps in fresh problems from the same stage.
LabRecommended next stepXOR BoundaryHard

3Serving, Batching and Caching

0/6

Keep the endpoint busy: decode-latency estimates, prefix-cache reuse, dynamic batch admission, the batch-latency tradeoff, latency SLA budgets, and a continuous-batching slot simulation.

Stage checkpoint

0/2 to pass

A checkpoint is a short retrieval test drawn from this stage: solve 2 of 3 from memory to mark the stage complete. The stage stays open either way.

Tried this set and did not pass? Record it — the next set swaps in fresh problems from the same stage.
LabRecommended next stepXOR BoundaryHard

4LLM Inference Capacity

0/6

Capacity planning for serving: GQA cache savings, KV-cache bytes per token, the roofline crossover batch size, tokens per second from MFU, prefill-decode throughput blending, and the latency you can promise at a target throughput.

Stage checkpoint

0/2 to pass

A checkpoint is a short retrieval test drawn from this stage: solve 2 of 3 from memory to mark the stage complete. The stage stays open either way.

Tried this set and did not pass? Record it — the next set swaps in fresh problems from the same stage.
LabRecommended next stepXOR BoundaryHard

5Monitoring, Drift and Retraining

0/6

Production is not static: one-sample KS and chi-square drift statistics, feature-drift KS, retraining cost-benefit, feature-store consistency, and retrain cadence.

Stage checkpoint

0/2 to pass

A checkpoint is a short retrieval test drawn from this stage: solve 2 of 3 from memory to mark the stage complete. The stage stays open either way.

Tried this set and did not pass? Record it — the next set swaps in fresh problems from the same stage.
LabRecommended next stepLogistic Regression, From ScratchEasy

Capstone

Collection

Close the path with one real artifact from the catalog.

CUDA & GPU Performance

FLOPs, memory, and MFU: the systems math behind the serving budgets this path computes.