Projects
Multi-step labs that build a working system one problem at a time.
5 projects · progress saves automatically
From the papers
35 builds from the reading list
- Fit a Compute-Optimal Scaling LawIntermediateDeepSeek LLM: Scaling Open-Source Language Models with Longtermism
- Route Tokens Through a Tiny MoEIntermediateDeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
- Build a Repository-Level FIM PipelineStarterDeepSeek-Coder: When the Large Language Model Meets Programming - The Rise of Code Intelligence
- Make GRPO Beat REINFORCE on a BanditIntermediateDeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Compress an Image into a Fixed Token BudgetStarterDeepSeek-VL: Towards Real-World Vision-Language Understanding
- Compress the KV cache into an MLA latentIntermediateDeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
- Fill in the middle with a tiny n-gram modelStarterDeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence
- Fine-tune only the experts the task routes toIntermediateLet the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language Models
- Tune a prefetch policy across SSD, DRAM, and HBMIntermediateFire-Flyer AI-HPC: A Cost-Effective Software-Hardware Co-Design for Deep Learning
- Balance MoE experts with bias, not gradientIntermediateAuxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts
- Dynamic tiling with content-routed expertsIntermediateDeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding
- Tile-scale FP8 plus a two-token MTP headAdvancedDeepSeek-V3 Technical Report
- Roofline a toy GEMM to find the crossoverStarterInsights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architectures
- Pool KV cache reads with an LRU/LFU simulatorIntermediateDualPath: Breaking the Storage Bandwidth Bottleneck in Agentic LLM Inference
- Proof Search in a Toy Rewrite SystemStarterDeepSeek-Prover: Advancing Theorem Proving in LLMs through Large-Scale Synthetic Data
- RMaxTS: Verifier-Guided Proof Tree SearchIntermediateDeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Search
- GRPO vs REINFORCE on a Verifiable BanditIntermediateDeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Condensed Features Beat Surface MatchingStarterCodeI/O: Condensing Reasoning Patterns via Code Input-Output Prediction
- Plurality Voting and DUCB Budget AllocationIntermediateInference-Time Scaling for Generalist Reward Modeling
- Subgoal Decomposition with a Proof CacheIntermediateDeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decomposition
- Thinking Budget vs AccuracyStarterDeepSeek-R1-0528 Report
- Self-Verified Proof Generation for Toy IdentitiesAdvancedDeepSeekMath-V2: Towards Self-Verifiable Mathematical Reasoning
- Two doors, one core: decouple understanding and generation codesIntermediateJanus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation
- Rectified flow vs a DDPM baseline on a 1D GaussianIntermediateJanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generation
- Scale the decoupled design without changing itIntermediateJanus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling
- Three-branch sparse attention with learned gatesIntermediateNative Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
- Route a two-mode agent on a toy code puzzleStarterDeepSeek-V3.1 Release: Hybrid Inference for the Agent Era
- Top-k block selection with MQA attentionStarterDeepSeek-V3.2-Exp: Boosting Long-Context Efficiency with DeepSeek Sparse Attention
- Find the compression knee of a toy optical readerIntermediateDeepSeek-OCR: Contexts Optical Compression
- Scale the rollout budget on a verifiable banditStarterDeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
- Bound the residual mixing with a 2x2 Sinkhorn projectionAdvancedmHC: Manifold-Constrained Hyper-Connections
- Hash trigram memory and measure the collision costIntermediateConditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
- Reorder a two-column page and decode it causallyAdvancedDeepSeek-OCR 2: Visual Causal Flow
- Spend effort where block selection is uncertainAdvancedDeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- Cache one extreme per block, raw only when neededIntermediateDeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression