Prefill Decode Throughput Blend

Problem

Compute end-to-end token throughput for a request: total time is prompt_tokens / prefill_tokens_per_s + generated_tokens / decode_tokens_per_s, and blended throughput is (prompt_tokens + generated_tokens) / total time.

The signature is blended_throughput(prompt_tokens, generated_tokens, prefill_tokens_per_s, decode_tokens_per_s).

Hint

Decode dominates wall time even when prefill moves far more tokens per second.

Test cases

case 1

in: [500,100,5000,50]

expected: 285.7142857143

case 2

in: [0,100,5000,50]

expected: 50

case 3

in: [500,0,5000,50]

expected: 5000

case 4

in: [1000,1000,2000,25]

expected: 49.3827160494

Starter codePython

def blended_throughput(prompt_tokens, generated_tokens, prefill_tokens_per_s, decode_tokens_per_s):
    # Your code here
    pass

4 test cases · run your code against them in the editor