Speculative Speedup Estimate

Problem

Estimate speculative decoding speedup. Expected accepted draft tokens = sum of alpha^i for i = 1..num_draft; each cycle yields accepted + 1 tokens (the bonus token) at a cost of num_draft * draft_cost_ratio + 1 target-model-equivalents. Return the ratio.

The signature is speculative_speedup(acceptance_rate, num_draft, draft_cost_ratio).

Hint

Higher acceptance rates and cheaper drafts both increase the speedup.

Test cases

case 1

in: [0.8,4,0.1]

expected: 2.4011428571

case 2

in: [0,4,0.1]

expected: 0.7142857143

case 3

in: [1,4,0.1]

expected: 3.5714285714

case 4

in: [0.5,2,0.25]

expected: 1.1666666667

Starter codePython

def speculative_speedup(acceptance_rate, num_draft, draft_cost_ratio):
    # Your code here
    pass

4 test cases · run your code against them in the editor