Tokens Per Second From MFU

Problem

Invert the model FLOPs utilization definition to plan serving capacity: tokens_per_second = mfu * num_gpus * peak_flops_per_gpu / flops_per_token.

The signature is tokens_per_second_from_mfu(mfu, num_gpus, peak_flops_per_gpu, flops_per_token). Return tokens per second.

Hint

Multiply the utilization by total peak FLOPs per second, then divide by FLOPs per token.

case 1

in: [0.4,8,312000000000000,6000000000]

expected: 166400

case 2

in: [0,8,312000000000000,6000000000]

expected: 0

case 3

in: [0.25,1,100000000000000,2000000000]

expected: 12500

case 4

in: [0.5,4,50000000000000,1000000000]

expected: 100000

Starter codePython

def tokens_per_second_from_mfu(mfu, num_gpus, peak_flops_per_gpu, flops_per_token):
    # Your code here
    pass

4 test cases · run your code against them in the editor