MFU Utilization

Problem

Compute model FLOPs utilization: achieved_flops = model_flops_per_token * tokens_per_sec, capacity = gpu_peak_flops * num_gpus, MFU = achieved / capacity.

The signature is mfu_utilization(model_flops_per_token, tokens_per_sec, gpu_peak_flops, num_gpus).

Hint

MFU ignores attention's quadratic term, so long-context runs can exceed it.

Test cases

case 1

in: [2000000000,1000,1000000000000000,1]

expected: 0.002

case 2

in: [6000000000,5000,300000000000000,8]

expected: 0.0125

case 3

in: [1000000000,0,1000000000000000,1]

expected: 0

case 4

in: [2000000000,100,100000000000000,4]

expected: 0.0005

Starter codePython

def mfu_utilization(model_flops_per_token, tokens_per_sec, gpu_peak_flops, num_gpus):
    # Your code here
    pass

4 test cases · run your code against them in the editor