Quantization Memory Savings Ratio

Problem

Compute the fraction of fp16 memory saved by quantization: 1 - quant_bytes / (2 * num_params), where quant_bytes = num_params*bits/8 plus one scale of scale_bits bits per group when group_size > 0. Zero parameters returns 0.0.

The signature is quantization_memory_savings(num_params, bits=4, group_size=0, scale_bits=16).

Hint

Finer groups shrink the payload but the scale overhead grows in the denominator.

Test cases

case 1

in: [1000000,4,0]

expected: 0.75

case 2

in: [1000000,4,64]

expected: 0.734375

case 3

in: [1000000,8,0]

expected: 0.5

case 4

in: [100,16,0]

expected: 0

case 5

in: [0,4,0]

expected: 0

Starter codePython

def quantization_memory_savings(num_params, bits=4, group_size=0, scale_bits=16):
    # Your code here
    pass

5 test cases · run your code against them in the editor