Int4 Vs Int8 Memory Reduction

Problem

Compute the fraction of memory saved by moving from int8 to int4 weights. Each scheme pays num_params * bits / 8 bytes plus one scale_bits-wide scale per group when group_size is positive. Return 1 - int4_bytes / int8_bytes.

The signature is int4_vs_int8_reduction(num_params, group_size=0, scale_bits=16).

Hint

Group scales are a fixed overhead, so they dilute the ideal 50 percent saving.

Test cases

case 1

in: [1000000,0,16]

expected: 0.5

case 2

in: [1000000,64,16]

expected: 0.4848484848

case 3

in: [1000000,128,16]

expected: 0.492308177

case 4

in: [100,0,16]

expected: 0.5

Starter codePython

def int4_vs_int8_reduction(num_params, group_size=0, scale_bits=16):
    # Your code here
    pass

4 test cases · run your code against them in the editor