Effective Bits With Overhead

Problem

Compute the effective bits per weight including scale overhead: bits + scale_bits / group_size. A non-positive group_size means no overhead, so return bits as a float.

The signature is effective_bits_with_overhead(bits, group_size, scale_bits=16).

Hint

A group of 64 int4 weights plus a 16-bit scale costs 4.25 bits per weight.

Test cases

case 1

in: [4,64,16]

expected: 4.25

case 2

in: [4,128,16]

expected: 4.125

case 3

in: [8,32,16]

expected: 8.5

case 4

in: [16,0,16]

expected: 16

case 5

in: [4,32,8]

expected: 4.25

Starter codePython

def effective_bits_with_overhead(bits, group_size, scale_bits=16):
    # Your code here
    pass

5 test cases · run your code against them in the editor