Global Batch Size

Problem

Compute the effective global batch size: micro_batch_size * accum_steps * data_parallel. Accumulation multiplies samples per rank and data parallelism multiplies across ranks.

The signature is global_batch_size(micro_batch_size, accum_steps, data_parallel). Return an integer.

Hint

All three factors multiply together.

Test cases

case 1

in: [4,8,16]

expected: 512

case 2

in: [1,1,1]

expected: 1

case 3

in: [32,4,2]

expected: 256

case 4

in: [0,8,4]

expected: 0

Starter codePython

def global_batch_size(micro_batch_size, accum_steps, data_parallel):
    # Your code here
    pass

4 test cases · run your code against them in the editor