Ring All-Reduce Steps

Problem

Count the communication steps of a ring all-reduce: reduce-scatter takes num_gpus - 1 steps and all-gather takes another num_gpus - 1, so the total is 2 * (num_gpus - 1).

The signature is ring_all_reduce_steps(num_gpus). Return an integer.

Hint

Latency grows linearly with the number of GPUs in a ring.

Test cases

case 1

in: [1]

expected: 0

case 2

in: [2]

expected: 2

case 3

in: [4]

expected: 6

case 4

in: [8]

expected: 14

Starter codePython

def ring_all_reduce_steps(num_gpus):
    # Your code here
    pass

4 test cases · run your code against them in the editor