Reduce-Scatter Round Traffic

Problem

Compute the traffic of one ring reduce-scatter of full_bytes across num_gpus ranks: each rank forwards num_gpus - 1 chunks of full_bytes / num_gpus, so per-rank traffic is (num_gpus - 1) * full_bytes / num_gpus and the fabric total is (num_gpus - 1) * full_bytes. Return [per_rank_bytes, total_bytes].

The signature is reduce_scatter_round_traffic(full_bytes, num_gpus).

Hint

Each rank keeps one reduced chunk and forwards the other num_gpus - 1 chunks around the ring.

Test cases

case 1

in: [4000,4]

expected: [3000,12000]

case 2

in: [1000,1]

expected: [0,0]

case 3

in: [8000,8]

expected: [7000,56000]

case 4

in: [100,2]

expected: [50,100]

Starter codePython

def reduce_scatter_round_traffic(full_bytes, num_gpus):
    # Your code here
    pass

4 test cases · run your code against them in the editor