Async SGD Staleness Update

Problem

Combine gradients from asynchronous workers with staleness weighting: each worker's gradient is weighted by decay^staleness, and the result is the weighted average per coordinate. Lower weights go to more stale updates.

The signature is async_sgd_update(worker_grads, staleness, decay). Return the aggregated gradient vector.

Hint

Staleness weighting prevents slow workers from applying outdated gradients with full strength.

Test cases

case 1

in: [[[1,2],[3,4]],[0,1],0.5]

expected: [1.6666666667,2.6666666667]

case 2

in: [[[2],[4]],[2,0],0.5]

expected: [3.6]

case 3

in: [[[1,1],[1,1],[1,1]],[0,0,0],0.9]

expected: [1,1]

case 4

in: [[[0,0],[10,20]],[1,0],1]

expected: [5,10]

Starter codePython

def async_sgd_update(worker_grads, staleness, decay):
    # Your code here
    pass

4 test cases · run your code against them in the editor