SVRG Update

Problem

Run the SVRG inner loop starting from a snapshot point.

Beginning with w = w_tilde, for each sampled index k compute v = grads_current[k] - grads_snapshot[k] + grad_full and update w = w - lr * v. Return the final w after all inner steps.

Hint

The snapshot gradient corrects the per-sample gradient without extra full passes.

Test cases

case 1

in: [[1,1],[0.5,0.5],[[0.4,0.6]],[[0.5,0.5]],0.1]

expected: [0.94,0.96]

case 2

in: [[0,0],[1,-1],[[0.5,-0.5],[1.5,-1.5]],[[1,-1],[0.5,-0.5]],0.2]

expected: [-0.30000000000000004,0.30000000000000004]

case 3

in: [[1,0],[0.5,0.5],[[0.5,0],[0,0.5]],[[1,0],[0,1]],0.1]

expected: [0.85,-0.15000000000000002]

Starter codePython

def svrg_update(w_tilde, grad_full, grads_snapshot, grads_current, lr):
    # Your code here
    pass

3 test cases · run your code against them in the editor