TDC Update

Problem

Take one TDC (temporal-difference with gradient correction) step.

With delta = r + gamma*w.phi_next - w.phi_s and xv = v.phi_s, update w by alpha*delta*phi_s - alpha*gamma*xv*(phi_next - phi_s) and v by beta*delta*phi_s. Return [new_w, new_v].

Hint

TDC applies the gradient correction almost surely while GTD2 applies it in expectation.

Test cases

case 1

in: [[1,0],[0.5,0.5],[1,0],[0,1],1,0.1,0.05,0.9]

expected: [[1.045,-0.045000000000000005],[0.5,0.5]]

case 2

in: [[0,0],[0,0],[1,1],[1,0],2,0.2,0.1,0.5]

expected: [[0.4,0.4],[0.2,0.2]]

case 3

in: [[0.5,0.5],[1,-1],[0,1],[1,0],0.5,0.05,0.2,0.95]

expected: [[0.5475,0.47625000000000006],[1,-0.905]]

Starter codePython

def tdc_update(weights, v, phi_s, phi_next, r, alpha, beta, gamma):
    # Your code here
    pass

3 test cases · run your code against them in the editor