DDPG Actor Update

Problem

Apply one deterministic policy gradient (DDPG) actor update and return the new parameter list.

params is the actor parameter vector, policy_grads holds d(mu)/d(theta), q_grad is the critic gradient with respect to the chosen action, and lr is the learning rate. Return params[i] + lr * q_grad * policy_grads[i].

Hint

The actor ascends the critic's value, so the Q gradient scales the policy gradient.

Test cases

case 1

in: [[1,2],[0.5,-1],2,0.1]

expected: [1.1,1.8]

case 2

in: [[0],[1],-3,0.5]

expected: [-1.5]

case 3

in: [[2,4],[1,1],0,0.3]

expected: [2,4]

case 4

in: [[1],[2],1,-0.1]

expected: [0.8]

Starter codePython

def ddpg_actor_update(params, policy_grads, q_grad, lr):
    # Your code here
    pass

4 test cases · run your code against them in the editor