Actor-Critic Update

Problem

Apply one actor-critic update step and return [new_log_prob, new_value].

First compute the TD error delta = reward + gamma * next_value - value. Then nudge the log-probability by actor_lr * delta and update the critic by critic_lr * delta.

Hint

The same TD error scales both the actor step and the critic step.

Test cases

case 1

in: [-0.5,2,3,1,0.9,0.1,0.2]

expected: [-0.31,2.38]

case 2

in: [0,0,0,0,0.99,0.05,0.1]

expected: [0,0]

case 3

in: [-1.5,4,-2,1,0.5,0.2,0.3]

expected: [-2.6,2.35]

Starter codePython

def actor_critic_update(log_prob, value, reward, next_value, gamma, actor_lr, critic_lr):
    # Your code here
    pass

3 test cases · run your code against them in the editor