Inverse RL Max-Entropy Step Lite

Problem

Take one max-entropy inverse RL gradient step on the reward weights.

The gradient is the expert feature expectation minus the current policy feature expectation, and the weights move toward the expert by lr. Return weights[i] + lr * (expert[i] - policy[i]).

Hint

Max-entropy IRL matches the expert's expected feature counts.

Test cases

case 1

in: [[1,0],[2,3],[1,1],0.1]

expected: [1.1,0.2]

case 2

in: [[0,0],[1,1],[1,1],0.5]

expected: [0,0]

case 3

in: [[-1,2],[0,0],[2,2],0.25]

expected: [-1.5,1.5]

Starter codePython

def maxent_irl_step(weights, expert_feats, policy_feats, lr):
    # Your code here
    pass

3 test cases · run your code against them in the editor