CMDP Dual Step

Problem

Apply one constrained MDP dual step and return the penalized reward plus the new multiplier.

penalized = reward - lambda * cost and new_lambda = max(0, lambda + lr * (cost - limit)). Return [penalized, new_lambda].

Hint

The current multiplier penalizes the reward before the dual update.

case 1

in: [1,2,0.5,1,0.1]

expected: [0,0.6]

case 2

in: [2,0,1,0.5,0.5]

expected: [2,0.75]

case 3

in: [-1,3,0,2,0.2]

expected: [-1,0.2]

case 4

in: [0,0,2,1,1]

expected: [0,1]

Starter codePython

def cmdp_dual_step(reward, cost, lambda_, limit, lr):
    # Your code here
    pass

4 test cases · run your code against them in the editor