DQfD Margin

Problem

Compute the DQfD margin loss for one demonstration transition.

Return max(0, max_other_Q + margin - Q(expert_action)), where max_other_Q is the largest Q value among the non-expert actions. A single-action vector gives 0.0.

Hint

The margin keeps the expert action ahead of all alternatives by a fixed gap.

Test cases

case 1

in: [[1,3],0,0.8]

expected: 2.8

case 2

in: [[5,3],0,0.8]

expected: 0

case 3

in: [[2],0,1]

expected: 0

Starter codePython

def dqfd_margin(q_values, expert_action, margin):
    # Your code here
    pass

3 test cases · run your code against them in the editor