Dueling Split

Problem

Combine a state value and advantages into Q values using the dueling architecture.

If mode is "max", subtract max(advantage); otherwise subtract the mean of the advantages. Then Q(a) = value + advantage[a] - baseline.

Hint

Subtracting a baseline makes the advantage stream identifiable without changing the argmax.

case 1

in: [5,[1,2,3],"mean"]

expected: [4,5,6]

case 2

in: [5,[1,2,3],"max"]

expected: [3,4,5]

case 3

in: [0,[-1,0,1],"mean"]

expected: [-1,0,1]

case 4

in: [2,[4,4],"max"]

expected: [2,2]

Starter codePython

def dueling_combine(value, advantage, mode):
    # Your code here
    pass

4 test cases · run your code against them in the editor