GAIL Discriminator Step

Problem

Take one gradient step on a GAIL discriminator weight vector.

The binary cross-entropy gradient with respect to the weights is (d_value - label) * features, so return weights - lr * (d_value - label) * features. label is 1.0 for expert samples and 0.0 for policy samples.

Hint

The sign flips between expert and policy samples, pushing the scores apart.

Test cases

case 1

in: [[1,1],[2,0],1,0.8,0.1]

expected: [1.04,1]

case 2

in: [[1,1],[2,0],0,0.3,0.5]

expected: [0.7,1]

case 3

in: [[0.5,-0.5],[1,1],1,1,0.2]

expected: [0.5,-0.5]

Starter codePython

def gail_discriminator_update(weights, features, label, d_value, lr):
    # Your code here
    pass

3 test cases · run your code against them in the editor