TD3+BC Policy Term

Problem

Compute the TD3+BC policy loss for one sample.

lambda = alpha / mean_abs_q (0 when mean_abs_q is 0) and the loss is -lambda * q_value + (action - data_action)^2, balancing Q maximization against staying close to the dataset action.

Hint

Normalizing lambda by the average Q magnitude keeps the BC term competitive across tasks.

Test cases

case 1

in: [1.5,1,2,2.5,10]

expected: -0.25

case 2

in: [0,1,-3,1,0]

expected: 1

case 3

in: [2,2,5,0.5,4]

expected: -0.625

Starter codePython

def td3bc_policy_term(action, data_action, q_value, alpha, mean_abs_q):
    # Your code here
    pass

3 test cases · run your code against them in the editor