Target KL Controller Step

Problem

Adapt the KL coefficient from the current KL: multiply coef by factor when current_kl > 1.5 * target_kl, divide by factor when current_kl < target_kl / 1.5, and keep it unchanged otherwise. A non-positive target_kl or factor returns coef.

Hint

This two-sided controller holds the measured KL near its target band.

Test cases

case 1

in: [0.1,0.05,1,2]

expected: 2

case 2

in: [0.01,0.05,1,2]

expected: 0.5

case 3

in: [0.05,0.05,1,2]

expected: 1

case 4

in: [0.1,0,1,2]

expected: 1

Starter codePython

def target_kl_controller(current_kl, target_kl, coef, factor):
    # Your code here
    pass

4 test cases · run your code against them in the editor