Softmax Exploration Probabilities

Problem

Compute softmax exploration probabilities over action values.

Return exp((q - max(q)) / temperature) normalized to sum to 1, subtracting the max for numerical stability. If temperature is not positive, return a one-hot list at the argmax with the lowest index winning ties. An empty input returns [].

Hint

Lower temperature concentrates probability on the best action.

Test cases

case 1

in: [[1,2,3],1]

expected: [0.09003057317038046,0.24472847105479764,0.6652409557748218]

case 2

in: [[1,1,1],1]

expected: [0.3333333333333333,0.3333333333333333,0.3333333333333333]

case 3

in: [[-1,0,1],0.5]

expected: [0.015876239976466765,0.11731042782619838,0.8668133321973349]

case 4

in: [[1,2],0]

expected: [0,1]

case 5

in: [[],1]

expected: []

Starter codePython

def softmax_exploration(q_values, temperature):
    # Your code here
    pass

5 test cases · run your code against them in the editor