Soft Value Update

Problem

Compute the soft (log-sum-exp) value of an action-value vector with temperature alpha.

V = alpha * log(sum of exp(Q[a] / alpha)) computed with the max-subtraction trick. If alpha is not positive, return max(Q).

Hint

Subtract the max before exponentiating so large values stay stable.

case 1

in: [[0,0],1]

expected: 0.6931471805599453

case 2

in: [[1,2,3],1]

expected: 3.4076059644443806

case 3

in: [[0,0],0.5]

expected: 0.34657359027997264

case 4

in: [[5],1]

expected: 5

case 5

in: [[1000,1000],1]

expected: 1000.6931471805599

Starter codePython

import math
def soft_value_update(Q, alpha):
    # Your code here
    pass

5 test cases · run your code against them in the editor