AlphaZero Visit Counts

Problem

Turn MCTS visit counts into an improved policy using a temperature exponent.

weights[a] = counts[a] ** (1 / temperature) and the result is normalized to sum to 1. If the total weight is zero return a uniform distribution over the action list.

Hint

Temperature below 1 sharpens the visit distribution; temperature above 1 flattens it.

Test cases

case 1

in: [[10,0,0],1]

expected: [1,0,0]

case 2

in: [[4,1,1],1]

expected: [0.6666666666666666,0.16666666666666666,0.16666666666666666]

case 3

in: [[4,1,1],0.5]

expected: [0.8888888888888888,0.05555555555555555,0.05555555555555555]

case 4

in: [[1,1,1],2]

expected: [0.3333333333333333,0.3333333333333333,0.3333333333333333]

Starter codePython

def alphazero_visit_policy(counts, temperature):
    # Your code here
    pass

4 test cases · run your code against them in the editor