PUCT Exploration Term

Problem

Compute the exploration term that AlphaZero adds to a child's action value.

Return c * prior * sqrt(parent_visits + 1) / (1 + child_visits). Adding one inside the square root and the denominator keeps the term finite for unvisited children.

Hint

The prior scales how strongly an unvisited but promising child is pushed.

Test cases

case 1

in: [0.5,100,10,1]

expected: 0.45681252823276774

case 2

in: [0.25,4,0,2]

expected: 1.118033988749895

case 3

in: [0.5,0,0,1]

expected: 0.5

case 4

in: [0.1,25,3,0.5]

expected: 0.06373774391990981

Starter codePython

import math
def puct_exploration_term(prior, parent_visits, child_visits, c):
    # Your code here
    pass

4 test cases · run your code against them in the editor