MuZero Value+Policy Target Lite

Problem

Build MuZero targets for one search: a discounted reward value and a normalized visit-count policy.

value = sum of gamma^t * rewards[t], and policy = counts / sum(counts). If all counts are zero return an all-zero policy. Return [value, policy].

Hint

The value target can be any n-step return; visit counts give the policy target.

case 1

in: [[1,1],0.9,[4,2,2]]

expected: [1.9,[0.5,0.25,0.25]]

case 2

in: [[2],0.5,[1,0]]

expected: [2,[1,0]]

case 3

in: [[],0.9,[3,1,0,0]]

expected: [0,[0.75,0.25,0,0]]

Starter codePython

def muzero_targets(rewards, gamma, visit_counts):
    # Your code here
    pass

3 test cases · run your code against them in the editor