Optimistic Initialization Effect

Problem

Simulate a deterministic multi-armed bandit to measure how long optimistic initialization drives exploration.

All Q values start at q_init and pulling arm a returns its true mean. At each step choose the greedy arm (smallest index on ties), update its sample average, and stop the first time the chosen arm is a true optimal arm. Return the 1-based step number, or -1 if the optimal arm is not chosen within 1000 steps.

Hint

High initial values make every arm look promising until it has been sampled down.

Test cases

case 1

in: [[1,2,3],5]

expected: 3

case 2

in: [[1,2,3],0]

expected: -1

case 3

in: [[1,2,3],10]

expected: 3

case 4

in: [[3,2,1],0]

expected: 1

case 5

in: [[5,5],10]

expected: 1

Starter codePython

def optimistic_init_steps(means, q_init):
    # Your code here
    pass

5 test cases · run your code against them in the editor