Options Termination Value

Problem

Compute the expected discounted value of an option that terminates each step with probability beta.

While active the option collects rewards; after each reward it terminates with probability beta, collecting value_after one step later, and if it survives the whole sequence it collects value_after at the end. Return the expectation.

Hint

Weight each step by the survival probability (1 - beta) raised to that step.

Test cases

case 1

in: [[1,1],0.5,0.9,10]

expected: 10

case 2

in: [[1,1,1],0,0.5,4]

expected: 2.25

case 3

in: [[2],1,0.9,5]

expected: 6.5

case 4

in: [[0],0.2,1,0]

expected: 0

Starter codePython

def option_termination_value(rewards, beta, gamma, value_after):
    # Your code here
    pass

4 test cases · run your code against them in the editor