Novelty Benchmark

Problem

Measure the fraction of test states that were never visited: count test_states not in visited_states divided by the number of test states. An empty test set returns 0.0.

Hint

A novelty benchmark checks behavior on states the agent has not seen.

Test cases

case 1

in: [["a","b","c"],["a","x"]]

expected: 0.6666666666666666

case 2

in: [["a","b"],["a","b"]]

expected: 0

case 3

in: [[],["a"]]

expected: 0

Starter codePython

def novelty_benchmark(test_states, visited_states):
    # Your code here
    pass

3 test cases · run your code against them in the editor