Nested CV Score

Problem

Evaluate a nested cross-validation run. For each outer fold, select the hyperparameter with the highest inner score (smallest index on ties) and read the outer score at that position. Return the mean of the selected outer scores, or 0.0 when no fold can be scored.

Hint

The inner loop picks hyperparameters; the outer loop estimates how well that whole procedure generalizes.

Test cases

case 1

in: [[[0.8,0.9],[0.7,0.6]],[[0.5,0.4],[0.3,0.2]]]

expected: 0.35

case 2

in: [[[0.9,0.9],[0.5]],[[1,0],[0.5]]]

expected: 0.75

case 3

in: [[[],[0.2]],[[9.9],[0.8]]]

expected: 0.8

case 4

in: [[],[]]

expected: 0

case 5

in: [[[0.5,0.6]],[[0.1]]]

expected: 0

Starter codePython

def nested_cv_score(inner_scores, outer_scores):
    # Your code here
    pass

5 test cases · run your code against them in the editor