HardNLPnlp-177

Bootstrap CI for Accuracy

Problem

Return a seeded bootstrap confidence interval for accuracy. Resample flags with replacement n_resamples times using random.Random(seed), compute the accuracy of each resample, sort them, and return [lower, upper] at indices int(0.025 * n_resamples) and int(0.975 * n_resamples).

Return [0.0, 0.0] for empty input.

Hint

Run the resampling loop in a fixed order so the seeded sequence is reproducible.

Test cases

case 1

in: [[1,1,1,0],100,42]

expected: [0.25,1]

case 2

in: [[1,1,1,0],10,1]

expected: [0,1]

case 3

in: [[1,1],50,3]

expected: [1,1]

case 4

in: [[],10,1]

expected: [0,0]

case 5

in: [[0],7,2]

expected: [0,0]

Starter codePython

import random
def bootstrap_ci_seeded(flags, n_resamples, seed):
    # Your code here
    pass

5 test cases · run your code against them in the editor