Best-of-N Pass At K

Problem

Compute the unbiased pass@k estimator from n generated samples with c correct ones: 1 - C(n - c, k) / C(n, k). Return 1.0 when n - c < k or c >= n, and 0.0 when n <= 0 or k <= 0. Use math.comb.

Hint

The combinatorial form is an unbiased estimate, unlike simply counting pass rates.

Test cases

case 1

in: [5,2,2]

expected: 0.7

case 2

in: [10,2,1]

expected: 0.2

case 3

in: [5,0,3]

expected: 0

case 4

in: [3,3,1]

expected: 1

case 5

in: [5,1,5]

expected: 1

Starter codePython

import math
def pass_at_k(n, c, k):
    # Your code here
    pass

5 test cases · run your code against them in the editor