All problems
Nucleus Cutoff Index
Problem
Return how many tokens nucleus (top-p) sampling keeps from a probability list. Sort the probabilities descending and add them one at a time until the running sum reaches or exceeds p.
Return the number of tokens added, or 0 when probs is empty or p is not positive. If the total probability is below p, all tokens are kept.
Hint
Stop as soon as the cumulative mass reaches p and report the count.
Test cases
case 1
in: [[0.4,0.3,0.2,0.1],0.5]
expected: 2
case 2
in: [[0.25,0.25,0.25,0.25],0.5]
expected: 2
case 3
in: [[0.25,0.25,0.25,0.25],0.9]
expected: 4
case 4
in: [[0.9,0.1],0.1]
expected: 1
case 5
in: [[],0.5]
expected: 0
Starter codePython
def nucleus_cutoff_index(probs, p):
# Your code here
pass5 test cases · run your code against them in the editor