Perplexity Ratio of Two Models

Problem

Given two lists of per-token negative log-likelihoods in nats, return the perplexity ratio of model B over model A. The ratio equals exp(mean(NLL_B) - mean(NLL_A)). Return 1.0 if either list is empty.

Hint

Perplexity is exp of the mean negative log-likelihood, so the ratio is exp of the difference of means.

Test cases

case 1

in: [[1,1],[2,2]]

expected: 2.718281828459045

case 2

in: [[0,0],[0,0]]

expected: 1

case 3

in: [[2],[0]]

expected: 0.1353352832366127

case 4

in: [[],[1]]

expected: 1

case 5

in: [[1,3],[2,2]]

expected: 1

Starter codePython

import math
def perplexity_ratio(nll_a, nll_b):
    # Your code here
    pass

5 test cases · run your code against them in the editor