Krippendorff Alpha Lite

Problem

Compute nominal Krippendorff's alpha for a list of units, where each unit is a list of ratings from different coders. Build the coincidence matrix with weight 1 / (m_u - 1) per ordered within-unit pair, then return 1 - Do / De, where Do is off-diagonal coincidence mass divided by the total number of ratings and De is the expected off-diagonal mass under independence. Return 0.0 when fewer than two ratings exist or De is zero.

Hint

The coincidence matrix mass equals the number of ratings when units are complete.

Test cases

case 1

in: [[["a","a"],["b","b"],["a","b"]]]

expected: 0.4444444444444444

case 2

in: [[["a","a"],["a","a"]]]

expected: 0

case 3

in: [[["a"],["b"]]]

expected: 0

case 4

in: [[["a","b"],["b","a"],["a","a"]]]

expected: -0.25

Starter codePython

def krippendorff_alpha_lite(units):
    # Your code here
    pass

4 test cases · run your code against them in the editor