Variation of Information

Problem

Compute the variation of information in bits between the two variables of a joint distribution joint[x][y]:

VI = H(X|Y) + H(Y|X)

Build the conditionals from the row and column marginals. Skip zero-probability rows or columns.

Hint

VI is a metric: it is 0 only when the variables determine each other perfectly.

case 1

in: [[[0.25,0.25],[0.25,0.25]]]

expected: 2

case 2

in: [[[0.5,0],[0,0.5]]]

expected: 0

case 3

in: [[[0.1,0.2],[0.3,0.4]]]

expected: 1.8406371956566698

case 4

in: [[[0.5,0.25],[0,0.25]]]

expected: 1.188721875540867

Starter codePython

import math
def variation_of_information(joint):
    # Your code here
    pass

4 test cases · run your code against them in the editor