Deterministic Information Bottleneck

Problem

Evaluate the deterministic information bottleneck objective for a joint pxy and a hard assignment of each x to a cluster t (assignment[x]). Return

I(X; T) - beta * I(T; Y)

in bits, where I(X; T) is the entropy of the cluster marginal because T is a deterministic function of X.

Hint

Collapsing the classes reduces I(X; T) but also destroys predictive information.

case 1

in: [[[0.5,0],[0,0.5]],[0,1],1]

expected: 0

case 2

in: [[[0.5,0],[0,0.5]],[0,0],1]

expected: 0

case 3

in: [[[0.5,0],[0,0.5]],[0,1],2]

expected: -1

case 4

in: [[[0.4,0.1],[0.1,0.4]],[0,1],1]

expected: 0.7219280948873623

Starter codePython

import math
def deterministic_information_bottleneck(pxy, assignment, beta):
    # Your code here
    pass

4 test cases · run your code against them in the editor