ELBO Value

Problem

For a Gaussian mixture with variational responsibilities r, the evidence lower bound is the sum over points n and components k of r[n][k] * (log(weights[k]) + log N(data[n]; means[k], sigmas[k]^2) - log(r[n][k])). Terms with zero responsibility contribute nothing. Return the ELBO, or 0.0 when the inputs are empty, mismatched, or a used component has a nonpositive weight or standard deviation.

Hint

The ELBO is the Q-function plus the entropy of the responsibilities.

Test cases

case 1

in: [[1],[0],[1],[0],[[1]]]

expected: -0.9189385332046727

case 2

in: [[0.5,0.5],[0,1],[1,1],[0,1],[[0.8,0.2],[0.3,0.7]]]

expected: -2.362904701936155

case 3

in: [[0.3,0.7],[-1,1],[1,2],[0,2],[[0.5,0.5],[0.25,0.75]]]

expected: -4.328902000098392

case 4

in: [[0.5,0.5],[0,0],[1,1],[],[]]

expected: 0

case 5

in: [[0.5,0.5],[0,0],[1,1],[0],[[1,0]]]

expected: -1.612085713764618

Starter codePython

import math


def gmm_elbo_value(weights, means, sigmas, data, responsibilities):
    # Your code here
    pass

5 test cases · run your code against them in the editor