Variational Free Energy

Problem

Return the variational free energy in nats:

F = sum_i q_i * (ln(q_i) - ln p_i)

over states with q_i > 0, where q_probs is the variational posterior and log_p_joint gives ln p(x, z). Free energy is the negative evidence lower bound.

Hint

Free energy equals negative log evidence plus KL(q || posterior).

Test cases

case 1

in: [[0.5,0.5],[0,0]]

expected: -0.6931471805599453

case 2

in: [[1,0],[0,-100]]

expected: 0

case 3

in: [[0.5,0.5],[-1,-1]]

expected: 0.3068528194400547

case 4

in: [[0.25,0.75],[-2,-0.5]]

expected: 0.3126648553811917

Starter codePython

import math
def free_energy_decomposition(q_probs, log_p_joint):
    # Your code here
    pass

4 test cases · run your code against them in the editor