KL of Bernoulli Natural Parameters

Problem

Return the KL divergence in nats between two Bernoulli distributions given their natural parameters eta1 and eta2. Convert each with p = 1 / (1 + exp(-eta)) and evaluate

D = p * ln(p / q) + (1 - p) * ln((1 - p) / (1 - q))

Hint

KL divergence in the natural parameterization is a Bregman divergence.

Test cases

case 1

in: [0,0]

expected: 0

case 2

in: [1,0]

expected: 0.11094407167172735

case 3

in: [-1,1]

expected: 0.46211715726000974

case 4

in: [2,0.5]

expected: 0.16834459010395766

Starter codePython

import math
def kl_bernoulli_natural(eta1, eta2):
    # Your code here
    pass

4 test cases · run your code against them in the editor