MediumNLPnlp-315

Focal Loss for Token Classification

Problem

Compute the focal loss for one token: -alpha * (1 - p_t) ** gamma * log(p_t), where p_t is the softmax probability of the target class.

logits is the per-class score list, target is the correct class index, gamma is the focusing parameter, and alpha is the class weight. Compute the softmax with the max-subtraction trick and return 0.0 for empty logits.

Hint

When gamma is 0 the focal loss reduces to weighted cross-entropy.

Test cases

case 1

in: [[2,0],0,0,1]

expected: 0.12692801104297263

case 2

in: [[2,0],0,2,1]

expected: 0.0018035628352403813

case 3

in: [[0,0],0,1,0.5]

expected: 0.17328679513998632

case 4

in: [[0,0],1,0,1]

expected: 0.6931471805599453

case 5

in: [[1,1],0,3,1]

expected: 0.08664339756999316

Starter codePython

import math
def focal_loss_token(logits, target, gamma, alpha):
    # Your code here
    pass

5 test cases · run your code against them in the editor