Minibatch Ratio Recompute

Problem

Recompute PPO probability ratios for consecutive minibatches.

Form ratio[i] = exp(new_log_probs[i] - old_log_probs[i]) and return the mean ratio inside each chunk of minibatch_size samples, keeping the final partial chunk. A non-positive minibatch size or an empty batch returns [].

Hint

Ratios must be recomputed under the current policy at every PPO epoch.

Test cases

case 1

in: [[-0.1,-0.2,-0.3],[-0.2,-0.2,-0.2],2]

expected: [1.0525854590378239,0.9048374180359596]

case 2

in: [[0,0,0,0],[0,0,0,0],2]

expected: [1,1]

case 3

in: [[-0.5],[-1],0]

expected: []

case 4

in: [[],[],2]

expected: []

Starter codePython

import math
def minibatch_mean_ratios(new_log_probs, old_log_probs, minibatch_size):
    # Your code here
    pass

4 test cases · run your code against them in the editor