Permutation Importance MSE

Problem

Measure permutation importance as the MSE increase when a feature column is shuffled. Build base predictions w0 + x . w, then shuffle column `feature` with random.seed(seed) and random.shuffle, recompute the MSE, and return permuted_mse - base_mse.

Empty X returns 0.0.

Hint

Break the relationship between the feature and the target by shuffling, then measure the harm.

Test cases

case 1

in: [[[1],[2],[3],[4]],[2,4,6,8],0,[2],0,42]

expected: 14

case 2

in: [[[1],[2],[3],[4]],[3,5,7,9],0,[2],0,7]

expected: 14

case 3

in: [[[1,2],[2,1],[3,4],[4,3]],[1,1,2,2],0.5,[0.5,0.5],1,0]

expected: 0.125

case 4

in: [[],[],0,[1],0,1]

expected: 0

Starter codePython

import random
def permutation_importance_mse(X, y, w0, w, feature, seed):
    # Your code here
    pass

4 test cases · run your code against them in the editor