Mini-Batch SGD for Linear Regression

Problem

Fit y = w*x + b using mini-batch stochastic gradient descent with MSE loss.

Initialize w = 0, b = 0. For n_epochs epochs, iterate over X in batches of batch_size (no shuffling).

Per batch of m samples: gw = (2/m) * sum((w*xi + b - yi) * xi) gb = (2/m) * sum(w*xi + b - yi) w = w - lr * gw b = b - lr * gb

Return [w, b].

Hint

Standard gradient: dL/dw = 2*err*xi, dL/db = 2*err. Average over the batch.

case 1

in: [[1,2,3,4,5,6],[2,4,6,8,10,12],0.01,2000,2]

expected: [2,0]

case 2

in: [[1,2,3],[3,5,7],0.05,1000,1]

expected: [2,1]

case 3

in: [[1,2,3],[1,3,5],0.05,2000,1]

expected: [2,-1]

Starter codePython

def sgd_linear(X, y, lr, n_epochs, batch_size):
    # Returns [w, b]
    # Your code here
    pass

3 test cases · run your code against them in the editor