Deformable Attention Sample Points

Problem

Compute the sampling locations of deformable attention: each location is reference + offset, clamped to the normalized [0, 1] range.

The signature is deformable_sample_points(reference, offsets). Return the list of [x, y] locations.

Hint

Offsets are usually predicted per query, head and sampling point, then interpolated bilinearly.

case 1

in: [[0.5,0.5],[[0.1,0],[-0.1,0.2],[0,-0.6]]]

expected: [[0.6,0.5],[0.4,0.7],[0.5,0]]

case 2

in: [[0,1],[[-0.5,0.5]]]

expected: [[0,1]]

case 3

in: [[0.5,0.5],[]]

expected: []

case 4

in: [[0.9,0.1],[[0.2,-0.2],[-1,1]]]

expected: [[1,0],[0,1]]

Starter codePython

def deformable_sample_points(reference, offsets):
    # Your code here
    pass

4 test cases · run your code against them in the editor