What in an image lets me measure motion?
Start with one row of pixels. Call its brightness $I(x)$, where $x$ is measured in pixels. Treat brightness as smooth between the samples for now.
Let $I_1$ and $I_2$ be the first and second images. Suppose the pattern moves right by $u$ pixels. If its brightness does not change,
\[I_2(x+u) = I_1(x).\]For a small displacement,
\[I_2(x+u) \approx I_2(x) + I'_2(x)u.\]$I’_2(x)$ is the local brightness gradient, measured in brightness units per pixel.
Writing $\Delta I = I_2(x)-I_1(x)$ gives
\[\Delta I + I'_2(x)u \approx 0.\]If the gradient is 10 brightness units per pixel and the pixel gets 5 units darker, this predicts a half-pixel shift to the right.
If the gradient is zero, this equation tells me nothing about the displacement. A uniform patch can move without changing its appearance. A lighting change could also change brightness, so unchanged appearance is an assumption here.
What happens when motion has two components, $u$ and $v$, but a pixel still gives only one brightness value?