Quiver plot with optical flow?

Question

Recently I'm working in cloud motion tracking using images, but in many examples when is used in video implementations shows a quiver plot that moves according the object tracked.

Quiver documentations takes four argumets principally ([X, Y], U, V), when X and Y are the starting points and U and V the directions. In the other hand, optical flow based on this example returnsp1 (the displacements) with a shape (m, n, l) of the image with shape of (200,200). My confusion is in how to order the parameters, because also goodFeaturesToTrack return the same as p1

¿How can I join both components to plot a quiver of the cloud motion?

user_7946792 · Answer

I found a pretty good solution. I explain all my example here using the Hamburg taxi sequence:

Download the taxi sequence.

$ curl -O ftp://ftp.ira.uka.de/pub/vid-text/image_sequences/taxi/taxi.zip
$ unzip -q taxi.zip

Get all images and pick two random frames

from pathlib import Path
import numpy as np
import cv2 as cv
from PIL import Image
import matplotlib.pyplot as plt

taxis_fnames = list(Path('taxi').iterdir())
taxi1 = Image.open(taxis_fnames[rand_idx])
taxi2 = Image.open(taxis_fnames[rand_idx + 4])

Compute the optical flow

flow = cv.calcOpticalFlowFarneback(np.array(taxi1), 
                                   np.array(taxi2), 
                                   None, 0.5, 3, 15, 3, 5, 1.2, 0)

Plot the quiver

step = 3
plt.quiver(np.arange(0, flow.shape[1], step), np.arange(flow.shape[0], -1, -step), 
           flow[::step, ::step, 0], flow[::step, ::step, 1])

The step is to downsample the number of optical flow vectors picked. The x positions goes from 0 to image width, while the y positions are inversed (otherwise the optical flow will be up side down) from image height to 0. In some occasions, you will have to change the step so the height and with are divisible by it.

The resulting image:

user_365102 · Answer

Here is a general method for plotting a quiver field easily and accurately.

def plot_quiver(ax, flow, spacing, margin=0, **kwargs):
    """Plots less dense quiver field.

    Args:
        ax: Matplotlib axis
        flow: motion vectors
        spacing: space (px) between each arrow in grid
        margin: width (px) of enclosing region without arrows
        kwargs: quiver kwargs (default: angles="xy", scale_units="xy")
    """
    h, w, *_ = flow.shape

    nx = int((w - 2 * margin) / spacing)
    ny = int((h - 2 * margin) / spacing)

    x = np.linspace(margin, w - margin - 1, nx, dtype=np.int64)
    y = np.linspace(margin, h - margin - 1, ny, dtype=np.int64)

    flow = flow[np.ix_(y, x)]
    u = flow[:, :, 0]
    v = flow[:, :, 1]

    kwargs = {**dict(angles="xy", scale_units="xy"), **kwargs}
    ax.quiver(x, y, u, v, **kwargs)

    ax.set_ylim(sorted(ax.get_ylim(), reverse=True))
    ax.set_aspect("equal")

Example usage:

flow = cv2.calcOpticalFlowFarneback(
    frame_1, frame_2, None, 0.5, 3, 15, 3, 5, 1.2, 0
)

fig, ax = plt.subplots()
plot_quiver(ax, flow, spacing=10, scale=1, color="#ff44ff")

Quiver plot with optical flow?

2 Answers