Research/Local-motion based tracking

The purely intensity-based measure of presence cannot deal with situations in which the tracked object is occluded by another, visually-similar object. This is demonstrated in the video example below.

Video example of color-only tracking of a hand

The cause of failure is the color ambiguity which is shown in the figure below: here we see an example of tracking a hand, and along with it we also see an example of the color-based likelihood function, which corresponds to the histogram of the tracked hand. Note that the likelihood function is bimodal; one mode corresponds to the tracked hand and the other mode corresponds to the other hand. This in effect introduces the ambiguity in the hand’s position and causes tracker to fail.


We therefore propose to use the optical flow to resolve the color-induced ambiguity. An example of this is shown below, where we use the notion that the tracked hand is moving to the left (provided by the tracker) and the other hand is not. By comparing the direction of the hand’s motion to the observed optical, we can generate the local-motion-based likelihood function. When this function is combined to the color-based likelihood function, only the mode which corresponds to the tracked hand remains. Thus the ambiguity is resolved.


In our implementation we use the pyramid implementation of the Lucas-Kanade optical flow, which produces poorly estimated optical flow in poorly textured regions. We therefore first determine the regions in the image which contain enough texture and compute the optical flow vectors only at those positions — these are taken as the valid flow vectors. The image below shows extraction of the valid flow vectors and their superposition on the original image.

An example of tracking with a purely color-based visual model and the with the proposed local-motion/color-based visual model is shown below. While the tracker which used the purely color-based model fails 14 times, the local-motion/color-based model fails only twice.

Reference model (avi)
Proposed model (avi)

A tracker which used the local-motion feature was applied to a variety of examples from surveillance to sports to evaluate different properties of local-motion feature and the scheme for its adaptation.

For further results and video examples, see the following page:

and the paper: “ “A Local-motion-based probabilistic model for visual tracking” published at the Pattern Recognition.

Video Presentation:

Relevant Publications:

   A Local-motion-based probabilistic model for visual tracking
   Authors: M. Kristan, J. Perš, S. Kovačič and A. Leonardis
   Published in: Pattern Recognition
Tracking people in video data using probabilistic models
   Authors: M. Kristan
   Published in: Phd thesis