Robust Visual Correspondence

This research topic deals with the study of novel matching measures to robustly deploy visual correspondence with regards to typical disturbance factors found in real application scenarios. Robust matching measures are very important for many basic computer vision tasks such as change detection, pattern matching, image registration, stereo vision. The disturbance factors which typically occur in these tasks are photometric distortions, noise and occlusions.

Proposed approach

We have been developing [1] novel measures implicitly based on the so-called order consistency, or order-preservation hypothesis, that is the presence of the considered distortions does not violate the ordering between neighboring pixels. In particular, the proposed approach aims to quantify how well the order is preserved between corresponding pairs of neighboring pixels in a 3x3 patch centered at the two candidates for which correspondence is being evaluated. To this purpose, for both candidates we consider the subset of pixel pairs shown in the Figure below, and compute the Normalized Cross-Correlation of their brightness differences. Studies on how to efficiently implement these measures were also performed.

Applications

Preliminary results are shown dealing with the application of our robust measures to pattern matching and motion detection.

Pattern Matching

In the pattern matching application, the location of a pattern has to be identified in a image, whereas between pattern and image strong distortions occurr, such as photometric distortions, slight changes in pose, noise, partial occlusions of the pattern. The first experiment shown below deals with 2 patterns, each to be found within 4 images. The 4 images are subject to strong illumination changes as well as to high noise due to the cheap camera sensor used. Instead, experiment 2 deals with 1 single pattern to be seeked within 8 images. In this Case the main distortions are represented by partial occlusions, though illumination changes, changes in pose and noise are still notable. 

NOTE: Both experiments concerns a dataset which has been deployed for a performance evaluation between robust measures in the pattern matching context. You can find more about this work and download the dataset here.

Experiment 1: 4 Images, 2 Templates = 8 matching instances

Distortions: illumination changes, noise, slight occlusions, JPEG compression

Our approach: 8 correct correspondences out of 8

Experiment 2: 8 Images, 1 Template = 8 matching instances

Distortions: mainly occlusions, illumination changes, noise, JPEG compression 

Our approach: 7 correct correspondences out of 8

Change detection for a security application

This application deals with singularisation and empty space control in interlocks, in particular it aims at monitoring access to a high security gate. For this aim, we apply a change detection algorithm to incoming frames in order to detect foreground regions by comparison against a background image (taken when the gate is empty). The change detection algorithm relies on the proposed robust measure: at each point of the current frame and background image is computed on the window centred on the current point, then a threshold is used to discriminate background points from foreground points.

Images are subject to heavy photometric distortions due to reflections on the gate floor, changes in indoor illumination and light coming from outside. The figure below shows the results dealing with 3 frames acquired in different moments and with different subjects are compared with the same background image acquired previously (shown on the left). The 3 images on the right show the shape of the region detected as the gate floor using a fixed set of parameters and, as post-processing, a fixed sequence of simple morphological operators such as erosion and dilation. Results show that the proposed measure is able to extract a good shape of the gate floor with good robustness towards the ongoing disturbance factors.

Change detection robust to sudden illumination changes

In [2] a novel approach to deal with the problem of accurately segment the foreground from the background under the presence of heavy illuminations changes was proposed. Please click here for more details.

References