SHOT: Unique Signatures of Histograms for Local Surface Description

The ability of computing similarities between 3D surfaces (surface matching) is a key for computer vision tasks such as 3D object recognition and surface alignment. These tasks find numerous applications in fields such as robotics, automation, biometric systems, reverse engineering, search in 3D object databases. The most popular trend for surface matching exploits a compact local representation of the input data, known as descriptor, and shares basic motivations with the successful approaches for matching 2D images that rely on local invariant features. Local correspondences established by matching 3D descriptors (Fig. 1) can then be used to solve higher level tasks such as 3D object recognition. This approach allows for dealing effectively with issues such as occlusion, clutter and changes of viewpoint. As a result, a variety of proposals for 3D descriptors can be found in recent literature.

Fig. 1

Our first contribution is a specific study upon local 3D Reference Frames (RFs) for 3D descriptors. We have carried out an analysis of repeatability and robustness on proposed local RFs, and provided experiments that demonstrate the strong impact of the choice of the RF on the performance of a 3D descriptor. Given the impact of such a choice, our second contribution is a robust local RF that, unlike all other proposals, is unique and unambiguous. Our third contribution is a novel 3D descriptor. Its design, inspired by an analysis of the successful choices performed in the related field of 2D descriptors, has been explicitly conceived to achieve computational efficiency, descriptive power and robustness.

The design of the descriptor naturally allows for simultaneously encoding multiple cues. e.g. shape and texture. In [2] we studied what are the most effective and efficient ways to include texture information in the descriptor. The inclusion of the texture cue results in more effective descriptions, with limited impact on efficiency. Therefore, the extended version of our descriptor qualifies as a natural tool to attain point-to-point correspondences in RGB-D data, such as that provided by the recently released Microsoft Kinect.

Finally, in [3] we summarize all the research activity carried out on this topic by presenting all the details and pseudo-code to reproduce the methods, an enriched and revised taxonomy of 3D descriptors, and an extended comparative evaluation which adds to the previous results three additional prominent proposals, a novel dataset acquired by means of a Kinect sensor and a study on the performance of the considered descriptors in a shape retrieval scenario.

Dataset

It is possible to download all the datasets used for the experimental evaluation in [1], [2] and [3]. All files contained in the datasets are in PLY format.

Descriptor Matching

All the datasets in this section are relative to the descriptor matching experiments of our paper. They include several models and scenes, containing some models as well as other objects (clutter). Clutter objects and models are partially occluding each other.

The datasets also include, for each scene:

a "Config" file (ConfigSceneXX.ini), defining the number of models and the scene file name for that scene. It also defines, for each model present in that scene, the model file names and the model groundtruth files.
as many groundtruth files as the models in that scenes (SceneXX-rsY.xf), each defining the rotation and translation to be applied to each model in order to align it to its corresponding instance in each scene.

Descriptor Matching - Dataset 1 & 2 (Stanford)

These datasets, created from some of the models belonging to the Stanford 3D Scanning Repository, are composed of 6 models and 45 scenes. Each of the 45 scenes contains a subset of the 6 models randomly chosen and randomly rotated and translated so to create clutter conditions. In particular, out of the 45 scenes the dataset contains:

15 scenes with 3 models each
15 scenes with 4 models each
15 scenes with 5 models each

Dataset 1 is used in [1] for Experiment 1 (with additional Gaussian noise on each scene, at 3 different levels of noise) and dataset 2 in Experiment 2.

The models of these datasets can be downloaded either as:

a single ZIP file (7 MB)
a single RAR file (6,5 MB)

The scenes of dataset 1 can be downloaded either as:

a single ZIP file (201 MB)
a single RAR file (173 MB)

The scenes of dataset 2 can be downloaded either as:

a single ZIP file (25 MB)
a single RAR file (21,5 MB)

Descriptor Matching - Dataset 3 (Spacetime Stereo)

This dataset, acquired in our lab by means of the Spacetime Stereo technique, is composed of 8 models and 15 scenes, divided into 4 sets as follows:

Set 1: 2 models, 4 scenes
Set 2: 2 models, 4 scenes
Set 3: 2 models, 4 scenes
Set 4: 2 models, 3 scenes