Fast Bilateral Stereo (FBS)

Accurate and efficient cost aggregation strategy for stereo correspondence

This page provides additional experimental results concerned with the Fast Bilateral Stereo (FBS) approach proposed in: 

S. Mattoccia, S. Giardino, A. Gambini, "Accurate and efficient cost aggregation strategy for stereo correspondence based on approximated joint bilateral filtering", Asian Conference on Computer Vision (ACCV2009), September 23- 27, 2009,  Xi'an, China  

The FBS algorithm has been recently mapped [12] on GPUs with CUDA. Compared to the original CPU implementation, the GPU version of FBS on a medium-class NVIDIA 460 GTX GPU, allows to obtain speed-ups > 70X (i.e. 300 ms on Teddy and Cones) with equivalent results. On a high-end Tesla C2070 the measured speed-up is > 100X (i.e. 200 ms on Teddy and Cones) . Additional experimental results are available here.

S. Mattoccia, M. Viti, F. Ries, "Near real-time Fast Bilateral Stereo on the GPU", Best Paper Award at 7th IEEE Workshop on Embedded Computer Vision (ECVW20011), CVPR Workshop, June 20, 2011, Colorado Springs (CO), USA


FBS combines the effectiveness of state-of-the-art cost aggregations strategies, that adapt their weights to image content, with the efficiency of fast incremental calculation schemes (i.e. integral images or box filtering) typically deployed by conventional stereo matching algorithms (often referred to as correlative approaches). FBS computes the weights on a block basis and the matching cost on a point-basis; this strategy enables, in both cases, to deploy efficient box-filtering (or integral images) incremental calculation schemes.


The block basis weight computation dramatically reduces the number of operations and enables to increase robustness to noise. Experimental results show that FBS has an accuracy comparable to top performing state-of-the-art cost aggregation strategies (a complete evaluation of cost aggregation strategies, including FBS, is available here). Nevertheless, the execution time is significantly reduced (you can evaluate the performance of the FBS algorithm downloading the software implementation available below). Moreover, it is noteworthy that the performance of FBS can be significantly improved deploying the LC technique proposed in [11]. 

Further details can be found in the ACCV09 paper and in this presentation 

SOFTWARE
The implementation of the FBS algorithm for Linux and Windows (tested with Linux Ubuntu 10.04 (32 bit) and Windows 7 (32 bit)) is available here: software (last update: August 12th, 2010). Please, read the README.txt file for further details. The code was written in C/C++ and requires (for reading, saving and displaying images) the open source OpenCV library (tested, on Linux and Windows, with version 2.1) available at:

http://sourceforge.net/projects/opencvlibrary/
http://opencv.willowgarage.com/wiki/


If you have any question feel free to contact me, Stefano Mattoccia, at: smatt AT ieee DOT org

 



FBS vs AW, SS, SB, Rel, VW: experimental results on the Middlebury dataset [8]

The FBS approach has been evaluated on the Middlebury site and the results are available here (FBS is referred to as FastBilateral). Moreover, FBS has been compared to state approaches according to the CVPR2008 paper [9]; experimental results are available here. Among state-of-the-art aprpoaches evaluated in [9], FBS is ranked 2nd (see here for details). Below are reported the supports extracted in the 6 points defined in [9]. The FBS approach has also been deployed in conjunction with the Locally Consistent (LC) technique [11] (see here for details) ranking 1st according to the methodology described in the CVPR2008 paper [9]. 


Radius 3x3    Radius 3x3


Radius 5x5    Radius 5x5



Radius 7x7    Radius 7x7



Radius 9x9    Radius 9x9


Below are reported the disparity maps on the Middlebury dataset [7,8] computed by FBS and the 5 top performing cost aggregation strategies according to [9]. For Adaptive Weights (AW) [2], Segment Support (SS) [3], Segmentation Based (SB) [4], Reliability (Rel) [5] and Variable Windows (VW) [6] the disparity maps are concerned with the original algorithms proposed by the authors (these disparity maps are available here, section "Original"). For each algorithm the execution time, on the same Intel Core Duo 2.14 GHz processor, for the Teddy stereo pair is reported. For Segmentation Based [4], differently by [9], we report here updated and improved execution time.



Fast Bilateral Stereo 19(3) [1] - 32 seconds 

          

Adaptive Weights [2] - 1221 seconds

          

Segment Support [3] - 2358 seconds

          

Segmentation Based [4] - 2 seconds

          

Reliability [5] - 803 seconds

          

Variable Windows [6] - 26 seconds

          




FBS vs AW: additional experimental results on the Middlebury 2006 Dataset [8,10]

In this section we report additional experimental results concerned with FBS and our implementation of AW [2] on the Middlebury 2006 Dataset [8,10]. For both algorithms we deployed the optimal parameters found  on the previous dataset (i.e. Tsukuba, Venus, Teddy, Cones). From left to right, we report: the reference image, the groundthtruth and the disparity maps computed by FBS [1] and AW [2].

 
 
 
  
 
     
 
      
 
     
 
    
 
      
 
      
 
     
 
    
 
      
 
    
 
       
 
      
 
  
 
  
 
         
 
       
 
  
 
    
 
    





FBS: additional experimental results on the .enpeda.. dataset

In this section we report additional experimental results concerned with the ..enpeda.. (Enviromental Perception and Driver Assistance) dataset  available at:
http://www.mi.auckland.ac.nz/index.php?option=com_content&view=article&id=44&Itemid=67 


ENPEDA "SuburbanFollow" sequence: Frame #0000





Left: FBS_39(3), 11 seconds* Right: FBS_45(5), 5 seconds*


ENPEDA "SuburbanFollow" sequence: Frame #0046





Left: FBS_39(3), 11 seconds* Right: FBS_45(5), 5 seconds*


ENPEDA "SuburbanFollow" sequence: Frame #0223





Left: FBS_39(3), 11 seconds* Right: FBS_45(5), 5 seconds*


* Image size 320x256, Intel Core 2 Quad @ 2.49GHz




If you are interested in stereo vision, you might find interesting this:

    Stefano Mattoccia
    "Stereo vision: algorithms and applications" []
    Extended version of the talk given at the University of Twente, April 1st 2009




References

[1] S. Mattoccia, S. Giardino, A. Gambini, "Accurate and efficient cost aggregation strategy for stereo correspondence based on approximated joint bilateral filtering", Asian Conference on Computer Vision (ACCV2009), September 23- 27, 2009,  Xi'an, China, [Abstract] [Pdf] [Bibtex]

[2] K. Yoon and I. Kweon,  "Adaptive support-weight approach for correspondence search",. IEEE Trans. PAMI, 28(4):650–656, 2006
.
[3] F. Tombari, S. Mattoccia, L. Di Stefano,“Segmentation-based adaptive support for accurate stereo correspondence", IEEE Pacific-Rim Symposium on Image and Video Technology  (PSIVT 2007), Lecture Notes in Computer Science 4872, pp 427-438, Springer 2007, December 17-19, 2007, Santiago, Chile

[4] M. Gerrits and P. Bekaert, "Local Stereo Matching with Segmentation-based Outlier Rejection", In Proc. Canadian Conf. on Computer and Robot Vision (CRV 2006), pp 66-66, 2006.

[5] S. Kang, R. Szeliski, and J. Chai, "Handling occlusions in dense multi-view stereo", In Proc. Conf. on Computer Vision and Pattern Recognition (CVPR 2001), pages 103–110, 2001.

[6] O. Veksler, "Fast variable window for stereo correspondence using integral images", In Proc. Conf. on Computer Vision and Pattern Recognition (CVPR 2003), pp 556–561, 2003.

[7] D. Scharstein and R. Szeliski, "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms",International Journal of Computer Vision, 47(1/2/3):7-42, April-June 2002 / Microsoft Research Technical Report MSR-TR-2001-81, November 2001

[8] D. Scharstein and R. Szeliski, Middlebury Stereo Vision Page, URL: "vision.middlebury.edu/stereo/"

[9] F. Tombari, S. Mattoccia, L. Di Stefano, E. Addimanda, “Classification and evaluation of cost aggregation methods for stereo correspondence",  IEEE International Conference on Computer Vision and Pattern Recognition (CVPR 2008), 2008.

[10] D. Scharstein and C. Pal, "Learning conditional random fields for stereo", In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2007), Minneapolis, MN, June 2007

[11] S. Mattoccia, "A locally global approach to stereo correspondence", IEEE Workshop on 3D Digital Imaging and Modeling  (3DIM2009), October 3-4, 2009, Kyoto, Japan [Abstract] [[Pdf] [Bibtex] [Additional experimental results]

[12] S. Mattoccia, M. Viti, F. Ries, "Near real-time Fast Bilateral Stereo on the GPU", 7th IEEE Workshop on Embedded Computer Vision (ECVW20011), CVPR Workshop, June 20, 2011, Colorado Springs (CO), USA, [PDF] [Bibtex]




 
Home
 
  

last update on: August 30, 2011