research-article

Stereo fusion

Authors:

Seung-Hwan Baek,

Min H. KimAuthors Info & Claims

Computer Vision and Image Understanding, Volume 146, Issue C

Pages 52 - 66

https://doi.org/10.1016/j.cviu.2016.02.006

Published: 01 May 2016 Publication History

Abstract

A stereo fusion system that combines binocular and refractive stereo is presented.Our stereo fusion outperforms traditional binocular and refractive stereo.An efficient calibration method for refractive stereo is proposed. Display Omitted The performance of depth reconstruction in binocular stereo relies on how adequate the predefined baseline for a target scene is. Wide-baseline stereo is capable of discriminating depth better than the narrow-baseline stereo, but it often suffers from spatial artifacts. Narrow-baseline stereo can provide a more elaborate depth map with fewer artifacts, while its depth resolution tends to be biased or coarse due to the short disparity. In this paper, we propose a novel optical design of heterogeneous stereo fusion on a binocular imaging system with a refractive medium, where the binocular stereo part operates as wide-baseline stereo, and the refractive stereo module works as narrow-baseline stereo. We then introduce a stereo fusion workflow that combines the refractive and binocular stereo algorithms to estimate fine depth information through this fusion design. In addition, we propose an efficient calibration method for refractive stereo. The quantitative and qualitative results validate the performance of our stereo fusion system in measuring depth in comparison with homogeneous stereo approaches.

References

[1]

D. Scharstein, R. Szeliski, A taxonomy and evaluation of dense two-frame stereo correspondence algorithms, Int. J. Comput. Vis., 47 (2002) 7-42.

Digital Library

[2]

K. Kawashima, S. Kanai, H. Date, As-built modeling of piping system from terrestrial laser-scanned point clouds using normal-based region growing, J. Comput. Des. Eng., 1 (2014) 13-26.

[3]

A. Kadambi, R. Whyte, A. Bhandari, L.V. Streeter, C. Barsi, A.A. Dorrington, R. Raskar, Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles, ACM Trans. Graph., 32 (2013) 167.

Digital Library

[4]

A. Levin, R. Fergus, F. Durand, W.T. Freeman, Image and depth from a conventional camera with a coded aperture, ACM Trans. Graph., 26 (2007) 70:1-9.

Digital Library

[5]

C. Rhemann, A. Hosni, M. Bleyer, C. Rother, M. Gelautz, Fast cost-volume filtering for visual correspondence and beyond, IEEE, 2011.

[6]

M. Okutomi, T. Kanade, A multiple-baseline stereo, IEEE Trans. Pattern Anal. Mach. Intell., 15 (1993) 353-363.

Digital Library

[7]

F. Zilly, C. Riechert, M. Mller, P. Eisert, T. Sikora, P. Kauff, Real-time generation of multi-view video plus depth content using mixed narrow and wide baseline, J. Vis. Commun. Image Represent., 25 (2013) 632-648.

Digital Library

[8]

C. Gao, N. Ahuja, Single camera stereo using planar parallel plate, 2004.

[9]

C. Gao, N. Ahuja, A refractive camera for acquiring stereo and super-resolution images, 2006.

[10]

Z. Chen, K.-Y.K. Wong, Y. Matsushita, X. Zhu, Depth from refraction using a transparent medium with unknown pose and refractive index, Int. J. Comput. Vis., 102 (2013) 3-17.

Digital Library

[11]

Y. Nakabo, T. Mukai, Y. Hattori, Y. Takeuchi, N. Ohnishi, Variable baseline stereo tracking vision system using high-speed linear slider, 2005.

[12]

S.-H. Baek, M.H. Kim, Stereo fusion using a refractive medium on a binocular base, Springer, Singapore, Singapore, 2015.

[13]

Y. Furukawa, J. Ponce, Accurate, dense, and robust multiview stereopsis, IEEE Trans. Pattern Anal. Mach. Intell., 32 (2010) 1362-1376.

Digital Library

[14]

D. Gallup, J.-M. Frahm, P. Mordohai, M. Pollefeys, Variable baseline/resolution stereo, 2008.

[15]

S.M. Seitz, B. Curless, J. Diebel, D. Scharstein, R. Szeliski, A comparison and evaluation of multi-view stereo reconstruction algorithms, 2006.

[16]

E. Hecht, Addison-Wesley, Reading, Mass, 1987.

[17]

Y. Nishimoto, Y. Shirai, A feature-based stereo model using small disparities, 1987.

[18]

D. Lee, I. Kweon, A novel stereo camera system by a biprism, IEEE Trans. Robot. Autom., 16 (2000) 528-541.

[19]

M. Shimizu, M. Okutomi, Reflection stereo-novel monocular stereo using a transparent plate, IEEE, 2006.

[20]

M. Shimizu, M. Okutomi, Monocular range estimation through a double-sided half-mirror plate, IEEE, 2007.

[21]

Z. Chen, K. Wong, Y. Matsushita, X. Zhu, M. Liu, Self-calibrating depth from refraction, 2011.

[22]

D.G. Lowe, Distinctive image features from scale-invariant keypoints, Int. J. Comput. Vis., 60 (2004) 91-110.

Digital Library

[23]

C. Liu, J. Yuen, A. Torralba, Sift flow: dense correspondence across scenes and its applications, IEEE Trans. Pattern Anal. Mach. Intell., 33 (2011) 978-994.

Digital Library

[24]

Z. Zhang, A flexible new technique for camera calibration, IEEE Trans. Pattern Anal. Mach. Intell., 22 (2000) 1330-1334.

Digital Library

[25]

S.J. Gortler, MIT Press, London, 2012.

[26]

R.A. Waltz, J.L. Morales, J. Nocedal, D. Orban, An interior algorithm for nonlinear optimization that combines line search and trust region steps, Math. Program., 107 (2006) 391-408.

Digital Library

[27]

M.H. Kim, J. Kautz, Characterization for high dynamic range imaging, Comput. Graph. Forum (Proc. EUROGRAPHICS 2008), 27 (2008) 691-697.

[28]

R.T. Collins, A space-sweep approach to true multi-image matching, IEEE, 1996.

[29]

C. Kim, H. Zimmer, Y. Pritch, A. Sorkine-Hornung, M. Gross, Scene reconstruction from high spatio-angular resolution light fields, ACM Trans. Graph., 32 (2013) 73:1-12.

Digital Library

[30]

R.O. Duda, P.E. Hart, Wiley, 1973.

[31]

D. Comaniciu, P. Meer, Mean shift: a robust approach toward feature space analysis, IEEE Trans. Pattern Anal. Mach. Intell., 24 (2002) 603-619.

Digital Library

[32]

K. He, J. Sun, X. Tang, Guided image filtering, Springer, 2010.

[33]

S. Mattoccia, S. Giardino, A. Gambini, Accurate and efficient cost aggregation strategy for stereo correspondence based on approximated joint bilateral filtering, Springer, 2010.

[34]

K.-J. Yoon, I.S. Kweon, Adaptive support-weight approach for correspondence search, IEEE Trans. Pattern Anal. Mach. Intell., 28 (2006) 650-656.

Digital Library

[35]

Z. Ma, K. He, Y. Wei, J. Sun, E. Wu, Constant time weighted median filtering for stereo matching and beyond, 2013.

[36]

S.T. Barnard, Stochastic stereo matching over scale, Int. J. Comput. Vis., 3 (1989) 17-32.

[37]

J.-S. Chen, G. Medioni, Parallel multiscale stereo matching using adaptive smoothing, 1990.

[38]

Y. Boykov, O. Veksler, R. Zabih, Fast approximate energy minimization via graph cuts, IEEE Trans. Pattern Anal. Mach. Intell., 23 (2001) 1222-1239.

Digital Library

[39]

M. Bleyer, C. Rhemann, C. Rother, Patchmatch stereo-stereo matching with slanted support windows., 2011.

[40]

G. Nam, M.H. Kim, Multispectral photometric stereo for acquiring high-fidelity surface normals, IEEE Comput. Graph. Appl., 34 (2014) 57-68.

[41]

H. Lee, M.H. Kim, Building a two-way hyperspectral imaging system with liquid crystal tunable filters, Springer, Normandy, France, 2014.

[42]

M.H. Kim, T.A. Harvey, D.S. Kittle, H. Rushmeier, J. Dorsey, R.O. Prum, D.J. Brady, 3D imaging spectroscopy for measuring hyperspectral patterns on solid objects, ACM Trans. Graph. (Proc. SIGGRAPH 2014), 31 (2012) 38:1-11.

Digital Library

[43]

M.H. Kim, H. Rushmeier, J. ffrench, I. Passeri, D. Tidmarsh, Hyper3d: 3d graphics software for examining cultural artifacts, ACM J. Comput. Cult. Herit., 7 (2014) 1:1-19.

Digital Library

Cited By

Zhao GYang RJing XZhang HWu ZSun XJiang HLi RWei XFountas SZhang HFu L(2023)Phenotyping of individual apple tree in modern orchard with novel smartphone-based heterogeneous binocular vision and YOLOv5sComputers and Electronics in Agriculture10.1016/j.compag.2023.107814209:COnline publication date: 5-Jun-2023
https://dl.acm.org/doi/10.1016/j.compag.2023.107814
Kuznetsov AGorevoy AMachikhin A(2019)Image rectification for prism-based stereoscopic optical systemsComputer Vision and Image Understanding10.1016/j.cviu.2019.02.003182:C(30-37)Online publication date: 1-May-2019
https://dl.acm.org/doi/10.1016/j.cviu.2019.02.003
Baek SGutierrez DKim M(2016)Birefractive stereo imaging for single-shot depth acquisitionACM Transactions on Graphics10.1145/2980179.298022135:6(1-11)Online publication date: 5-Dec-2016
https://dl.acm.org/doi/10.1145/2980179.2980221

Stereo fusion
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
      2. Image and video acquisition
  2. Computer graphics

Recommendations

Robustness of ToF and stereo fusion for high‐accuracy depth map

A depth map can be used in many applications such as robotic navigation, driverless, video production and 3D reconstruction. Both passive stereo and time‐of‐flight (ToF) cameras can provide the depth map for the captured real scenes, but they both have ...
Accurate Camera Calibration from Multi-View Stereo and Bundle Adjustment

The advent of high-resolution digital cameras and sophisticated multi-view stereo algorithms offers the promise of unprecedented geometric fidelity in image-based modeling tasks, but it also puts unprecedented demands on camera calibration to fulfill ...
Auto-calibrating photometric stereo using ring light constraints

We propose an auto-calibration method for photometric stereo. Our method exploits constraints placed on light sources to recover their positions and the surface normals of an object up to a scaling and planar rotation ambiguity. The ambiguity is ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Computer Vision and Image Understanding

Computer Vision and Image Understanding Volume 146, Issue C

May 2016

99 pages

ISSN:1077-3142

Issue’s Table of Contents

Copyright © Elsevier Inc.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 May 2016

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhao GYang RJing XZhang HWu ZSun XJiang HLi RWei XFountas SZhang HFu L(2023)Phenotyping of individual apple tree in modern orchard with novel smartphone-based heterogeneous binocular vision and YOLOv5sComputers and Electronics in Agriculture10.1016/j.compag.2023.107814209:COnline publication date: 5-Jun-2023
https://dl.acm.org/doi/10.1016/j.compag.2023.107814
Kuznetsov AGorevoy AMachikhin A(2019)Image rectification for prism-based stereoscopic optical systemsComputer Vision and Image Understanding10.1016/j.cviu.2019.02.003182:C(30-37)Online publication date: 1-May-2019
https://dl.acm.org/doi/10.1016/j.cviu.2019.02.003
Baek SGutierrez DKim M(2016)Birefractive stereo imaging for single-shot depth acquisitionACM Transactions on Graphics10.1145/2980179.298022135:6(1-11)Online publication date: 5-Dec-2016
https://dl.acm.org/doi/10.1145/2980179.2980221

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents