Multi-view stereo of an object immersed in a refractive medium

Robin Bruneau (1, 2) , Baptiste Brument (1, 3) , Lilian Calvet (4) , Matthew Cassidy (1, 3) , Jean Mélou (1, 3) , Yvain Quéau (5) , Jean-Denis Durou (1) , François Lauze (6)

1 IRIT-REVA - Real Expression Artificial Life
2 DIKU - Department of Computer Science [Copenhagen]
3 Toulouse INP - Institut National Polytechnique (Toulouse)
4 Balgrist University Hospital
5 Equipe Image - Laboratoire GREYC - UMR6072
6 UCPH - University of Copenhagen = Københavns Universitet

In this article, we show how to extend the multi-view stereo (MVS) technique when the object to be reconstructed is inside a transparent – but refractive – medium, which causes distortions in the images. We provide a theoretical formulation of the problem accounting for a general, non-planar shape of the refractive interface, and then a discrete solving method. We also present a pipeline to recover precisely the geometry of the refractive interface, considered as a convex polyhedral object. It is based on the extraction of visible polyhedron vertices from silhouette images and matching across a sequence of images acquired under circular camera motion. These contributions are validated by tests on synthetic and real data.

Mots clés: 3D reconstruction; Multi-view stereo; Refraction

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]

Robin Bruneau, Baptiste Brument, Lilian Calvet, Matthew Cassidy, Jean Mélou, et al.. Multi-view stereo of an object immersed in a refractive medium. Journal of Electronic Imaging, 2024, 33 (03), ⟨10.1117/1.JEI.33.3.033005⟩. ⟨hal-04567615⟩