Deep Learning for 3D reconstruction and Simulation of Aesthetic Procedures

Type Start End
National Jul 2017 Jun 2020
Responsible URL
Xavier Giró-i-Nieto Industrial Doctorates

Reference

2017-DI-028

Description

  • Industrial doctorate jointly offered with Crisalix.

The aim of this project is to increase the accuracy and realism of 3D reconstruction and 3D aesthetic procedure simulations using novel deep learning techniques.

Deep learning has flooded several technical fields which years ago were approached by totally different algorithms. The availability of huge amounts of data, the improvements in parallel processing hardware and the open sourcing of frameworks to train deep neural models have been key factors to train these models successfully in reasonable amounts of time, reaching state-of-the-art results in many subfields of natural language processing and image processing. These techniques are not limited to 1D or 2D; they are also suitable for 3D. As shown in recent publications, different deep neural network architectures combined with multi-view geometry theory can be employed to solve problems like 3D reconstruction, 3D recognition and 3D shape alignment. Moreover, a subbranch called generative models is obtaining very promising results using the novel Generative Adversarial Networks (GANs and its variants), especially in image synthesis.

In this project, we will explore these novel deep learning techniques and how to combine them with multi-view geometry fundamentals to create new pipelines that increase the accuracy and the realism of existing methods for 3D reconstruction of body parts and its simulation in both geometric and texture spaces

Publications

Ramon E. Few-shot 3D Reconstruction of Body Parts with Deep Neural Networks Moreno F, Garcia J, Giró-i-Nieto X. Signal Theory and Communications. 2022 ;Excellent.
Ramon E, Triginer G, Escur J, Pumarola A, Garcia J, Giró-i-Nieto X, Moreno F. H3D-Net: Few-Shot High-Fidelity 3D Head Reconstruction. In: International Conference on Computer Vision (ICCV). International Conference on Computer Vision (ICCV). Virtual: IEEE/CVF; 2021. (6.08 MB)
Escur J. Attention-based multi-view 3D reconstruction models Ramon E, Giró-i-Nieto X. 2020 .
Ramon E, Escur J, Giró-i-Nieto X. Multi-View 3D Face Reconstruction in the Wild using Siamese Networks. In: ICCV 2019 Workshop on 3D Face Alignment in the Wild Challenge Workshop (3DFAW). ICCV 2019 Workshop on 3D Face Alignment in the Wild Challenge Workshop (3DFAW). Seoul, South Corea: IEEE/CVF; 2019. (227.12 KB)
Ramon E, Ruiz G, Batard T, Giró-i-Nieto X. Hyperparameter-Free Losses for Model-Based Monocular Reconstruction. In: ICCV 2019 Workshop on Geometry Meets Deep Learning. ICCV 2019 Workshop on Geometry Meets Deep Learning. Seoul, South Corea: IEEE / Computer Vision Foundation; 2019. (749.7 KB)
Ramon E, Villar J, Ruiz G, Batard T, Giró-i-Nieto X. Plug-and-Train Loss for Model-Based Single View 3D Reconstruction. BMVA Technical Meeting: 3D vision with Deep Learning. 2019 . (3.97 MB)
Ramon E. Deep Learning algorithms for 3D Reconstruction and Simulation of Aesthetic Procedures Giró-i-Nieto X. 2018 .

Collaborators