TY - GEN
T1 - Anatomy Completor
T2 - 2023 International Workshop on Shape in Medical Imaging
AU - Li, Jianning
AU - Pepe, Antonio
AU - Luijten, Gijs
AU - Schwarz-Gsaxner, Christina
AU - Kleesiek, Jens
AU - Egger, Jan
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.
PY - 2023
Y1 - 2023
N2 - In this paper, we introduce a completion framework to reconstruct the geometric shapes of various anatomies, including organs, vessels and muscles. Our work targets a scenario where one or multiple anatomies are missing in the imaging data due to surgical, pathological or traumatic factors, or simply because these anatomies are not covered by image acquisition. Automatic reconstruction of the missing anatomies benefits many applications, such as organ 3D bio-printing, whole-body segmentation, animation realism, paleoradiology and forensic imaging. We propose two paradigms based on a 3D denoising auto-encoder (DAE) to solve the anatomy reconstruction problem: (i) the DAE learns a many-to-one mapping between incomplete and complete instances; (ii) the DAE learns directly a one-to-one residual mapping between the incomplete instances and the target anatomies. We apply a loss aggregation scheme that enables the DAE to learn the many-to-one mapping more effectively and further enhances the learning of the residual mapping. On top of this, we extend the DAE to a multiclass completor by assigning a unique label to each anatomy involved. We evaluate our method using a CT dataset with whole-body segmentations. Results show that our method produces reasonable anatomy reconstructions given instances with different levels of incompleteness (i.e., one or multiple random anatomies are missing). Codes and pretrained models are publicly available at https://github.com/Jianningli/medshapenet-feedback/tree/main/anatomy-completor.
AB - In this paper, we introduce a completion framework to reconstruct the geometric shapes of various anatomies, including organs, vessels and muscles. Our work targets a scenario where one or multiple anatomies are missing in the imaging data due to surgical, pathological or traumatic factors, or simply because these anatomies are not covered by image acquisition. Automatic reconstruction of the missing anatomies benefits many applications, such as organ 3D bio-printing, whole-body segmentation, animation realism, paleoradiology and forensic imaging. We propose two paradigms based on a 3D denoising auto-encoder (DAE) to solve the anatomy reconstruction problem: (i) the DAE learns a many-to-one mapping between incomplete and complete instances; (ii) the DAE learns directly a one-to-one residual mapping between the incomplete instances and the target anatomies. We apply a loss aggregation scheme that enables the DAE to learn the many-to-one mapping more effectively and further enhances the learning of the residual mapping. On top of this, we extend the DAE to a multiclass completor by assigning a unique label to each anatomy involved. We evaluate our method using a CT dataset with whole-body segmentations. Results show that our method produces reasonable anatomy reconstructions given instances with different levels of incompleteness (i.e., one or multiple random anatomies are missing). Codes and pretrained models are publicly available at https://github.com/Jianningli/medshapenet-feedback/tree/main/anatomy-completor.
KW - Anatomical Shape Completion
KW - Diminished Reality
KW - MedShapeNet
KW - Residual Learning
KW - Shape Inpainting
KW - Shape Reconstruction
KW - Whole-body Segmentation
UR - http://www.scopus.com/inward/record.url?scp=85177423064&partnerID=8YFLogxK
U2 - 10.1007/978-3-031-46914-5_1
DO - 10.1007/978-3-031-46914-5_1
M3 - Conference paper
AN - SCOPUS:85177423064
SN - 9783031469138
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 1
EP - 14
BT - Shape in Medical Imaging
A2 - Wachinger, Christian
A2 - Paniagua, Beatriz
A2 - Elhabian, Shireen
A2 - Li, Jianning
A2 - Egger, Jan
PB - Springer Science and Business Media Deutschland GmbH
Y2 - 8 October 2023 through 8 October 2023
ER -