Simulation of Hand Anatomy Using Medical Imaging
ACM SIGGRAPH Asia 2022

People

• Mianlun Zheng*
  University of Southern California
• Bohan Wang*
  University of Southern California, Massachusetts Institute of Technology
• Jingtao Huang
  University of Southern California
• Jernej Barbič
  University of Southern California

*=joint first authors

Project material

• Paper (PDF, 45 MB)
• Video (Quicktime MP4, 149 MB)
• Short Video (Quicktime MP4, 12 MB)
• Supplementary PDF document (PDF, 0.2 MB)
• Supplementary material (meshes) (ZIP, 69 MB)
• 5-min presentation (video with audio) (Quicktime MP4, 57 MB)
• 15-min presentation (video with audio) (Quicktime MP4, 398 MB)
• MRI dataset

Citation

• Mianlun Zheng, Bohan Wang, Jingtao Huang, Jernej Barbič:
  Simulation of Hand Anatomy Using Medical Imaging, ACM Transactions on Graphics 41(6) (SIGGRAPH Asia 2022), Dec 2022. BIBTEX

Abstract

Precision modeling of the hand internal musculoskeletal anatomy has been largely limited to individual poses, and has not been connected into continuous volumetric motion of the hand anatomy actuating across the hand's entire range of motion. This is for a good reason, as hand anatomy and its motion are extremely complex and cannot be predicted merely from the anatomy in a single pose. We give a method to simulate the volumetric shape of hand's musculoskeletal organs to any pose in the hand's range of motion, producing external hand shapes and internal organ shapes that match ground truth optical scans and medical images (MRI) in multiple scanned poses. We achieve this by combining MRI images in multiple hand poses with FEM multibody nonlinear elastoplastic simulation. Our system models bones, muscles, tendons, joint ligaments and fat as separate volumetric organs that mechanically interact through contact and attachments, and whose shape matches medical images (MRI) in the MRI-scanned hand poses. The match to MRI is achieved by incorporating pose-space deformation and plastic strains into the simulation. We show how to do this in a non-intrusive manner that still retains all the simulation benefits, namely the ability to prescribe realistic material properties, generalize to arbitrary poses, preserve volume and obey contacts and attachments. We use our method to produce volumetric renders of the internal anatomy of the human hand in motion, and to compute and render highly realistic hand surface shapes. We evaluate our method by comparing it to optical scans, and demonstrate that we qualitatively and quantitatively substantially decrease the error compared to previous work. We test our method on five complex hand sequences, generated either using keyframe animation or performance animation using modern hand tracking techniques.

Comments, questions to Jernej Barbič.

Related projects

• Modeling of Personalized Anatomy using Plastic Strains
• Hand Modeling and Simulation Using Stabilized Magnetic Resonance Imaging

Acknowledgments

• NSF (IIS-1911224)
• USC Annerberg Graduate Fellowship to Mianlun Zheng and Bohan Wang
• Bosch Research
• Adobe Research

Disclaimer

Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

Copyright notice

The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Unique accesses: