Tech Talk: Virtual Reality Healthcare Training at University of Maryland
DC News Now reporter Tosin Fakile testing out the Holocamera
Published in DC News Now | May 7, 2025
The University of Maryland, College Park campus is using a 3D virtual human avatar to diagnose patients in a new medical training virtual reality scenario.
WATCH VIDEO HERE.
“About five years ago, we had this vision to create cameras that could create 3D, virtual, avatars of humans that were cinematically real, very high fidelity,” Professor Amitabh Varshney said.
That resulted in the creation of the Holocamera; Cinematic Avatar Imaging Studio.
“Instead of a camera person, for instance, in a movie, the director directs how the user will experience a scene. We are now able to allow users to directly figure out how they would like to immerse and see, experience a particular environment,” Varshney said.
That full immersion happens at the volumetric capture facility at UMD’s College of Computer, Mathematical and Natural Sciences.
“It has 300 cameras. Each camera is capturing information at 4k resolution, 300 frames a second. So overall, we are getting 100 billion samples per second in this facility,” Varshney said.
It’s designed for high-precision learning and simulation.
Varshny said a few years ago, they carried out a study that showed users can recall things much better if they experience it in an embodied fashion in a VR environment than on a desktop.
“If you look at the practice of surgery as an example, you have situations where no more than a small handful of residents, or interns, or healthcare professionals, students can participate and see the surgery,” Varshney said. “We would be able to allow students to experience and stand in the shoes of the surgeon, add their own. Well, further, it is scalable.”
It took a few years and a few versions to get to this point, and there’s more still planned for the Holocamera.
“One of the other things that we are using this facility for is to understand how artificial intelligence can be used to generate. So, we will capture one set of actions in this facility, and can we then use generative AI to generate other actions and other modalities from this facility? So that we can try a lot of what-if scenarios, in an interactive manner,” Varshney said. “How can we use this as a tool to train the next generation of computer scientists, artificial intelligence scientists and engineers, and technologists in the underlying technology for this?”
