A Look into Medical Applications of AR/VR With Expert Professor Anderson Maciel

October 31, 2017

Professor Anderson Maciel, Doctor of Science, Federal University of Rio Grande do Sul (UFRGS) and IEEE Member

IEEE Transmitter: What is your area of AR/VR expertise?

Professor Maciel: Most of my experience is related to medical applications in VR and AR. This includes modeling and simulation of human bodies, surgery simulation, and visualization of specific patient anatomy both in VR and AR. I’ve worked on improving the feeling of presence in virtual environments, focusing especially on tactile and force feedback (the simulation of physical attributes, such as weight in VR.)

IEEE Transmitter: Why is AR/VR important? How is it benefitting society today, and how will it benefit society in the future?

Professor Maciel: VR and AR have improved learning and training techniques in several areas over the last two decades. Think about flight simulators for the training of pilots, both civil and military. Think about exploration of 3D anatomical atlases, about training and planning complex surgeries, such as laparoscopy. Some of these surgical techniques may take 10 years of training for a good professional; VR helps to reduce this learning time.

In marketing, sales and architecture, clients can visit new airplanes, buildings, etc. in VR far before the products are manufactured or built. Designers can assess prototypes in many contexts and environments before the final product starts production.

Rich entertainment environments are available using VR and AR, especially in games.

IEEE Transmitter: How, specifically, are you are working with AR/VR? What makes your work special?

Professor Maciel: We are working in a few key areas.

Visual realism: VR depends on fooling the human senses. As humans favor vision over the other senses, realistic computer graphics are necessary for VR applications such as surgery training and planning. We are currently working on a new class of appearance models to represent living tissue that is based on biophysics.
True x-ray vision: We are working on AR interfaces based on tablets and smartphones that work like a window to the interior of the body. These setups start from 3D image volumes from a person’s CT and MRI scans, and then align the volume with the real body. This gives the physician the ability to see inside the body using a tablet as a window to understand pathology, plan treatment, explain procedures to patients, and to guide a catheter, needle or endoscope to a specific region in the body, always maintaining the exterior of the body as a spatial reference.

Haptics: Besides the vision and auditory senses, the sense of touch is also very important in VR. It is especially important because the skin is our largest organ and because the brain circuits that process touch are very fast — 20 times faster than vision. We are working to make haptic actuation a rich and accurate means to fool the sense of touch of a user immersed in VR. At the same time, we are also working to provide haptic communication during real-world activities, where in addition to visual augmentations, haptic augmentations will help users to feel information in the surrounding AR environments. They will also communicate with other users by proactively articulating sentences based on tactile vocabularies, and with other objects and intelligent agents in an Internet of Things world.

IEEE Transmitter: How can AR/VR benefit the world’s next leaders in research and teaching, medicine, and exploration?

Professor Maciel: The same way all areas of the economy and research were affected by the appearance of the computer in the 80s, the Internet in the 90s and the mobile devices in the 2000s, VR and AR will affect them in the near future. AR will provide a natural interface to replace today’s smartphones and smartwatches. Consequently, it will spread to the masses as a must-have technology for communication, learning and working.

Telepresence systems, based on VR technology, will allow people to be virtually present in distant locations to interact with actual people and objects as if they were actually there. This includes telesurgery, where a human surgeon in a reference hospital of a great metropolis will guide a robot surgeon to operate on a patient located far from the urban areas, in the battlefield or in space.

The crucial difference between what is done today with computers and the Internet and what can potentially be done with VR and AR is to change the perspective of the human users from that of a spectator to that of a participant. In research, any new therapy, spatial design, psychological or marketing effect can be tested in controlled virtual environments by human subjects. In teaching, students will be participating in VR environments of trips to the Arctic and the deserts in the same day. They will learn history, living it in VR.

IEEE Transmitter: Where do you see AR/VR heading the next 5-10 years?

Professor Maciel: Most VR and AR concepts have been here for more than 30 years, but they never reached the masses due to technological limitations. As the smartphone boom enabled the cost of sensor and display devices to dramatically decrease in the last decade, cameras, batteries, inertial sensors and high-resolution, hand-sized displays became inexpensive enough to make VR technology widely available.

I believe the widespread use of VR and AR devices in ordinary homes in the next five years will bring new demands for the industry of VR and AR experiences. New applications, new experiences and new jobs will develop. In a short-term, people will be sharing, not pictures or videos, but immersive cinematic experiences through a new generation of social networks. Such experiences will eventually become live and interactive.

IEEE Transmitter: Please add anything else you deem important regarding AR/VR technology.

Professor Maciel: The VR and AR industry today is still greatly sustained by entertainment. This is more a cultural feature than a technological limitation. New generations will be more open to using VR in real jobs because they are comfortable with the technology, just like it is possible for the current generation to make lawyer consultations, doctor appointments and business in general through Skype, while the previous generation needed a meeting in person.

In terms of technology, we will soon have computational displays* in comfortable form factors to turn your cumbersome VR or AR glasses into a designer frame fashion object with VR and AR capabilities. Later, they will be offered also in a format of contact lenses with integrated sensors.

* displays that use an array of microlenses to redirect light to specific parts of your retina. Algorithmically modified images in an LED lattice will then be transformed into a light field that can be perceived by the user as a sharp 3D display always in focus.