How VR can help patients understand cancer

A cutting edge virtual reality project is helping patients better understand cancer cells.

Health Agenda magazine
October 2018

Cancer researchers at the University of New South Wales (UNSW) have embarked on a project that combines virtual reality with advanced medical scanning technologies to take patients and doctors on an educational medical expedition.

The UNSW ‘Journey to the Centre of the Cell’ is in some ways reminiscent of the 1966 American sci-fi movie classic Fantastic Voyage, in which a submarine and crew shrunk to microscopic size were sent inside the body of a brilliant inventor to remove a potentially fatal blood clot, thereby saving the world.

Shrinking people and spaceships to the size of microbes remains a far-fetched fantasy, but in the UNSW laboratory researchers are taking innovation to the next level and making a microscopic human body a reality, virtually.

The life-saving path of virtual reality

Virtual reality (VR), found more commonly in entertainment and gaming, can now be used to visualise and interact with scientific data.

The UNSW Journey to the Centre of the Cell project started by isolating a single breast cancer cell, which was then mapped in 3D, at nanometre (one billionth of a metre) precision using a high-definition electron microscope.

The data from that scan was then imported into a 3D modelling software program to construct a blueprint of the cell. Then ‘renders’ were used to show the different structures (‘organelles’) within the cell. From there, the data was transferred into 3D gaming software to create a real-time, interactive, virtual-world representation of the cancer cell.

As you dive into the virtual world wearing a VR headset, headphones, and hand controllers, you land on the external cell surface. Shrunk to the tiny size of 40 one-billionths of a metre, you can walk across the surface of the cell.

Users can explore the cell’s surface and witness animated simulations of cellular activity, before leaving the dark surface for the colourful cell centre. Key structures are signposted with added information about structure and function.

The artist and the doctor

Associate Professor John McGhee leads the UNSW 3D Visualisation Aesthetics Lab where this technology is being developed.

The award-winning lab is part of UNSW’s Art and Design Faculty, and is a cross-disciplinary research hub exploring ways in which the arts can be used to help us visualise detailed scientific and biomedical data.

As well as allowing scientists to walk through the breast cancer cell, the VR innovation lets them observe the drugs transported in nanoparticles.

Assoc Prof McGhee says one of the current problems with breast cancer treatment is that medical scans are difficult for patients to understand.

“MRI scans, CT scans and X-rays are often black and white and are designed for medics; they’re not designed for patients,” he says. “I think designers have a role to play in the future of how we make sense of disease and illness.” 

It’s not just breast cancer patients who’ll be able to benefit. Professor Maria Kavallaris, director of the Australian Centre for NanoMedicine at UNSW, says that the technology could reduce anxiety and stress in children being treated for cancer, which in itself can improve their response to treatment.

“We could bring children along on the journey, showing them in ways they can understand what happens when doctors give them needles, why we are injecting drugs or doing procedures and what we are looking to find out. This could help them understand not just what’s going to happen, but also why.”

There are already instances of VR being used to explain MRIs or radiation therapy to children before they undergo treatment, says Prof Kavallaris.

Speaking the same visual language

Through this VR world, experts from different disciplines – who often focus on very different aspects of cancer treatment and human biology – are able to work on the same case and communicate through a shared immersive experience, says Prof Kavallaris.

Taking other specialists through the process of treatment can bring a level of understanding that sparks further innovation, she says.

“When a nanoparticle goes in and affects a particular organelle inside the cell, we might observe some disruption when we look at a 2D view through the microscope, but we won’t see the whole effect.

“By seeing what’s happening inside the cell, we can then modify the nanoparticles and materials. We’ll get a very strong impression of the impact of these changes on a particular organelle within the cell.”

Assoc Prof McGhee says combining data in this way may lead to better scientific discovery, and more accurate diagnosis and treatment. 

“Data from MRI scans, PET-CT scans, microscopy and even histology could be brought together to create a total explorable 3D picture of a patient all the way from the macro level right down to the microscopic.”