Augmented Reality and Virtual Reality interaction and quality of experience for industry and healthcare

Academics from the School of Computing and Digital Technology have been researching new tools, methods, and approaches for providing improved Augmented and Virtual reality (AR/VR) applications. These tools and processes have been used across a variety of industries to improve product knowledge, user experience, virtual environment interaction and procedure training.

Mixed Reality and Human Computer Interaction group large

Research summary

Their research into improved AR/VR has focused on addressing the challenges to creating more realistic interactions and more valuable experiences to make AR/VR technology easier and more natural to use in real-world settings and offer valuable end user experiences.

It’s this focus that has seen the team work on impactful projects with leading industry, notably the global pharmaceutical company GlaxoSmithKline (GSK) and with healthcare practitioners in the NHS. This has resulted in AR/VR systems that can be utilised to improve industry use of AR technology and improve practitioner training.

Work led by Dr Ian Williams and Dr Maite Frutos of the Digital Media Technology Lab has developed a novel suite of visualisation tools for GSK, that allowed their consumer healthcare division to virtually prototype products, packaging, store layouts and customer experiences in real time. These tools, and associated training, has given GSK the means to better evaluate their products and services before taking them to market and led to sector growth in the use of AR/VR technology. Dr Ian Williams and Dr Maite Frutos Pascual’s work highlights the potential for AR/VR to be used in real world environments to trial concepts and initial ideas digitally allowing potential users to experience and realistically interact with new products and services digitally before going to market.

Research led by Dr Andrew Wilson, has resulted in the development of a VR ophthalmology training tool which allows medical students to recognise abnormalities in the eye before diagnosing real patients, thus increasing their experience and confidence. Ophthalmology is an important diagnostic skill whose use in clinical practice has been declining since 2013. Junior doctors now feel far less confident in their ability to perform this procedure on patients and traditional learning methods rely on senior staff taking the time to guide students through these skills. Obstacles often arise due to time constraints in busy clinics, leading to scenarios where staff are not always available for training. Adoption of VR tools allow students to become more familiar with the procedure while reducing teaching time.

Research outcomes

Outcomes from the research have been multi-faceted with benefits and impact from increased product sales, product trials and training experience. GSK has, as part of their Shopper Science Lab, used Dr Williams and Dr Frutos’ work to create growth in the number of their AR/VR projects undertaken from 2016 to 2019 resulting in increased businesses value. Additionally, the research has obtained global reach and influenced projects over a large geographic area, notably across the five market regions of Europe, Middle East, Africa, America, and Asia-Pacific. 

In addition, the team’s research also enabled GSK to develop the AR Playbook, an AR/VR document detailing results, tools, and applications for consumer healthcare. The playbook has been used across GSK to communicate the potential of AR/VR technology and guide stakeholders as they execute new and exciting immersive technology projects. This is impacting company policy leading to increased business education, awareness and traction for AR and VR technology throughout divisions GSK worldwide.

From 2015 to 2020 there have been over 1100 downloads of the VR ophthalmology application resulting from Dr Wilsons’ research. Wide reaching and international positive feedback have been gained from ophthalmologists and physicians in the UK, USA, Singapore, France, and Poland. Additionally, in the NHS, the tool has been embedded into training and can be accessed by the approximate 150 Medical students per year Sandwell and West Birmingham Hospitals NHS Trust.

REF 2021

Birmingham City University's hub for everything related to the Research Excellence Framework.

REF 2021

Dr Ian Williams

Associate Professor and Head of the Digital Media Technology Lab (DMT Lab)

Dr Williams is an Associate Professor and head of the Digital Media Technology Lab (DMT Lab). He leads the research strategy and vision for the DMT Lab which encompasses the four core domains of: Human Computer Interaction (HCI), Computer Graphics and Computer Vision, Sound and Music Analysis and Broadcast Systems. He gained his PhD from Manchester Metropolitan University in 2008 in low level feature analysis and Ai for multiple-scale biomedical image analysis. His current research interests centre on the visual and interactive computing domains with a key emphasis on creating novel methods, theories, and paradigms for interacting with, and improving on, the Quality of Experience for users of Augmented Reality (AR) and Virtual Reality (VR).

Having an industry and academic background, Dr Williams' research has many interfaces with industry and delivers impacting applications which shape company policy, applications, and processes. He is lead academic on many Knowledge Transfer Partnership (KTP) programmes, with national and international companies. These include GlaxoSmithKline, SBS Insurance and Sherlock Healthcare.

Dr Maite Frutos-Pascual

Dr Maite Frutos-Pascual

Lecturer in Digital Media Technology

Dr Maite Frutos-Pascual is a senior lecturer and active researcher at DMT Lab. She specialises in Human Computer Interaction (HCI), immersive technologies (Augmented Reality and Virtual Reality AR/VR), usability, user analysis, interactive systems and sensor data analysis and integration. She oversees research and industry contributions within the field of immersive technologies at DMT Lab.

Maite earned her PhD in Computer Science in 2016 with “Multi Variable Analysis of Gaze and Interaction for Real Time Player Profiling in Cognitive Intelligent Therapies based on Serious Games” at the University of Deusto, Spain where she was awarded the Ignacio Ellacuria Award to outstanding PhD contributions.

Dr Andrew Wilson

Associate Professor

Andrew is an Associate Professor in the Department of Computing and Data Science,School of Computing and Digital Technology. He has developed and led many undergraduate and post graduate courses.

As well as teaching he is an active researcher in the field of digital health,  working across faculties, NHS Trusts and European collaborators.