Some people, particularly the elderly, suffer from dizziness and loss of balance, which affects their mobility, independence and self-confidence and can burden the healthcare system when falls and other events require hospitalisation. According to research, around a third of people over 65 who live at home fall annually.
In a number of studies, customised balance rehabilitation programmes were found to be significantly more effective in resolving symptoms than generic exercises.
However, there is a shortage of physiotherapists and specialists to assist patients. “So we created a virtual coach based on augmented reality, a hologram who gives personalised instructions to the person doing the exercises and helps motivate them to continue and improve,” says HOLOBALANCE project coordinator Dimitrios Fotiadis, professor of Biomedical Engineering at the University of Ioannina, Greece, where he is head of the Unit of Medical Technology and Intelligent Information Systems.
A suitcase-sized HOLOBALANCE system was developed during the 4-year project involving 13 consortium partners from 7 countries. The main technological challenge was to ensure the hologram performs the exercises with 100 % accuracy for the patient to follow, according to Fotiadis.
The patient is monitored and supervised remotely using data from wearable motion capture sensors – including an activity tracker, head and back IMU sensors which monitor direction and speed of movement, and sensorised insoles.
“We collect signals from the person’s body to understand what they are doing and we created a scoring system to evaluate their performance. The hologram can understand if the exercise is performed correctly or not and give instructions on how to exercise better.”
Personalised programme
The system uses machine learning software to personalise the programme in real time, for example, if the patient needs to go faster or slower. “Using the scoring method, our system understands what the person is doing, with an accuracy of 80-90 %, which is very high,” he notes.
“We can also extract other factors from the data like reaction time, tiredness, stress or frustration so the system learns and the hologram coach can come back to the patient with new instructions – it is a progressive learning system.”
The system also monitors the patient’s motivation and their satisfaction with the training using emotional computing and behavioural modelling.
Testing the system
A version for clinics and hospitals produces a full-size hologram coach on the Holobox. The portable version allows people to use the virtual coach on a mobile phone. “Once this is mounted in an augmented reality headset, it feels like a full-sized person,” according to Fotiadis.
The system was tested on 80 participants in Germany, Greece and the United Kingdom using the exercise programme for 8 weeks for around 20 minutes a day. Its efficiency was compared with a group of 80 patients who did not use the system. Around 88 % of the trial participants showed clinically meaningful improvement in gait and 82 % in balance.
Impact on daily activities was measured using questionnaires asking whether patients were happy to go out, managed to stay independent and could adopt a more active, healthy lifestyle.
“We reached a very high level of satisfaction and efficiency. Patients enjoy it and they understand that they gain from the system. Fear of falling decreased,” Fotiadis notes.
He adds: “We achieved an adherence rate of 83 % which is very high when you consider these were mostly elders and some of them don’t know about technology or smartphones.”
Keywords
HOLOBALANCE, hologram, balance, falls, dizziness, exercise, sensors, physiotherapist, rehabilitation, machine learning