University News Last updated 27 May 2016
A groundbreaking project which uses pioneering technology to transform individual hand movements into a musical and visual experience will be unveiled in Berlin this weekend (Sunday 29 May).
The interactive system, created by research student Balandino Di Donato of Birmingham Conservatoire, enables the use of motion to introduce a range of sounds and visual changes into a performance.
The technology maps each movement of the hand or arm using the wearable technology Thalmic Myo armband, alongside software developed at the Conservatoire’s Integra Lab.
Motion is then used to drive sound spatialisation and filtering, visual elaborations through drawing on a screen and processing videos, and to alter lights with the strength of each movement dictating their brightness.
The system will be showcased by Balandino in collaboration with award-winning beatboxer Grace Savage at Music Tech Fest Berlin on Sunday.
The technology will be demonstrated in front of hundreds of musicians, music fans and technology experts who are flocking to the German capital this weekend.
Balandino said: “The system was born to drive sound spatialisation through beatboxers’ already learnt gestures, but then working on Music Gesture Beatbox with Grace brought additional visual elements including control over video and lighting.
Grace has used the technology to create her own unique performance, which allows her to easily change the sound, visual and light design, as well as immerse herself and the audience into the musical experience.
The collaboration between the pair will be the world’s first public performance of the new system.
Balandino came up with the project as part of his PhD research, which is focused on live sound and the influence of space and movement.