Transforming Movements to Music
Researcher: Balandino Di Donato
PhD Conservatoire student, Balandino Di Donato, has collaborated with developers to create an interactive gadget in the form of an armband named Myo. The armband, developed using the Conservatoire’s Integra Lab, transforms hand and arm movements into a musical and visual show. Balandino created this technology as part of his PhD research on live sound and the influence of space and movement.
A combination of HCI (Human Computer Interaction), gesture recognition, machine learning, mixed reality and interaction design is brought together to bring this project to life.
This project aims to:
- Transform the gestures of beatboxers into vocal sounds
- Track realtime data from the ‘Myo’ armband
- Control sound simply using movements
- Bring together findings from experiments to finetune Myo prototype
- Give performers the ability to physically manipulate sound and visuals on stage
Through the development of the Myo Balandino established two hand poses, grabbing (a fist) and throwing (an outstretched hand), that will trigger the sounds. Balandino’s reasoning for this is that the technology will be easy for users to learn and it improves the ability to track movements.
The gadget was put into action on the 29 May by beatboxing champion, Grace Savage at music tech festival, MTFBerlin. This ground-breaking performance allowed Grace to sample, loop and process her voice live on stage, controlling visuals and lighting which immersed the performer and audience.
Find out more on Balandino's research website.