In the heart of Rome, at the International Academy of Music and Art, an alternative kind of music lesson is underway.
Researchers are trying to create music in real time through electric brain signals generated by the performer. The hope is that sometime in the near future a kind of shortcut will allow the composer to “think” directly the music onto the page, without having to write it down.
Pianist Francesco Prode is wearing a neural sensor on his head, which picks up his brainwaves and sends them to a laptop. In front of him is a Kinect motion sensor of the kind used in video games. As he passes his hand in front of the device, just like an orchestra director, the pianist activates the reproduction of a fragment of piano sound previously recorded by a computer.
“We can intervene in real time on the sound material that I have played live. It is recorded in real time and then I can recall it by passing my hand in front of a sensor, so that at each step I can create a different polyphony and a different outcome to the composition,” says Francesco Prode.
However, the music recalled by the pianist is not just a simple reproduction of what he played before. It is different because it has been elaborated by special software according to the pianist’s brain activity, which varies according to his emotions.
The software analyses his level of attention, his consciousness and other brain activity. The musical notes themselves are unchanged, but the velocity of the sound reproduced, its intonation and its texture vary according to the pianist’s mood. This creates a a continually-evolving polyphony.
“This sensor can detect any kind of brain activity depending on how awake or how alert the player is,” explains composer Riccardo Santoboni. “The level of activity of the brain reveals a performer or a composer’s level of attention, whether he’s calm or relaxed. On the computer you can see the neural activity of Francesco Prode right now, as well as a range of software that has been created to produce or process sounds generated by the piano, that have somehow been modified through the pianist’s emotions.”
Researchers are now looking to the future, with the aim of not only modifying existing music through emotions, but creating brand new scores.
They are working on software that would be able to associate different musical instruments, musical textures and durations to specific brain waves, creating an entirely new melody.
The creative intervention of the composer would still play an important role, however, as it would set a number of parameters like associating a particular mood with a particular musical instrument, for example.
“Our goal is very ambitious. In fact, today, we have the means of reaching this goal. It subverts the idea of a composer or a performer related to a medium, a mechanical artifact. It would allow us to substantially get rid of mediation tools and move on to creating music simply by thought,” says Riccardo Santoboni.
Similar research is underway in other places like Plymouth University in the UK, where researchers are investigating techniques to generate music with a Brain-Computer Music Interface aimed at inducing specific emotions on listeners.