Apr 16, 2020
Have you ever wondered what your brain signals sound like when you think? We developed an apparatus to translate your brain signals to music.
Music is the wallpaper of our minds – Jamie Wheals
We believe music has the capacity to communicate the ineffable, or inexpressible. And humans have this urge to language these inexpressible thoughts when the words don’t work. And these thoughts are instrumented into existence by artists through poems, visuals and music. Our innovations of doing so have been through instruments, mediated through our body (hands, mouth, feet) but WHAT IF WE COULD ACTUALIZE THE BRAIN IN CREATING MELODY DIRECTLY?
We attempted to communicate these very thoughts through a melody by translating the brain signals into a musically satisfying melody using machine learning.
Brain signals are electrical impulses which became the carriers of memory
During the experience, participants are free to think anything which they want to capture in the form of a melody. We encouraged them to think of a cherished memory, run wild with their imaginations, things that excite them. These thoughts give rise to series of ionic currents in the brain. These ionic currents are captured by the brain reader headband.
These analogue signals are further converted into digital musical notes, and a computer uses machine learning to transform them into a classical music style melody. This is then transferred to the music punching machine. The machine then transforms the melody into punch hole pattern which is then punched into a sheet of paper. This sheet of music can be played with a physical cranking music box. The music is also available for the user online by scanning a QR code which will led the user into a digital platform for them to download their unique melody.
With this experimental project, we attempted to actualize the human imagination or recall a memory in the form of music. Using machine learning, we blend the brain signals into a Bach style classical melody. This melody is further presented in a form of punched sheet to be played with a cranking music box. The process of the transformation of the electrical brain signals into digital pulses in the software run by machines, to a tangible sheet of music exhibits the fluidic flow of information from the physical to digital and then back to physical form. This truly is a phygital experience at play.
What it feels like to listen to music created by your brain signals
Over hundred participants have experienced their brain melodies. And ironically, found it very difficult to express what they felt. Recording the brain signals being fascinated by the idea and intrigued to listen the final music, they seem to fall short of words while expressing how they felt in the end. It almost always pushed them to be an artist themselves, looking for metaphors, musical pieces, etc. to express the ineffable feelings.
As the creator of this experiment, we also witnessed the joy of our participants whenever they heard the music their brain had to sing. It was remarkable to see the curiosity during the process grow into discovering the final music as if it opened a new medium of self-expression.
Photos shot by João Nogueira and Ana Torres at the House of Beautiful Business 2019, Lisbon