Brain-computer interfaces (BCIs) that allow people with severe neuromotor or motor disorders to communicate are becoming more and more common. This is realised by scanning brainwaves using electroencephalography (EEG) and converting them accurately into words, letters or other objects that the user intends to replicate in their minds. BCIs are beneficial for those with extensive paralysis, ‘locked-in’ syndrome and other similar conditions.
EEG-powered BCIs have a number of advantages; it is non-invasive, (the brainwaves are ‘picked up’ through the skull using electrodes integrated into a head-hugging cap) well-validated, well recognized, and often relatively cheap. In addition, certain ranges of brainwave (or event-related potentials (ERPs), as some are also known) have also been exhaustively studied and exploited for the purposes of BCI. They include the posterior-dominant rhythm, which is associated with the high-fidelity selection of notes and other musical objects on virtual instruments, and the P300 wave that enables patients to complete tasks such as selecting numbers in a BCI interface. As such, some researchers suggest that P300 could also be used to specify and select musical notes in those who need to write music using BCI. A recent study published in PLOS One suggests that this is indeed possible. This is good news for patients who want to write their own music completely hands-free as well as play it.
Composing music through thoughts
This team, based at the Institutes of Neural Engineering and Psychology at Graz University of Technology, developed a BCI interface that was also loaded with software supporting musical composition. They based their work on existing BCIs that allow patients to paint using software somewhat similar to many conventional drawing or art apps found on everyday computers, and also previous studies that showed a 75 percent accuracy rate in selecting notes from the C major scale using a modified BCI spelling programme. The team, supervised by Gernot Müller-Putz of the Neural Engineering institute, hypothesised that such BCI musical composition was also applicable to a more extensive range of options. In other words, they linked their EEG receivers to the full musical composition suite MuseNote. A previous pilot study evaluating this approach in five healthy individuals found that this group could input a given melody into the software via the BCI with an accuracy of up to approximately 96 percent, although 40 percent of them could complete the task at an accuracy of about 50 percent. Therefore, the group designed a new experiment in which they recruited 18 healthy volunteers with musical backgrounds, including a professional composer, to test their ability to spell, write a full, pre-determined melody and compose original sequences using this BCI system.
The team used a fairly conventional BCI system, which consist of EEG detection and recording hardware, P300-wave-based software (written largely in C with Matlab for signal processing) and the MuseNote software, which was controlled in turn by the P300 software. The volunteers were able to control MuseNote while wearing the EEG cap; this involved being able to select notes, then various features such as chords, accidentals or dots for each note, and finally the pitch of each note. Users could also delete erroneous notes at will. The participants were trained to use this and a spelling matrix controlled by the P300 software at the start of the experiment. They were then asked to spell pre-determined words, such as ‘Music’ in German or ‘Liszt’, write an existing melody titled ‘Alouette’ in MuseNote, and finally compose freely using MuseNote. The volunteers also wrote out the melody by hand, and used MuseNote with a mouse to do the same, to habituate them to the tasks before attempting their completion using the BCI system.
The researchers set a cut-off of 70 percent for the accuracy with which the volunteers should complete the BCI tasks. All of them save four exceeded this threshold for all three activities. In fact, the professional composer scored between 90 and 100 percent for all tasks, while the others scored approximately 88 percent in the ‘spelling’ task (although this figure was 92 percent when corrected for the non-finishers). The non-professionals who completed the ‘Alouette’ task did so with an accuracy of approximately 89 percent, whereas the composer did so with nearly 94 percent accuracy. On the other hand, the non-professionals (n=13) who completed this task took an average of approximately 21 minutes to do so.
The amateur participants were given 30 minutes to compose their own piece of BCI music, and the majority (13) took all of this time to do so with while remaining above the threshold for accuracy. The group’s relative accuracy in this task was 76.5±17.2%, or 84.3±9.6% when corrected for participants who fell below the threshold. They composed approximately 18 notes each, and the composer wrote 14 notes. However, this participant’s accuracy in doing so was 98 percent. The participants were also given behavioural and user-experience questionnaires to evaluate this area of implementing the BCI. The researchers found that the ‘workload’ associated with using the system was moderate to high, although most users reported that they enjoyed working with it. One participant found note selection complicated, and more than one reported that the ability to pause the system in order to think about their next note would be useful.
Top image: Music brain. (CC BY-SA 4.0)
Pinegger A, Hiebel H, Wriessnegger SC, Müller-Putz GR. Composing only by thought: Novel application of the P300 brain-computer interface. PLoS ONE. 2017. 12(9): e0181584. https://doi.org/10.1371/journal.pone.0181584
Deuel TA, Pampin J, Sundstrom J, Darvas F. The Encephalophone: A Novel Musical Biofeedback Device using Conscious Control of Electroencephalogram (EEG). Frontiers in Human Neuroscience. 2017;11(213).