By Lina Sorg
“The intersection of mathematics and music is a very old subject,” Daniel Forger of the University of Michigan said. “When people would teach math and music a long time ago, during the time of the Greeks, these things were very much intertwined.” Recent research has identified clear overlap of these two subjects with neuroscience as well, particularly as functional magnetic resonance imaging reveals detailed analysis of brain patterns and digitized music yields insight into the minds of musical performers. “There’s something in the middle of all three, and our job it to figure out what that is,” Forger said.
During a minitutorial presentation at the 2018 SIAM Conference on the Life Sciences, currently taking place in Minneapolis, Minn., Forger presented a series of talks exploring the mathematics behind auditory processing, musical representation and music theory, and the neuroscience of performance. In Western music, the musical keyboard is divided into a repeating sequence of 12 possible notes. Groups of two or (typically) three notes form chords. “One of the remarkable things about our brain is that we can listen to several different melodies happening at the same time, track all of them, and appreciate all of them,” Forger said. “The interesting thing is how this is represented mathematically.”
In 1739, mathematician Leonhard Euler created an important conceptual lattice diagram called the Tonnetz to describe tonal space and harmonic relationships. He arranged all of the notes on the keyboard in a particular way, yielding many chords and revealing specific relationships between certain chords. Any triangle in the diagram represents a cord, with the chord’s three notes comprising the three vertices. A downward triangle indicates a major chord, while an upward triangle designates a minor chord. Using his Tonnetz, Euler argued that the relationship between chords was dependent on their distance from one another. His interpretation stood until about 10 years ago, when composer Dmitri Tymoczko used orbifold analysis to indicate a larger relationship in musical space. He rearranged Euler’s Tonnetz to yield a new diagram with a Mobius strip in which the boundaries become mirrors. Tymoczko argued that mathematical orbifolds accurately represent most music, suggesting that the majority of chords do not branch out particularly far from the center of the diagram.
Forger tested Tymoczko’s theory on various pieces of music and found that chords do not typically stay in the center, but rather jump around the diagram. He examined the chords of J.S. Bach’s Trio Sonatas for organ, a complicated set of pedagogical pieces consisting of 18 movements. Forger found that Bach implemented 354 of 364 possible chords, and used 6,582 unique chord progressions. “He’s got this tremendous palette, and he’s not using the same chord progression over and over,” he said.
Forger proceeded to plot Bach’s 354 chords in a linear way. “There’s a very clear structure to it, it’s not just some random matrix,” he said. Forger also noticed that when shifting from one chord to the next, usually only once voice moves at a time; this yields a vertical pattern in the progression. “What Bach is typically doing is having chords move by moving one of the voice parts up or down a little bit,” he said. “He likes to keep things as close together on the keyboard as he can.” Additional horizontal and vertical banding on Forger’s diagram marks when different arrangements/chord combinations travel back to the same chord. Keeping such patterns in mind, researchers can further examine the graphical properties of these musical representations on a network diagram. They can also use a standard Google PageRank centrality to further understand harmony, counterpoint, central chords, and the way in which these chords travel.
Forger then addressed the growing relationship between mathematics and neuroscience, which begins with the inner ear’s dynamics. When sound enters the inner ear, it vibrates in the eardrum. The delicate bones of the ear then transfer the energy from the soundwave to the cochlea, the ear’s key sound-processing organ. The cochlea is wound like a snail shell and contains two fluid-filled tubes, separated by the basilar membrane. Vibrating sound in the cochlea pushes against one of the tubes, which in turn pushes against and vibrates the basilar membrane. Upon feeling this vibration, the neurons on the membrane transfer the sound to the brain stem, which moves it to the higher brain for further processing and swiftly activates many different regions. “Music is this very large phenomenon that encompasses the whole brain,” Forger said. “If you want to study music in the brain, you have to model the entire thing.”
For example, the left and right auditory tracts try to localize a sound’s origin. The superior temporal gyrus, which contains the primary auditory cortex, distinguishes between different types of instruments and evaluates a sound’s timbre. “Neurons within the auditory cortex tend to fire depending on different frequencies that are aligned within a physical dimension,” Forger said. Furthermore, a lot of music is about expectation — the building and release of tension. This relates directly to emotion and consequently engages the amygdala in the cerebral hemisphere. The prefrontal cortex pertains to performance expression. Reading music employs the visual cortex, and recalling any memories or particular rehearsal technique actives the hippocampus. “The brain picks apart different parts of a song or melody for processing,” Forger said. “In many instances, it will look for patterns after hearing just a few notes.”
The pipe organ is an effective test case for studying the intersection of music and neuroscience. Early organs were quite mechanical and required performers to physically open the pipes, which was understandably inconvenient. However, most organs built after the 1850s contain a coding system that converts each note to an electrical signal and sends the signal to the pipes. Researchers can encode this signal with MIDI standards and accurately digitize entire pieces of music. Unlike other instruments, an organ thus supplies an exact replica of any given performance.
Comprised of three fairly strict voice parts, Bach’s Trio Sonatas present a tremendous challenge for an organist and are heavily taxing to the neural system. Preliminary digitation of these compositions show general patterning. “You can take individual parts and melodies and start to analyze and form segments,” Forger said. He focused specifically on the Toccata in F Major (BWV 540), which begins with a canon (round); one voice starts playing a particular theme, another begins the same theme after a certain duration, and a third follows suit. In this case, the left and right hands play the same notes on top of each other, and the foot pedal plays a similar melody with the same motifs.
“Attack and release are both important on the organ,” Forger said. “There must be silence between notes in Baroque performances.” When isolating and comparing the left hand, right hand, and foot pedal, he noticed messy inconsistencies. For instance, the mean duration of a sixteenth note played by the left hand is .16 seconds, and the mean time between sixteenth notes is .03 seconds. The mean duration of a sixteenth note played by the right hand is .14 seconds, and the mean time between notes is .05 seconds. Although the two hands are playing identical melodies, the right hand plays shorter notes with slightly longer gaps between each note. And the sixteenth notes played by the foot pedal have a mean during of .14 seconds, with the mean time between notes clocking in at .07 seconds. Thus, the pedal is the least accurate when compared to the written score.
Interestingly, the coefficient of variation pertaining to the length of each note is almost identical; this reaffirms the sonata’s strenuous nature. “By having the coefficient of variation so tightly controlled, we think we’re hitting a biophysiological logical limit,” Forger said. “It’s almost like watching an athlete perform.” This realization leads to a profusion of further questions related to the physiological limits of performance.
Nevertheless, Forger has made great strides in identifying the overlap and interconnectivity of mathematics, music, and neuroscience. By examining discrepancies in digitized organ performances, he gains a deeper understanding of the way in which the brain encodes music. “In this particular case, we have a really interesting system to understand how the brain processes these really difficult pieces,” he said.
Forger performed a concert that included Bach’s Trio Sonatas for organ at Westminster Presbyterian Church on the evening of Monday, August 7, putting his mathematical explanations into practice. Below are three audio/video snippets of that concert.
 |
Lina Sorg is the associate editor of SIAM News. |