Audio illusions and the predictive brain
HEADPHONES ON!
Although visual illusions have been extensively studied, audio illusions have been researched less and underapplied in artworks. In Musical Illusions and Phantom Words, psychologist Diana Deutsch explains audio illusions as follows.
Unconscious inference
Neuroscientific theories about the predictive brain reveal that what we hear is influenced by the background context of our prior beliefs, knowledge and expectations about the world.
Sine wave speech
Listen to this pattern. What is being said?
Initially, It may seem to be nonsense until we hear the sentence ‘the man read the newspaper at lunchtime’.
Listen again; with this knowledge, the pattern sounds meaningful!
Verbal transformation effect
Listen to this recording of a word repeated many times on a loop.
What did you hear- ‘farewell’ or ‘welfare’?
As our brain tries to make sense of the sound, we can hear new words or even hear syllables reversed if they are both meaningful words.
Phantom words
Listen to these recordings. They each contain two words or two syllables that are repeated over again but offset in time between the headphone speakers. What words can you hear?
We tend to hear a jumble of meaningless sounds but soon distinct words and phrases appear as the brain combines these sounds to make sense to them. A third stream of words and phrases can even seem to come somewhere between the speakers.
The actual words are:
PhantomWord_1: Boris
PhantomWord_2: Go Back
PhantomWord_3: Harvey
PhantomWord_4: Igor
PhantomWord_5: Nowhere
PhantomWord_6: Hilda
Listen again. With this knowledge, what do you now hear?
Principles for perceptual integration
Rather than collections of unrelated elements, we perceive auditory information as integrated wholes, according to key principles, such as proximity and continuation.
Proximity
In this example, two tones spaced an octave apart are alternated repeatedly, and the identical sequence is played to both ears simultaneously. However the tones are out of step with each other so when the right ear receives the high tone, the left ear receives the low tone, and vice versa. What can you hear?
The recording plays alternating high and low tones in each ear but we hear a single tone, switching from ear to ear and switching its pitch as it does so- high tone in one ear and low tone in the other ear. Switching the headphone around makes no difference to how the patterns are heard!
We perceive sounds that are close in pitch to have come from the same source than sounds further apart in pitch. This is the same principle behind the Shepherd Tone- the sonic analogue of the impossible Penrose Staircase.
Take a listen:
The Shepherd Scale involves a series of pure tones separated by octaves but with their amplitude modulated in ways that we draw on their pitch information to distinguish them, causing us to hear a circular succession of tones. If we vary clockwise the semitone steps between the tones, it sounds like an endless ascending scale. If varied anticlockwise, it sounds like an endless descending scale.
The Risset Glide is a variant where we glide through the tones instead of moving stepwise.
Continuation
Listen to this example. What do you hear?
First, we hear a long tone with a gap in the middle that is clearly heard. Then a noisy sound is heard that then seems to fill the gap in the long tone so that it sounds continuous. When we hear fragments of information we tend to fill the gaps and perceive continuities between them similar to what our eyes do when we see the Kanizsa triangle.
Relationship between music and speech
Speech to song
Usually we draw on cues to determine if a phrase is being sung or spoken. Listen to this recording:
By repeating a spoken phrase over and over, it’s soon heard as a song. Even when the sentence is played again it starts out by sounding like normal speech yet when it comes to the phrase (that had been repeated) it sounds like a song!
Repetition is common in songs (as well as hymns, poetry, speeches and protests) to create mesmerising and rousing effects unlike in speech since repetition is not helpful to convey information efficiently.
Spatial sound
Spatial sound is illusory since it exploits psychoacoustic effects to make real sound sources vanish perceptually in favour of virtual sources. Ambisonic recordings and specialised software allow the location and movement of sounds to be controlled on, and around, a sphere of sound.