IT SOUNDS like a firework, a bang followed by a crackle of faint sparkles. Then, a background hum builds. Soon, that is overtaken by what sounds like a crashing wave, followed by another and another, each louder than the one before. In between the waves, random notes beep.
This is the sound of a black hole. Specifically, a “black hole-star system” around 7800 light years from Earth called V404 Cygni. The firework is the sound of the black hole. The crashing waves are light echoes, bursts of energy that bounce off gas and dust in the vicinity. The random notes are individual stars.
Advertisement
This isn’t what a black hole would sound like in reality. It is a soundscape created by NASA to represent data from telescopes. Using sound this way, known as sonification, isn’t new. For decades, it has mostly been used for public outreach or by a handful of astronomers who are blind or partially sighted.
But in recent years, more and more astronomers are realising the benefits of “listening” to the universe. It enables them to sift through swathes of data they would otherwise struggle to analyse and even pick out signals they might have missed. “Our auditory system can often discern patterns and extract meaning, even when our visual system is not able to do so,” says at the Georgia Institute of Technology. Now, a movement is under way to transform into sound the influx of data from observatories around the world and beyond. The hope is this will offer an extraordinary new take on the universe and lead to more discoveries.
The first astronomical findings made this way were accidental. In 1932, US physicist and radio engineer Karl Guthe Jansky was tasked with identifying sources of static in overseas radio communications. Using a rotating antenna that he built and nicknamed “Jansky’s Merry-go-round”, he could hear a persistent hiss throughout the day. After ruling out all sorts of other possible explanations, Jansky eventually discovered the troubling hiss was, in fact, the result of radio waves coming from the centre of the Milky Way. This was the birth of radio astronomy.
Lightning on Saturn
It wasn’t until the 1960s that astronomers began listening to their data on purpose. And it was at the University of Iowa who pioneered the technique. When data came back from the Voyager 1 mission as it flew past Io, a moon of Jupiter, in 1979, Gurnett listened to the signals and identified low-frequency radio waves. In the 1980s, Gurnett and his colleagues used sonification to identify problems affecting the Voyager 2 mission as it traversed the rings of Saturn. When they converted signals from the craft’s radio and plasma wave science instrument into an audio representation, they heard sounds that they described as . This led to the discovery that the size of grains of dust were bombarding the probe.
Later, in 2004, Gurnett took data from the Cassini mission and turned it into sound to as well as radiation emitted by electrons in the planet’s auroral zones. The popping and crackling of the lightning sound like the kind of noise you might hear if you listen to a radio during a thunderstorm on Earth.
Gurnett, who died in January 2022, was keenly aware of the benefits of listening to, rather than just looking at data. For one, sonification creates a new way to digest complex data. Say you want to understand the way a galaxy gives off a certain frequency of light. You are trying to work out how that is related to that galaxy’s age, mass and other properties, like the way it moves or the gas it gives off. This is more than the four dimensions (height, width, depth and time) that can be displayed visually. If we use sound, we can portray up to 12 dimensions at once, says Anita Zanella at the University of Cambridge. “Sound has up to 12 parameters that characterise it, including frequency, rhythm, spatialisation and so on.”
A further benefit lies in the fact that we can focus on several sounds at the same time while filtering out unnecessary noise. This is the skill that helps us understand what our friends might be saying in the middle of a loud party, hence why it is dubbed the cocktail party effect.
That makes sonification a way to help pick out weak signals in noisy astronomical data. “A lot of [astronomical] research is about detecting faint signals from distant sources in space within a noisy background,” says Zanella. Gravitational waves are a good example. Take, for instance, two black holes merging. In one piece of research, Zanella simulated the gravitational wave signal this merger would give off and inserted it into an actual dataset from the Laser Interferometer Gravitational-Wave Observatory, or LIGO. “Visually, you cannot distinguish the signal from the noise,” she says, “but when you listen to the sonification, it is clearly distinguishable.”
Someone else who knows all about the value of this technique is , an astronomer at the European Gravitational Observatory in Cascina, Italy. She was a teenager when she started to lose her eyesight. “I would look at my mother,” she recalls, “and while I knew she was there and I could see everything else around her, I couldn’t see her.” As the years went by, her vision deteriorated until, at university studying physics, she faced the prospect of going blind. She thought she would have to quit.
This all changed when a friend played DĂaz-Merced the sonification of a coronal mass ejection, a giant expulsion of plasma from the sun. In that moment, she had an epiphany. With the help of colleagues, and a lot of time combing spreadsheets, DĂaz-Merced began turning all her data into sounds. She has since used sonification to study gravitational waves, black holes and gamma ray bursts – the brightest displays of light since the big bang.
On one occasion, DĂaz-Merced was listening to the sounds of a gamma ray burst when she heard strange chirps, squeaks and rustles within the noisy data. The sounds reminded her of . On the back of this, she and her collaborators discovered that the gamma ray bursts gave off energetically charged particles that are crucial to the formation of new stars. “We are used to listening to multiple sounds at the same time, separating signal from noise and determining how the various sounds are related – or not,” says Walker. “Sonification can convey information about multiple sources or events or data streams, and listeners can process and make sense of it.”
This is particularly important at the moment in a branch of astronomy that seeks out so-called transient phenomena. Also known as time-domain astronomy, it is the study of transient objects in the sky that change over relatively short timescales, for example the explosion of a dying star that only stays bright for a few weeks or the flash of a rapidly spinning neutron star seen for a matter of seconds. Many of the out-of-the-blue discoveries made in astronomy to date have been accidental detections of transient events and some people think transients are the best way to discover things we can’t even imagine exist. For this reason, the Vera C. Rubin Observatory in Chile, which is due to start observations in 2023, will be scanning the entire southern sky every three nights for 10 years for the Legacy Survey of Space and Time (LSST) – designed specifically to look for these kinds of flashes. The observatory’s newest project, , will offer anyone full access to LSST data represented as audio in an effort to expand access to its archive.
In Australia, a project called Deeper, Wider, Faster, which coordinates more than 30 telescope facilities worldwide and in space, has already started using sonification for monitoring transient astronomical phenomena. This has identified thousands of such phenomena within our own galaxy, including dozens of M-dwarf flares, which are stars that experience unpredictable and sudden increases in brightness over timescales of seconds to minutes, and type 1a supernovae, stellar explosions involving binary star systems.
You might wonder, given the complex nature of the datasets astronomers obtain from our telescopes, why artificial intelligence and machine-learning systems aren’t being used instead of sonification. However, automated analysis is generally based on known behaviour and phenomena. In other words, this kind of AI is good at picking out specific signals it is designed to look for, but not so good at noticing something different and saying “that’s interesting”. At least, not yet.
But the universe is full of phenomena we are yet to completely understand, and a large part of astronomy – particularly that conducted by telescopes like the Vera C. Rubin Observatory – involves scanning the sky searching for something new. For noisy and multidimensional datasets, sonification provides a chance for human input to help identify interesting features, as well as discern the significance of outliers and anomalies, train machine-learning software and decide what is relevant in the data. AI systems still have a way to go in emulating these features of human cognition, especially when confronted by uncertainty.
One stumbling block is that there are multiple ways to turn astronomical data into sound. Because of its piecemeal history, astronomers who use sonification today mostly do so using techniques passed down by colleagues or that they have developed themselves. These methods often aren’t rigorously tested for their efficacy and are neither published nor peer-reviewed, further limiting knowledge exchange and increasing the risk of reinventing the wheel many times.
To tackle this, Zanella has collaborated with the psychology department at the University of Padova in Italy to develop the first joint PhD in astronomy and psychoacoustics, which is the science of how we perceive sound. The programme began at Padova this October, with the aim of finding the best ways to sonify astronomical datasets and establish standardised methods, so that, one day, these will be the standard techniques available to everyone.
For DĂaz-Merced, this all adds up to an exciting step forwards in astronomy. More telescopes turning their data into sound improves the accessibility for people who are blind or visually impaired, and means fewer people will face the decision she was confronted with – whether or not to continue studying physics.
Beyond this, she is happy to see a technique she has used for so long start to be taken up by the wider community. “There is still a lot to be done,” she says. Many scientists are yet to accept that there are other ways to analyse information, but she remains hopeful. “There is a universe of possibilities and I’m really excited about it.”
