What is the Shape of Music? A Fractal Sound Analyzer

What is the Shape of Music?

The other tools on this site have tangible output; you can see it, you can touch it, or you can read it. Music as an output is untouchable; we can certainly interact with it, but the results are ephemeral. That means that we have to think of another way to quantify the “shape” of music.

How can we do this? Sound is perceived by the ears, and then it vanishes. Luckily for us, notes can be measured and recorded as data. In this tool’s case, we are going to break up the piece into tiny parts and measure the shape.

In particular, we are going to examine the fractal characteristics of music. It is expected that music and sound will fall into fractal-like patterns as all things physical fall into predictable shapes and patterns by dint of being influenced by physical systems.

How Do We Analyze Sound?

To analyze a downloaded sample, we first prepare the music by cutting the file into 46-millisecond sections of time. This makes the section consistently available as points of data.

This data is fed into a math operation called a Fourier transformation, which is just a way of taking each time segment and analyzing its character. In this case, we are looking at the frequency of the sound and how loud it is. .

Human hearing is most acute between 1000Hz and 4000Hz; frequencies above and below, we have a much harder time hearing. The data frequencies are remapped to a Mels scale (MFCCs — Mel Frequency Cepstral Coefficients), which is a scale that mirrors how human sound perception works.

The Mels scale output is further processed by a discrete cosine transformation the reduces the Mels scale to a set of 13 numbers, which are used to describe the timbre of that space of time. Our small section of music is reduced to a large set of data of thirteen numbers, each representing a period of time of 46-milliseconds.

Now the fun part, our tool takes this data and finds the three directions that contain the most variance and sets those as axes.

Finally, the points are placed in a 3D visualizer by time and connected with a line.

Beauty as Sound

And what could be more beautiful than sounds crystallized, playing softly and spinning gently; a galaxy unearthed, the same as the tool above, only optimized to feature the harmony of shape. Magic.