Sonification in Recreational Mathematics

1. Introduction

Many areas of recreational mathematics, such as Fractals or Tilings gain popularity because the concept they cover can be represented visually in an interesting and enticing manner, reeling in an audience with the use of a single image or thumbnail. The idea behind this article is to explore mathematical structures recreationally using sonification instead of visualisation, aiming to provide an accessible and fun experience for a blind or low vision audience with an interest in mathematics.

A cursory glance at pre-existing sonification of concepts in recreational mathematics points towards a wide range of attempts at endowing various mathematical structures with sound. Many of the ones I've seen can be categorised into two genres:

1. Those that use sound to convey a general idea of the structure being studied, while prioritising artistic expression over information conveyed. An example of this is NASA's sonification page, which is very cool to listen to and have much artistic merit to it, but acts more as aesthetic representations of the data, rather than a tool that provides a robust framework to understand the data or mathematics present.
2. Those that use sound as an extra dimension for understanding a mathematical structure or a dataset, but fail to make it accessible to a blind audience. Good examples of this include the sonification of Beijing Air Quality Data, which is used as the primary example of sonification on its wikipedia page, or CodeParade's video Sounds of the Mandelbrot Set. The second example sonifies an extra piece of information for every point on the Mandelbrot set (or similar fractals) which is otherwise not visually present, but this information remains inaccessible to listeners with visual impairments, as no method for navigating the Mandelbrot set without visual input is offered.

Another example of recreational sonification that I enjoyed, but that I believe has both the issues above is Matige KunstIntelligentie's Youtube video What do Fractals Sound Like (and Part II), as well as various other attempts at sonifying the Mandelbrot Set, which often seem somewhat arbitrary to me, but do make for some cool sound art, I suppose. (Niche observation, probably with few people who can relate, but these kind of reminded me of when the aperiodic tiling The Hat was discovered, and suddently people were discussing the physics of electrons propagating on the Hat, which just seemed incredibly arbitrary to me, all things considered).

Anyway, this blogpost aims to avoid the above outlined issues.

The two main sections of this article explore two different self-similar fractals, The Weierstrass Function and the Cantor Set, respectively.

2. Preliminaries

3. The Shepard Scale, the Weierstrass Function, and White Noise

Shephard Scale is a fun auditory illusion consisting of a tone whose pitch appears to perpetually increase (or decrease), while in reality the sound is periodic and therefore not monotone increasing (or decreasing respectively). Below is a sound clip of five periods of the scale.

As you can hear, although the tone seems to always rise in pitch, it is also fairly clear from listening that at no point in time does this tone reach any particularly high notes. Here is the same sound clip again, but with a clicking noise to delimit each period of the scale, which is repeated five times.

So how does this happen? At any fixed point in time in the Shepard Scale, the sound that plays is called a Shepard Tone, and consists of some note being played, along with that same note played at all octaves above and below it. Here is an audio clip of the Shepard Tone for the note C, for instance.

To form the Shepard Scale, successive Shepard's tones are played, with the "base note" increasing in pitch. When this scale reaches one octave above the stating note, the Shepard Tone at that point in time consists of that note being played in all octaves, which produces the same sound as the starting tone. So an increase of the Shepard Scale by one octave represents a single period thereof, and can be repeated from there.

[actual maths to be inserted from here on out]

4. The Cantor Set

1. Iterative construction of the Cantor Set, including audio output
2. Generalising the cantor set and introduction into the hausdorff dimension of a fractal
3. Fat cantor set plus some cool other cantor-like constructions and pathological factoids that can be briefly mentioned without too much accompanying sonification. I've just gone down a contor rabbithole on wikipedia so have a brain teeming with ideas for this.

5. More Ideas and Bounty Offers

The remaining section consists of ideas that I have tried and failed to implement in the writeup of this post, but that I would still be interested in developing. I offer a bounty for the resolution of each issue that I describe below. The value assigned to each bounty is dictated by a combination of how interested I am in it being hunted, as well as how difficult I estimate it will be to hunt, and I reserve total power to determine whether the conditions for any particular bounty have been met. Please feel free to email me (or contact me through other means) with your inputs! For every resolved bounty, I will be interested in working with you towards adding it to a section of the post above, obviously assigning due credit where relevant :)

Many widely popular fractals have much more complex structures than those discussed above, with nontrivial embeddings in two, three, or even higher dimensional space that cannot be expressed by simple functions or equations. Sonifying these in an intuitively accessible manner has proven difficult, and I offer a bounty of £25 for a sensible sonification of any reasonably interesting fractal. (As a rule of thumb, I expect the outcome of such a sonification to be that it allows for the essence of a fractal to become more accessible or intuitive to a new reader in the same way that a diagram would do so).

As explored in the Cantor Set section above, a set \(S\) of Reals or Integers can be sonified fairly well by letting time act as the positive real axis, and playing a tone at every time \(t\) that belongs to \(S\). Different sets \(S_i\) can (optionally) be distinguished from each other by assigning different tones to them, allowing several sets to be sonified at once. The drawback of this type of sonification is twofold.

1. Firstly, it gets very cluttered very quickly, so realistically, only a small number of sets can be sonified at once.
2. Secondly, time has the unfortunate property of always moving forward, which means that the information conveyed by the sonification is stored in short term memory rather than freely available for instant recall and inspection like a diagram would be.

Nevertheless, it is useful to look at ways to represent operations on such sonified sets.

1. Unions of sets are easy - an element \(t\) belongs to the union of several sets if and only if at least one tone is playing at time \(t\).
2. Intersections are more complicated - as the number of sets increases, it quickly becomes too difficult to distinguish whether all relevant tones are playing at a given time \(t\). In this instance, it behooves us to make use of De Morgan's Laws. Indeed, if we instead play a tone during the complement of each set, then the times \(t\) at which exactly no tone is played correspond exactly to the elements \(t\) that belong to the intersection of all the original sets.

The above observations are fun to think about, but ultimately I couldn't find any interesting topics to apply them to in an educationally meaningful manner, so instead I offer three distinct bounties for the different ways in which I can extend the above ideas:

1. £5 for each suggestion of real/integer sets and set systems to explore that would be viable to produce at least one mathematically stimulating paragraph in the above blogpost.
2. £15 for each original and well-developed idea for sonifying subsets of sets that aren't just the reals or integers (or some evil trivial/pathological sets you cheeky freaks are bound to come up with).
3. I also struggle with combining my suggestions of sonifications of unions and intersections, so a more robust framework for complex set operations is worth £5.

I am captivated by the idea of sonifying points on the complex plane with the help of the Shepard Scale. I believe that the Shepard Scale lends itself incredibly naturally to representing the argument of a complex number due to the fact that it is both periodic but perceptually monotone. Sensibly representing the modulus of the complex number in question, however, remains open.

1. Volume can just about do the trick, but feels a little limited.
2. Another option is to adjust the multiplicative factor \(b\) of the tone (i.e. play all notes a power of \(b\) apart in frequency, rather than the initial power of 2 representing an octave). The problem arises is finding a sensible way for \(b\) to vary with the modulus \(r\) without ever taking on the value of 1 (in which case the Shepard Tone would collapse to a single note, and we would lose the periodic property we were after).
3. My final idea is to let the distribution \(A\) of volumes for each of the notes played in the Shepard Tone to vary with the modulus, though again one would need to find a sensible dependence on \(r\).

A bounty of £25 is offered for a sensible and auditorily perceptible choice for sonifying the modulus of a complex number. A higher reward will be granted if this choice is in some way "mathematically consistent" when considering sums, products, or powers of complex numbers.

A suggestion I've been given is to introduce the idea of geodesic curves with the help of sonification. Unfortunately, I don't see a way of incorporating sonification into this without the help of a visual representation of what's going on alongside it, which would defeat the point. Any substantial work towards this will be rewarded with £10.

My all-time dream would be to behold an auditory proof of some mathematical concept, in the same way as the current visual proofs available. A bounty of £75 is on offer.

Meandering out of the realm of mathematics education and into the realm of personal interest, I'm happy to allow for the use of "arbitrarily many" time dimensions in such a proof. Most visual proofs use more than one spacial dimension, so I dont see why the same cannot be applied to an auditory proof. Imagine you are a being that can perceive several dimensions of time (in some interpretation that makes sense to you), but that time still moves in only one direction and always at the same speed. Can you provide me with an auditory proof of some mathematical concept? A bounty of £50 is on offer.