Recently, I found music visualization can be one of the possible approaches to immerse individuals with hearing loss in music and help them feel the part that cannot be heard, and it is also an artistic, interactive visual format to show the beauty of audio properties.

Music Visualization

Music Visualization means transforming music into visual formats, such as graphics and animations. The changes in the music’s loudness and frequency spectrum are among the properties used as input to the visualization. Effective music visualization aims to attain a high degree of visual correlation between a musical track’s spectral characteristics such as frequency and amplitude and the objects or components of the visual image being rendered and displayed.

What Is Processing?

Processing is an open-source graphical library and integrated development environment (IDE) built for the electronic arts, new media art, and visual design communities with the purpose of teaching non-programmers the fundamentals of computer programming in a visual context. In other words, it is super useful for artists or designers to present their ideas through coding.

Music Visualization Demo Made by Processing

Watch demo video and imagine what the original music sounds like.

Please listen to the original version music to see if your imagination is correct.

*Lower your headset volume before listening to the original music if you just finish watching the video.

Original music: Fragmented – Gavin Luke

It is obvious that if high frequencies are lost, the space of sound, the reverberation and the details of instruments will be weakened. And that can lead to the loss of the information (like emotions) to be conveyed.

In order to minimize the influence of high-frequency loss, I built this demo. It was created by Processing, based on my personal feelings when I listened to this song. First, this song is in a sad, sentimental mood, so its colors are in a cold tone. I also would like to reflect its floating part, thus purple, which gives a sense of mystery and futurism, was chosen. The outer ring of purple wave, moving according to the song’s waveforms, is made to display the ambient reverberation and the space of sound. The center circle is controlled by loudness and beats, to provide information about these properties. I added shining blue or green circles (controlled by both waveforms and loudness) to the demo to symbolize light spots in the dark, because I feel a sense of hope in this song even though it’s in a sad mood.

I am just a very beginner at Processing, so my demo is just the very beginning of my exploration of music visualization, especially my MRP audiences are deaf and hard of hearing. In addition, my demo is only based on my own feelings, which must be not accurate in transforming. My future steps will be cooperating with deaf or hard-of-hearing individuals to learn their thoughts and iterate my design.

Music visualization is not necessarily the future direction of my MRP, but I believe that it will be a valuable exploration.

Survey & Feedback

This is a simple anonymous survey to help me evaluate my exhibit and my design ideas. Participation will be competely voluntary (no stress!), but I will be glad to hear any feedback!

It has to be noticed that:

• The data will not be used in my MRP
• Its purpose is to evaluate my exhibit activities and medias, and to see your opinions on my design ideas
• You participation is competely anonymous and voluntary
• All your answers will be deleted after this exhibiton event
• All your answers will not be disclosure to others.