What is SoundPainter?
SoundPainter is a performance and recording instrument.
The system allows a performer to place audio in a three-dimensional space, and then explore the sound with their hand.
In the summer of 2015, I became obsessed with the idea of molding sound in a physical space. Several months later, SoundPainter is now a functional system stable enough for live performance.
It was chosen to represent the BYU Graduate School of Music at the BYU Grad Expo. SoundPainter was voted among the top 5 displays.
Find out how it works.
Download the Max Project Files
More Sound Paintings
How does it work?
SoundPainter begins with simple input from the Kinect and an audio signal.
The Kinect requires a USB 3 port. An i7 processor is recommended.
A wireless microphone connected to a simple audio interface allows for the performer to record vocal samples, though the audio signal can be from any source.
Software
The entire system is comprised of four applications:
KinectV2-OSC by Andrew McWilliams.
Max 7 by Cycling 74.
Ableton Live by Ableton.
A Python script for OSC routing written by Matthew Web (I originally used OSCRouter by ETC Labs, but the data began to lag several seconds after an OS update).
Max Patch
The SoundPainter Max project (download demo files) recieves OSC messages from KinectV2-OSC through Matthew Web's Python script. These messages allow Max to be aware of the location of several body points (x, y, z coordinates in meters). This system uses only the left hand.
When the record state is triggered, a function plots a series of points along the motion path made by the left hand. A sound clip from the audio source is also recorded.
This sound clip is then saved in a file path determined by a Max-for-Live granulator.
After the path and audio clip are both recorded, then patch enters a playback state. While in this state, the patch constantly returns the closest point of the motion path (in relation to the left hand) to MIDI CC. This is mapped to a corresponding location on the granulator in Ableton Live.
Volume is also ramped by proximity to the closest point.
This Max project is NOT in release condition, though anyone is free to use or reference the files.
Sound Paintings
What's Next?
Despite it being performance capable in its current state, I hope to make SoundPainter a stand-alone application.
The project will be open source and hosted on GitHub.
Currently, I am still improving my programming skills and I have much to learn. An application may be several years away.
I am currently pursuing my enthusiasm for gestural control through many avenues.
SoundPainter will remain companions with my many ongoing projects.
