Sound and Music

Developed through collaboration with the Sound and Music (formerly Sonic Arts Network), Whitefield Schools and Centre and the London Philharmonic Orchestra, LumiSonic is a sound application that visualises sound in real-time in a way that allows hearing-impaired individuals to interact with a specifically designed, graphical representation of sound.

The software can be used ‘out of the box’ with a mouse (or another special needs input device) or with a Nintendo Wiiremote, which provides additional vibration-based feedback. Users can load and play back sounds from the remote, whilst using the accelerometer to make real-time adjustments to the pitch, pan-position and volume. Beyond this, the musical output of the software can be set to a number of transposable scale systems. This allows for a wide range of musical interaction (i.e. You can use it as an musical instrument!).

What you can do with it…

There are four basic things you can do with this software:

1. You can use it to get Visual Feedback of a sound that is happening in real-time. This can also be used for speech therapy, and understanding harmonics
2. You can play sounds that are already recorded and manipulate them to create whole new sounds
3. You can do use it to manipulate sounds that are happening in real time. E.g. If you have a microphone recording a musical instrument, a child can alter the sound by using the software (possibly using the wii remote!)
4. You can use it to record sounds and then play them back

How does it work

Musical interaction is conducted through the alteration of the visualisation itself. Through learning to control a number of specific gestures, the users can alter the appearance of the visualisation instantaneously. The visualisation is then converted back into audible sound, perceptually occurring at the same time as the original sonic material. The sound and music is effectively 'translated' into useful visual information that can be sensibly interpreted and changed in real-time by users who are unable to hear and react to the sound.

Download

Lumisonic Lite PC (5.2MB)
Lumisonic Full Installation PC (30MB)
Lumisonic Mac (14MB)

Please note this software is only a prototype and still in a testing phase