The Wii-Painter Project

The aim of the Wii-Painter project is to produce a new, expressive audio-visual tool to benefit certain participants in iMUSE environments.

The prototype system utilises the accuracy and low cost of the Nintendo Wiimote hand-held sensor, along with the rapid prototyping benefits of the Max/MSP/Jitter development platform, which includes an extensive library of audio-visual primitive functions.

The Wiimote sends it's real-time orientation via bluetooth to the host PC. These signals are mapped by the Wii-Painter software to control two independent software synthesisers, and the drawing parameters of a graphical sub-system.

The participant can build up their own audio-visual composition by "painting" with the wimote; the visual results being displayed onto a screen projected in front of them, and the audio being heard and felt via speakers and a soundbox or soundchair.

Rather than emulate a standard "paint package", the sub-systems are configured and controlled by the therapist and the participant to result in aesthetically pleasing, dynamic, abstract audio-visual output.

Stills of the picture, or complete audio-visual movies can be recorded so that the participant can re-live the experience, or take away a colour printout as a memento.

Two examples of the display [image]      The Therapist Interface [image]

Figure 1: Examples of the display                  Figure 2: The therapist interface

Click to see the movie of the WiiPainter

Click for Movie of the WiiPainter (16Mb).

Other Resources

  • The Design of the the WiiPainter Application
    Powerpoint presentation (here in PDF format, 848 KB) with notes (sorry, no embedded movies), given by K. Brown at the AVPhD (Sound) Symposium, University of Sunderland, 4th July 2008.

Future work will address:

  • parameterised sound and graphic generation
  • more sopisticated and expressive mappings
  • enhanced configurability of mappings for different subjects.