Sound of Motion

Sound of Motion is an interactive musical experience. The game is set in the universe and users are displayed as point cloud shape using Microsoft Kinect depth data. The Kinect also detects the position of their hands. By waving hands wildly they can shoot particles out from their hands and the particles will bounce within the canvas and generate musical notes when they hit the floating “planets” on the screen.

The game is created with Processing. I was in charge of the visual and sound effect design and code implementation. I mapped our “planets” to pentatonic notes in a crystal-like timbre in order to avoid inharmonious sounds caused by random collisions. Basic chords (C, G, Am, F, Dm, Bb) in an ethereal timbre serve as ambient background sounds. I also set up the code structure in the main loop (draw() function).

Partner: Peter (Xincheng) Huang