The Super Mario Spacetime Organ by Chris Novello (aka “Paper Kettle“) is a “codebending” video game instrument that allows him to manipulate and create music from the RAM state of Nintendo’s classic video game, Super Mario Bros. Chris does so with the use of both his custom built illucia instrument and a Soundplane multi-touch music controller made by Madrona Labs. The technology and results seen in this demonstration video are quite amazing.
Mario’s universe is held in RAM, which the NES uses to draw his world for each frame of the game. By recording the entire state of the NES memory for every frame, I’m able to go back to any moment in Mario’s life.
So then I use the X-axis of the Soundplane to sweep through the timeline of Mario’s universe.
Not only that, but the Soundplane is multitouch, so I use a second finger to specify start and endpoints in a playback loop. Technically, this is similar to the way samplers and granular synths work in audio.. but with the entire memory state of the NES. Conceptually, it is like Super Mario meets Groundhog Day. Mario’s universe computer/time machine gets caught in hellish loops.
via Sam Merrell, Rob Sheridan
