[music]

Woman 1: In spring, 2007, UVS, sponsor of the Vervier [sp] Orchestra, asked TCNJ professor of music, Teresa Nakra, to produce an interactive musical experience for concert goers. Dr. Nakra, together with a development team that included interactive multi-media professor Chris Ault, produced a virtual maestro conducting kiosk, shown here.

The kiosk allows users to conduct a video of the Vervier Orchestra, using a hand held game controller. The Virtual Maestro kiosk is an offshoot of an earlier project that Dr. Nakra created.

Dr. Teresa Nakra: I'm Teresa Nakra, and I'm an assistant professor of music here at the College of New Jersey. I had designed an exhibit that's currently in the Boston Children's Museum, where the idea was that children could conduct a big video of an orchestra and learn what is it like to be a conductor.

So, someone had read about that work, and was interested in making a different version of that technology for the general public. Once we built the kiosk, the sponsors of the project decided that they wanted to promote this kiosk more than just for the Vervier Orchestra's tour. So, in fact, they build two copies of the kiosk, and those copies are going to different places all over the United States and Europe over the next year.

[music]

[applause]

Chris Ault: My name is Chris Ault, and I'm an assistant professor. I teach a number of courses in the interactive multi-media program. Part of the IMM program, we teach video game development courses. We emphasize the interdisciplinarity. I've had the opportunity to collaborate with Dr. Nakra on this course.

Teresa approached me about working on the project. My job, at that point, was to really make it look nice and make it easy and quick for people to use in the busy sort of environment in which this was going to be displayed.

Dr. Nakra: What I try to reach for in my work, in the use of these very new tools, these interactive technologies, to enhance the live performance situation. As a result, the Wii-mote ended up into this kiosk. And, I wouldn't have learned about the Wii-mote, except that Chris brought it to class.

Chris: The Wii-mote has been an amazing development in the field of games. Immediately, when it came out, you saw websites springing up left and right about hacking the Wii, hacking the Wii-mote. Nintendo, fortunately, has been very encouraging of this sort of thing.

It's an obvious sort of application for this, when you think of holding this white thing in your hand, and back and forth sort of hand gestures like that. And conduct, not as a real conductor would conduct, but to conduct how you think you're supposed to conduct. You just pick the thing up, start moving your hands, and it just works.

Dr. Nakra: We spent a lot of time, actually, particularly in the spring semester, studying the Wii and trying to understand what was good about it and what was bad about it. How the sensors got integrated into that unit. It's got accelerometers in it, so it's measuring how much force you're using when you wave it around.

You can move it around with a lot of force. And, the faster you move it, the faster the video plays. So, basically what you're doing is you're controlling what we call the "tempo" of the orchestra.

[Musical interlude]

Dr. Nakra: If you happen to be moving it in actual beats, then the orchestra will synch to your beats.

[music]

Dr. Nakra: If you make really forceful gestures, use a lot of effort, then they'll play louder.

[music]

Dr. Nakra: The music department is very eager to find ways in which we can integrate technology into our course offerings. So, there's definitely some possibility for learning about the design of interactive music systems.

In addition, I know that there's a lot of interest in the interactive multi-media department for future kinds of public interactive systems like this. And so this kind of kiosk will fit right into that mode. How do you design a system that anyone off the street can come in and try it, and that it'll work for them.

[Audio ends]