Drumless Drumming – Richard Lee
Full-time, Richard works as a video game programmer and he himself has said that such a mix of interests is a rarity. In the most extreme circumstance, this could almost be a juxtaposition of terms. However, it is people like Richard who continue to advance the capabilities of drummers, and the equipment they use. This is especially important in a time when technology is greatly changing the pace, and development of the role of the drummer in music. I say long live Richard and the Drumless Drumming Project!
OLD: How does it work?
Richard: The version of the system I have shown relies on a rather expensive optical motion capture set-up that uses multiple cameras to track the positions of small reflective markers in real-time.
By attaching these markers to drumsticks and shoes I can infer where they are, how they are oriented, and how fast they are moving. This information is then used to detect when a drum is hit and these hits result in a sound being played, which is selected based on the hit speed and stick orientation.
The orientation information is useful to detect rimshots or if the top or the edge of a cymbal was struck. A belt with additional reflective markers can optionally be worn which allows the drum kit to follow the user as they run or jump around. I like to think of this as aerobics for drummers – airdrummerobics, if you will.
OLD: You have said you were creating a new version – what will that be like?
Richard: The new version behaves similarly in that it tracks sticks with reflective markers attached to them. The big difference is that instead of relying on multiple motion capture cameras, it only requires a laptop with a webcam and a few electrical components costing about $10.
This makes it cheap and portable without compromising the quality, or responsiveness of the simulation. The only downside compared to the other version is that you have to be seated so that you are visible to the webcam.
OLD: Can you input multiple drum kit sounds or sampled sounds?
Richard: Yes, there is complete flexibility in the layout of the drum kit and what sounds are heard. You can easily trigger non-drum samples for instance, or make the sample selection dependent on the hit volume or hit angle. If the user already has drum samples, it is simple to use these as a substitute.
OLD: What inspired you to create this idea?
Richard: Unlike most other popular instruments, drums are loud. Really loud! From the time I started playing drums in my teens this was always a problem for me. I ended up striking a deal with my neighbors shortly after I got my first kit, which permitted me to practice drums for a fraction of the time I would have liked. Drums are also bulky. When I got older and joined bands the problem became how to arrange transport to get my kit to and from rehearsal.
The new version of the system I’ve developed is intended to avoid both problems. The fact that you don’t have to hit physical surfaces means that you can practice in complete silence with headphones. The fact that all the necessary equipment fits snugly in a shoulder bag means that you can easily bring your instrument for a jam at a friend’s house (or maybe the local park on a nice day).
Finally, for people who would like to learn to play drums but are put off by the expense of a drum kit, this system provides a cheap alternative and a great way to learn. With the touch of a button the system allows you to record, play back, and print the sheet music for whatever you just played.
OLD: Do you have to go down an academic route to do this kind of work?
Richard: In fact, this was only ever a hobby project I worked on in my spare time. Although the first version of the system does rely on equipment I was lucky enough to have access to in the academic research lab where I did my post-grad. However, the new version does not use any special hardware that would prevent anyone else from developing a similar system.
OLD: What advice then, would you give to anyone attempting similar ideas?
Richard: There are three keys to making the system feel natural. The first is allowing the user to be very imprecise in their hits so whether they hit a few inches above the snare or below it, a hit will still be registered. This makes it possible, for example, to play a double-stroke roll in the air. Even though your hands will naturally drift about, this won’t affect the hits that are detected by the system.
The second is responsiveness. If there is more than about one hundredth of a second delay between the time the user makes a hit and the time the sound is heard, this lag will be felt. This will then break the illusion of playing drums.
The third is speed detection. I found that accurately estimating the speed that the stick is moving,and faithfully translating this speed into volume is crucial to making the system feel realistic.
OLD: Could you explain the complications with producing this idea for something like Xbox Kinect?
Richard: People have attempted similar projects using the Wii controllers and using Xbox Kinect.
From what I have seen, lack of responsiveness is the fundamental reason these experiments have not been a big success. For example, the Kinect’s sensor outputs video at a rate of 30 frames per second. This means that you can strike a drum but not hear the response until 30 milliseconds later,which will certainly be felt by the user. In theory, it is possible for the software to predict ahead of time when a hit will occur but I’m not aware of any software that does this.
OLD: How do you see this idea becoming a product?
Richard: Unfortunately for me, I lack the resources to turn this into a commercial product. Fortunately for drummers everywhere, this means my current plan is to release the software for free when it’s finished, along with DIY instructions for making the necessary modifications to the drumsticks. I definitely don’t want to let this project die before I get it into the hands of others. I have already been contacted by a lot of drummers who are very interested in it, including a wheelchair user from the Netherlands who thinks it could help him play drums, and a professional Latin percussionist who was wondering about the system’s adaptability to other percussive instruments.
OLD: When is your next demo being posted? What are you planning for future events?
Richard: I am putting the finishing touches on the new version at the moment so I should have some new videos online in a couple of weeks. Some people who contacted me have been wary about the system’s ability to handle fast double strokes with no physical surface to strike so this is something I will specifically demo in the next video. Of course, if you prefer to hit real surfaces instead of thin air there is nothing stopping you from doing so. You can build your virtual drum kit out of pillows, magazines, whatever you like!
OLD: If you could describe your project in five words, which would they be?
Richard: Cheap, noiseless, portable,programmable drumming.
