Sensing the remote world
21 April 2009
Much has been written about the SixthSense gadget, but how does it really work?
Kriti Kohli, the new technical editor at EPD, has written this week’s comment.
How many times have you wished you had google or wikipedia at your fingertips? Well, that may literally be possible if Pattie Maes’ Media Lab at MIT has anything to do with it. Developing what they call a ‘wearable gestural interface’, Pattie Maes and her graduate students (read slaves) have come up with the next step in human-machine interface: SixthSense.
The Sixth Sense gadget augments the user’s reality by allowing natural gestures to access and interact with digital information. Our actions and decisions are largely based on our interactions with the objects and the environment around us. But the most useful information that can help us choose the right course of action is often unavailable at our fingertips. With the advances in miniaturisation and networking capabilities, computing power has been placed in our mobile phones and laptops, but digital information is confined within one realm, unable to cross over to the physical world. So far this sounds like metaphysical philosophical wishful thinking. So what exactly is it that the SixthSense enables the user to do?
Comprising a projector, mirror and camera, the prototype is currently a bulky and awkward looking device that hangs around your neck. Plastic marker caps to be worn on the fingers don’t exactly make you look uber-cool either. The projector and camera interface with a mobile computing device, possibly a mobile phone, in the user’s pocket where the software program processes the video stream captured by the camera and tracks the locations of the plastic markers for hand-gesture recognition using “simple computer-vision techniques.” The projector enables visual information to be spread across any surface.
Imagine shopping at a bookstore, picking up a book you like that you think may be too pricey, you use your little SixthSense gadget and realise you can get it for half-price on Amazon. Hands-free and instantaneous, what more could you ask for? Maybe you’re at the supermarket and want to know the green credentials of branded cornflakes versus the supermarket’s own brand, you bring the object within the camera’s view, and the company’s corporate social responsibility is projected onto the box.
What is innovative about the SixthSense is the interaction it provides with the physical environment using natural gestures tracked by the funny-looking plastic marker caps also known as visual tracking fiducials. Based on the movements of these fiducials, the gestures are interpreted into interaction instructions for the interface. Whilst it could be argued that this is similar to what Microsoft Surface does, it seems the SixthSense can work on any surface. For example, the drawing application allows the user to draw on any surface by tracking index finger movements. Some gestures are pre-programmed such as taking a picture with the camera when the user makes a “framing” gesture. Navigating a map uses gestures similar to the ones for the i-phone; and the gadget also allows users to draw symbols in the air, such as the @ symbol to check e-mail. Is anyone else thinking Minority Report?
Seamless integration is a much bandied-about buzzword. But here we have a working implementation of a demonstrable device that frees information from its confines and brings in out into the tangible world. But what ultimately decides the fate of these new gadgets is whether it is “cool enough.” Take the Segway for example, great idea, but people feel silly riding around on it. The real question is whether there will be an uptake of this technology or whether people will simply feel too ridiculous to have a projector-camera hanging around their neck and have coloured bits of plastic on their fingers.
Contact Details and Archive...
Most Viewed Articles...