Over the years, technologists have been providing easier means of interfacing the "normal" world with the digital world. We've seen bananas, stairs and so many other everyday items even running water from a tap producing music notes. With advancements in technology which resulted in miniaturisation of digital devices, we've never been able to link the digital devices and our interactions with the physical world. Information is confined traditionally on paper or digitally on a screen.

As humans we've evolved millions of years to sense the world around us. When we encounter something, someone or some place, we use our five natural senses to perceive information about it; that information helps us make decisions and chose the right actions to take. But arguably the most useful information that can help us make the right decision is not naturally perceivable with our five senses, namely the data, information and knowledge that mankind has accumulated about everything and which is increasingly all available online.

SixthSense bridges this gap, bringing intangible, digital information out into the tangible world, and allowing us to interact with this information via natural hand gestures. ‘SixthSense’ frees information from its confines by seamlessly integrating it with reality, and thus making the entire world your computer.

It was developed by Pranav Mistry, who comes from Palanpur, a small town of northern Gujarat, India. He graduated as Masters of Design from IIT Bombay and Masters of Media Arts and Sciences from MIT Media Lab. Before joining Media Lab, he worked with Microsoft as a UX Researcher. At present, he's a PhD student and Researcher at MIT Media Lab.

The device is pretty simple, comprising of a pocket projector, a mirror and a camera. The hardware components are coupled in a pendant like mobile wearable device. Both the projector and the camera are connected to the mobile computing device in the user’s pocket. The projector projects visual information enabling surfaces, walls and physical objects around us to be used as interfaces; while the camera recognizes and tracks user's hand gestures and physical objects using computer-vision based techniques. The software program processes the video stream data captured by the camera and tracks the locations of the colored markers (visual tracking fiducials) at the tip of the user’s fingers using simple computer-vision techniques. The movements and arrangements of these fiducials are interpreted into gestures that act as interaction instructions for the projected application interfaces. The maximum number of tracked fingers is only constrained by the number of unique fiducials, thus SixthSense also supports multi-touch and multi-user interaction.

The SixthSense prototype implements several applications that demonstrate the usefulness, viability and flexibility of the system. For example, a newspaper can show live video news or dynamic information can be provided on a regular piece of paper. The gesture of drawing a circle on the user’s wrist projects an analog watch.

http://ember365.com/e365/index.php/geek/item/sixth-sense-technology-anywhere-anyhow