TapSense is an enhancement to conventional interactive surfaces that allows not only localization of inputs, but also provides information about the object coming into contact with it. This information is derived by segmenting and classifying the sounds resulting from the impact of the object with the surface.

TapSense takes advantage of the fact that different materials have unique acoustic signatures, allowing for different tools and tangibles to be recognized. Additionally, the diverse anatomy of our fingers allows different parts to be recognized – including the tip, pad, nail and knuckle - opening several new interaction opportunities for finger touch input.

This work has seen press coverage from many sources including Wired, TechCrunch, Engadget, and MIT Techology Review, among others.

The technology is currently being commercialized as a part of a company I co-founded, Qeexo. The technology has been renamed "FingerSense", the video to the right gives a detailed overview of this promising technology.

Commercialization

Qeexo licenses FingerSense technology to OEMs. As of December 2015, the company is partnering with Chinese giants Huawei and Alibaba. FingerSense has shipped on multiple Huawei phone models, including flagship models like the P8 and Honor 7 (example commercials that include FingerSense). The company now has about 15 employees, has raised over $4M in funding, and has three offices worldwide.