The goal is create systems for drawing annotations and research methods for pixel-based graphics from and for an existing database (ar:searchbox/mediaTUM) using a multi-touch table. The results will be front-end applications for the Microsoft Surface (Samsung SUR40) and back-end components for image analysis with an interface connecting various processing algorithms with the “ar:searchbox”. For the execution of graphical queries a pen-based application (TouchTect) will be implemented for sketching a schematic floor plan on a multi-touch table. The pattern recognition translates the drawing into a query that can be compared to an existing database. These search results from the ar:searchbox can be viewed via a gesture controlled interface.

To accomplish this search a tool to identify documents is required. This is achieved by embedding semantic fingerprints of the buildings to each document in the ar:searchbox. A second multi-touch application (surfID) allows the creation and manipulation of these automatically generated fingerprints. To support the creation of a fingerprint, the raster image of the floor plan is analyzed and the position coordinates for each rooms are identified, depend upon the image quality. In the future, the modular system can be extended by components and plug-ins to operation with other applications, such as the game engine Quest3D.