In collaboration with Hrvoje Benko, we invented a set of cross-dimensional interaction techniques for a hybrid user interface that integrates existing 2D and 3D visualization and interaction devices. Our approach was built around one and two-handed gestures that supported the seamless transition of data between co-located 2D and 3D contexts. Our testbed environment combined a 2D multi-user, multi-touch, projection surface with 3D head-tracked, see-through, head-worn displays and 3D tracked gloves to form a multi-display augmented reality. We addressed some of the ways in which we can interact with private data in a collaborative, heterogeneous workspace. We also reported on a pilot usability study to evaluate the effectiveness and ease of use of the cross-dimensional interactions.

In collaboration with Hrvoje Benko, we created VITA (Visual Interaction Tool for Archaeology), an experimental collaborative mixed reality system for offsite visualization of an archaeological dig. The system allowed multiple users to visualize the dig site in a mixed reality environment in which tracked, see-through, head-worn displays are combined with a multi-user, multi-touch, projected table surface, a large screen display, and tracked hand-held displays. We focused on augmenting existing archaeological analysis methods with new ways to organize, visualize, and combine the standard 2D information available from an excavation (drawings, pictures, and notes) with textured, laser rangescanned 3D models of objects and the site itself. Users could combine speech, touch, and 3D hand gestures to interact multimodally with the environment. To verify our work, we conducted preliminary user tests with archaeology researchers and students.