Tangible User Interfaces

A typical virtual environment setup consists of a workbench display allowing an outside-in exocentric view of a data set, or a room display for fully immersive, inside-out egocentric views of life-sized, walkthrough data sets. Through SCAPE, we have explored a framework that combines and promotes both these interaction paradigms symmetrically and concurrently. However, often times the two discrete views provided by the workbench and wall displays alone are insufficient. For example, in the immersive view, the data set might appear overwhelmingly detailed or zoomed in, whereas on the workbench, the same visualization might appear at an inadequately low resolution, or otherwise look too zoomed out to distinguish notable characteristics. In such instances, a continuous range of intermediary scales and levels of detail is highly desirable to fine-tune the visualization for the given task.

We recently are exploring tangible Magic Lens interfaces in 3D augmented and virtual environments, to augment the bi-modal (micro- and macro-scales, exocentric and egocentric perspectives) visualization concept in SCAPE.

Hand-held magic lens

It allows zooming, clipping, and layering a 3D object selected by free-hand gesture. This was upgraded from our preliminary “Magnifier” implementation (UIST 2003 paper). Compared to the 2003 implementation which only had magnification function, we identified  multiple generic functions we consider applicable to many domains (i.e. zooming, clipping, and layering), implemented interface controls. Besides functional upgrades, we also revised the rendering algorithm for more accurate registration and depth perception as well as easier user adaptation.

Device Construction

Zooming

Clipping

Layering

Help Menu

Embedded Magic Lens

Allows zooming, clipping, and layering a 3D object selected by a physical prop. This new device design has similar functionality to the hand-held device, but it provides a significantly larger inset window and hand-free interface. While the embedded nature is not as intuitive as the hand-held (user has to make explicit selection using a physical prop placed on the micro-view), its larger window and  hand-free nature provides superior registration quality and may find advantages in some applications.

Device Construction

Zooming

Layering

Help Menu

Function Modes

We recently are exploring an application of the SCAPE concept and the Magic Lens interface in the medical training domain. SCAPE’s multi-scale and multi-perspective visualization capability is anticipated to provide a better learning tool and even a diagnosis tool for medical practitioners. This research is in its very infancy stage and comments are welcomed.

Conceptual Simulation

The Setup (patient on the bench)

Seeing the Patient’ Skeleton

Overlaying the Patient’s Intestine

Zooming to the Patient’s Intestine

Related Publications

1. Hong Hua, “Merging the worlds of atoms and bits: augmented virtual environments,” Optics and Photonics News, 17(10), 26-33, October 2006 (cover story).
2. L. Brown and Hong Hua, “Magic Lenses for augmented virtual environments,” IEEE Computer Graphics and Applications, 26(4), pp.64-73, July/August 2006.
3. Ji-Young Oh and Hong Hua, “User evaluations on form factors of tangible magic lenses,” Proceedings of 2006 IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR’2006), pp.23-32, October 2006 (29% acceptance).
4. Hong Hua, L. Brown, & C. Gao, “System and interface framework for SCAPE as a collaborative infrastructure,” in Presence: Teleoperators and Virtual Environments, 13(2), 234-250, April 2004.
5. Hong Hua, L. Brown, C. Gao, “SCAPE: Supporting Stereoscopic Collaboration in Augmented and Projective Environments,” in IEEE Computer Graphics and Application, 24(1), 66-75, January/February issue, 2004.
6. L. Brown, Hong Hua, and C. Gao, “A Widget Framework for Augmented Interaction in SCAPE,” in ACM Proceedings of User Interface of Software and Technology 2003 (UIST 2003), also in CHI Letters, 5(2), 1-10 (20% acceptance out of 116 submissions).
7. Hong Hua, L. Brown, and C. Gao, “A new collaborative infrastructure: SCAPE,” in Proceedings of IEEE VR 2003 Los Angeles, CA, March 2003, pp. 171-179 (28% acceptance out of 103 submissions) (Received Honorable Mention Award in IEEE VR 2003).