Modeling and Simulation of Eye Illumination in Eye-tracked Head-Mounted Display

This project aims at the optimization of illumination schemes for the development of a robust eye tracking system. Based on the Arizona Eye Model, we have been modeling a complete eye illumination and imaging system which includes the modeling of the optics of the human eye, IR illumination methods, and an eye image capture system. Using this simulation, we are able to study how to optimize illumination schemes to achieve good quality of IR-illuminated eye images for the development of robust eye tracking methods. In particular, we are able to study how the quality of eye images would change as eye rotates.

Eye Model in LightTools

Simulated Eye Features in LightTools

Optimizing Illumination Schemes of a Head Mounted Display with Eye-Tracking Capabilities

Head-mounted display (HMD) find a variety of applications in the field of 3D virtual and augmented environments, 3D scientific visualization, and wearable computers. While typical HMDs use head-pose methods to approximate line-of-sight, an alternative is to incorporate eye-tracking capability technologies into the HMD design, which leads to possibilities to reduce HMD trade-offs as well as add functionality. Additionally, by using a low-level design approach, other system optimizations for eye-tracking HMDs (ET-HMD) can be explored.

One such possible optimization can be performed upon the illumination pattern incident upon the eye. Eye-illuminating LEDs are mounted in a fixed configuration onto the ET-HMD platform, and the configuration can be selected such that the incident illumination leads to eye-images suitable to aid eye-tracking accuracy and robustness.

Our approach to identify an optimal illumination scheme utilizes a simulated eye-tracking model to capture and analyze simulated eye-images. The eye-tracking model consists of all elements in a physical eye-tracker: illumination sources, human eye structure, and imaging system. Figures 1 and 2 illustrate a simulated eye imaging system constructed in LightTools and resulting simulated eye image, respectively. The illumination schemes are controlled by selecting appropriate LED position parameters and are paired with an associated simulated eye image.

Figure 2 (missing)

Illumination pattern input control for the simulated model is given to what has been coined the “illumination generator,” and allows a method to translate general user-defined illumination functions to system specific parameters. The illumination generator is constructed such that the user defines a desired irradiance distribution to be incident upon a plane normal to the cornea vertex, and then compares the distribution against a database of irradiance patterns created by the numerous LED arrangements to select the closest match. As an example, a Gaussian user-defined function with a 60% irradiance falloff at the corners is matched to the irradiance pattern created by LEDs located 35mm from the cornea, 20mm above the optical axis, and 25 degrees from parallel to the optical axis, as shown in figure 3.

The simulated eye image created by the user-defined illumination is quantitatively scored by an “illumination evaluator” that compares the eye image to a set of evaluation masks. These masks are constructed according to the features important to the eye-tracking algorithm. When the pupil is being tracked, for example, the image mask is in the form of figure 4. This mask has uniform pupil and non-pupil areas, and emphasizes the sharp boundary between the two. The score comparison performed by the illumination evaluator identify the optimized user-defined illumination function.

This process is a foundation for using non-sequential ray-tracing software to optimize illumination schemes for ET-HMD systems. A more detailed approach overview can be found here [Link to paper]. Ongoing research includes improving this optimization system and validating simulation results in a physical setup. This work is supported by National Science Foundation Grants IIS-03-07227/04-11578 and 03-07189.

Reference

1. Hong Hua, C. Pansing, and J. P. Rolland, “Modeling of an eye-imaging system for optimizing illumination schemes in an eye-tracked head-mounted display,” Applied Optics, 46(32): 1-14, November 2007.