Foveated Imaging System (FIS)
In the process of creating a mixed- or augmented reality application, it is imperative to capture and interpret the physical world, especially as the world becomes more dynamic or complex. Image acquisition systems are an essential component in the loop of capturing a real-world scene and augmenting it with synthetic information. The capability of acquiring high-resolution, wide field of view and high dynamic range images at video rate is highly desired. Such capabilities are also critical for many vision-based applications such as surveillance and robot navigation. The field of view, resolution, and depth of field of a conventional digital camera, however, are limited by the choice of imaging sensors and optics. We have been developing various novel imaging systems to achieve the aforementioned capabilities.
One example of such an imaging system is a foveated imaging system, which mimics the human visual system. Foveated imaging system (FIS) captures a wide visual field with a dynamically embedded, high-resolution fovea region: the peripheral sensor captures the context for target detection and tracking; the foveated sensor, with a resolution many magnitudes greater than the peripheral sensor, captures the fine details for target recognition and detail examination. A scanning and tracking mechanism enable the active foveation of the attended area of interest (A-AOI) with high angular resolution across the entire visual field. This acquisition method not only serves as an input for the fovea-contingent display research discussed earlier, but can also find a wide range of applications in surveillance, robot navigation, and microscopic imaging. We analyzed the design algorithm for maximum bandwidth savings by using multiple imaging sensors and implement a prototype for future research and experimental developments.
-
- Figure 1 FIS conceptual design
-
- Figure 2 Simulated foveated images – Resolution: 1 a.u. at foveated FOV, 0.33 a.u. at peripheral FOV
-
- Figure 2 Simulated foveated images – Resolution: 1 a.u. at foveated FOV, 0.33 a.u. at peripheral FOV
-
- Figure 3 Bandwidth savings and optimal ratio of foveated FOV/overall FOV vs. overal FOV
-
- Figure 4 Photograph of the 2D scanning MEMS chip
-
- Figure 5 FIS prototype. Inset: Pupil conjugation on the MEMS chip
-
- Figure 6 Distortion simulations
-
- Figure 6 Distortion simulations
-
- Figure 7 Experimental results
Reference
- Hong Hua and Sheng Liu, “A dual-sensor foveated imaging system,” Applied Optics, 47(3): 317-27, January 2008.
- Sheng Liu, C. Pansing, and Hong Hua, “Design of a foveated imaging system using a two-axis MEMS mirror,” Proceedings of 2006 International Optical Design Conference, Vol. 6342, 63422W-1-W-8, Vancouver, Canada, June 2006.