Since the introduction of high-definition LCD televisions, large high-resolution displays have become reasonably inexpensive, and TVs with over two million pixels and sizes of 50 inches and larger are common in the home. But despite these devices' large sizes and sharp images, the realness of the image shown on a TV falls far short of even the simple experience of looking out a window.

Why? First, we have two eyes, which normally receive slightly different images, allowing us to judge distances and perceive depth. Second, our ability to move about near a window allows us to "look around." We move close to the window to see more, and we move side-to-side to get a better view. This first capability, stereoscopic 3D, is now commonplace in the movie theater and an increasingly inexpensive option on new TVs. This second capability is more interesting.

At the LSU CCT, Assistant Professor Robert Kooima and his group have produced this effect by combining common, inexpensive, off-the-shelf technologies. A Panasonic 58-inch 3D plasma TV provides the stereoscopy. A Microsoft Kinect, disconnected from its X-Box 360 and connected instead to an ordinary workstation computer, provides the "look around." The Kinect is a depth camera. It records high-resolution distance measurements rather than colors. Given these distances, a software package called OpenNI extracts images of the people in the room and analyzes these images to determine the positions of their bodies, arms, legs, hands, and heads. Results from this analysis are used to compute the positions of the users' eyes, and an interactive 3D stereoscopic image from that perspective can then be rendered.

The combination of 3D imaging and user-centered perspective is greater than the sum of its parts, and the experience of using this system is surprising. The image on the display appears to be attached to the room rather than to the TV. As you move about, you perceive that you are moving around the displayed scene. The image becomes an extension of the room in which you stand, connected at the rectangular frame of the screen, rather than being a separate scene detached from reality by the display plane. This effect enhances the illusion that the displayed objects are 3D, solid, and separate from the display. Their presence and realness are astonishing.

"The applications of this technique are extremely broad," said Kooima. "Interactive scientific visualization at the LSU CCT benefits directly from the improved realness of the rendered image, and these benefits apply regardless of field or subject matter. We've applied it to astrophysical simulation visualization and X-ray tomography, and we've even used stereo rendering in our video game design and computer graphics courses."

Of course, despite this work, there are many ways in which viewing a display screen falls short of normal vision. The contrast and brightness of modern TVs cannot approach the range of light intensities that we perceive. Also, the varying focus of our eyes, which gives our brains subtle clues as to distances and sizes, is not exercised when our eyes are focused only upon the plane of the TV. These issues, and others, will be addressed in the future as display technology improves, and researchers at CCT will be taking the lead in applying these advances to scientific research and education.

Publish Date: