E x p a n d e d   O m n i d i r e c t i o n a l   P a n o r a m a s

New ways to use and represent 360° panoramas


INTRODUCTION

Traditional computer graphics, as we know, was enterely designed to produce visualizations for static flat-panel. So the hardware was developed and specialized for this purpose. Nowadays we are experiencing a deep change in several aspects: on one hand, immersive visualization devices such Oculus Rift, HTC Vive, Samsung Gear VR and mobile screens in general are allowing visualization experiences impossible to reach on traditional displays. Furthermore, we also noticed an increasing use of real captured data in both modeling and content production processes.

This new immersive displays trends makes possible to create new, richer and unique experiences for each user; and allows to explore properties inherent to three-dimensional visualization, such as: view dependent lighting effects, and parallax effects, that enrich the user experience.

During the last years, we have been analyzing and exploring traditional representation and introducing new elements to produce better representations, capable to expand and enrich the uses of panoramas uses and applications computer graphics creation process. Part of the research was developed during Aldo Zang's doctorate and can be found in his thesis: "Expanded Omnidirectional Panoramas".

We started working on the problem of rendering using illumination maps and we studied out ways to do this efficiently. We also studied the problem of photorealistic rendering scenes of mixed reality using real illumination captured in HDR panoramas. Noticing the limitations of traditional panoramas in the context of mixed-reality rendering, we expanded the classical RGB representation to RGB-D representation introducing a spatial component that allows new uses and applications.


Benefits

  • Efficient high realism, allowed by the new representations.
  • Allows view dependent and parallax effects, both for conventional devices and new immersive platforms.
  • Achieve more realistic illumination effects, using both efficient light sampling schemes, and real captured data structured into new representations, which allows more accurate calculations.
  • New panoramas representations are more powerful to handle with mixed reality rendering.
  • The new structures reduces substantially storage and data redundancy, which is (particularly) especially desired for real time applications.


Research Topics

  • Analysis and compression of plenoptic data into multi layered panoramas.
  • Mesh extraction and simplification from real RGB-D data.
  • Real time visualization of multi layered panoramas.
  • Use of multi layered panoramas for illumination of mixed reality scenes.
  • Multi layered panoramas, with backside layers.


Disclaimer

The proposed representation for multi layered panoramas is well suited for a neighborhood of the capture point, which is sufficient for the kind of applications we are interested on. Besides, head mounted displays, in general were designed to be used with limited head movements and without spatial user displacements. So, we are concern only with parallax and view dependent illumination effects due to movements inside a neighborhood. In the future we intend study navigation between a set of multi layered panoramas.


Research

This project has been developed during Aldo Zang's doctorate. In addition to multi layered panoramas, Zang's thesis also addresses other research problems related to expanded panoramas en photorealistic rendering.


PROJECTS