Remote Expermentations of Artificial Intelligence in 3d Virtual World

Abstract

Visual Artificial Intelligence (AI) research has experienced significant advancements in recent years. To benefit from gathering a vast quantity of data across multiple conditions and environments. However, the task of gathering such data requires significant time and labor. In addition, the development and testing of visual AI algorithms aimed at multi-sensory models can be both costly and, in certain instances, pose potential risks in the real world. A 3D environment simulator that employs a view synthesis module to generate photo-realistic simulations and enables the adaptable configuration of multimodal sensors is designed to tackle these specific challenges. To enhance the capabilities of the view synthesis module, we integrate innovative techniques such as adaptive view selection, depth refinement, and layered rendering, with the objective of producing highly realistic imagery. This suggests that PreSim encompasses multiple advantages: (i) by showcasing a photo-realistic 3D environment, it facilitates the seamless integration of multisensory models within the virtual realm and enables these models to perceive and navigate scenes, (ii) Through the incorporation of an internal view synthesis module, PreSim facilitates the transfer of algorithms developed and tested in simulation onto physical platforms, eliminating the need for domain adaptation. (iii) Moreover, PreSim has the capacity to generate ample data for vision-based applications, encompassing object pose estimation and depth estimation.