WISE PEOPLE, S.L.U. is a company with registered offices in Las Palmas de Gran Canaria, founded in that city on 23 August 2017, under the name “Tricontinental Hotel Lab, S.L.”, which was modified to the current “Wise People, S.L.U.” by agreement adopted on 9 February 2023.

In recent years, this entity has been carrying out, among others, activities in the field of tourism, linking this activity to education and university training, for which it signed a collaboration agreement with the University of Las Palmas de Gran Canaria (ULPGC) and the INNOVA GRAN CANARIA business cluster for the creation of the TRAVEL TECH SCHOOL (TTS), a space located on the ULPGC campus that fosters creativity, innovation and technological training in the field of tourism and universities, with the aim of becoming a benchmark for tourism activity in the Canary Islands. The training, research and innovation initiatives developed in the TTS will be aimed at the business and professional sector.

Our company has a technological character. Its current assets are solutions under development as a result of R&D investments made in collaboration between partner companies. In the context of virtual experiences, photorealism plays a crucial role in the creation of environments and objects that seem real. This entity plays an important role in this field, as we provide the necessary tools to create these photorealistic virtual experiences.

The overall goal of this project is to create a plugin for Unreal Engine 5 capable of rendering NeRF volumes generated from real content on virtual devices.

Firstly, the aim is to analyse the architecture of Unreal Engine 5 in order to make code-level modifications to its rendering engine. This would allow loading, in a visual way, models generated by NeRF that represent, through a spatial volume, environments and/or three-dimensional elements without the need for a polygonal mesh. This integration would be carried out in a modular way, allowing it to be incorporated in addition to the main graphics engine without compromising any of its internal services or interfering with its correct functioning. This modification of the code in the form of a plugin will be carried out using the engine’s native programming language, C++. Additionally, the use of additional programming tools such as nVidia CUDA will be explored to increase computational performance when rendering NeRF volumes in real time within Unreal Engine 5.

Secondly, it is intended to create a 3D design pilot for Metaverso for multi-modal European multi-destination travel. This pilot will make use of the previously developed plugin to visualise NeRF models within a photorealistic virtual reality experience, and within this the different destinations can be selected in the form of journeys (with the structure of a travel package) to be visited and experienced.

To achieve this objective, the following general and specific objectives have been defined:

GO1. To develop a plugin for Unreal Engine 5 capable of displaying, in real time, photorealistic three-dimensional volumes created from NeRF neural models.

GO2. Optimise this plugin so that its operation in virtual reality is viable and stable, providing a comfortable experience for any user using a virtual reality device similar to the one used in this project.

GO3. Create a multimodal travel demonstrator pilot using the developed plugin to display the travel environments from NeRF volumes obtained from photographs or videos.

EO1.Conduct a thorough investigation of the state of the art to select the most suitable NeRF variant for the project.

EO2. Analyse the rendering code of Unreal Engine 5 in order to inject a visual process capable of displaying NeRF neural models in real time.

EO3.Develop a module (plugin) for Unreal Engine 5 capable of displaying, in real time, three-dimensional volumes created from the result of the creation of NeRF neural models in virtual reality helmets.

EO4.Design a European multimodal travel experience, with several different destinations to experience.

EO5.Develop and implement a virtual European multimodal travel experience within Unreal Engine 5, using the plugin developed in the previous specific objectives.

EO6. Analyse the data generated, the functioning of the plugin, and validate the results.

Expected results:

The expected results of this project consist of the development of a plugin for Unreal Engine 5 capable of visualising three-dimensional volumes created from NeRF models, and its use in a pilot multimodal travel experience within virtual environments created using these models.

• NeRF Plugin for Unreal Engine 5: Modular plug-in that, when added to the Unreal Engine 5 graphics engine, allows the visualization of computational models generated using Neural Radiance Fields (NeRF), as well as their inclusion in hybrid scenes with other non-NeRF 3D elements.
• Multimodal Journey Pilot: A system based on the metaverse paradigm in which a multimodal journey through various environments in Europe can be configured and visualised by means of a virtual reality device. These environments will be generated by NeRF models created from sets of images and/or videos.

This project makes major contributions in terms of research into technological trends, the creation of photorealistic content and its visualisation. Regarding the former, by creating a module that makes the real-time visualisation of NeRF neural models compatible with a highly democratised technology such as Unreal Engine 5, not only the representation and creation of photorealistic content is improved, but also its integration in all types of digital projects in a simple and intuitive way. In terms of content creation, the use of NeRF models opens the door to a new generation of content with high graphic detail in which hyperrealism is left behind to give way to a 100% accurate representation of reality, which can be easily captured from photographs or videos that are interpreted by NeRF to generate a neural model capable of representing this reality.