The interview: Using the Unreal Engine at AVSimulation

An interview with Valerie DOGNIN Product Owner, and Geoffroy Lacarrière, Development Design Engineer C++ at AVSimulation

Can you explain to me what unreal is?

G. Lacarriere: Unreal is primarily a 3D engine, but it is also a number of tools around this 3D engine that facilitate the creation of 3D applications, at any level; retrieve graphical elements and process the optimizations that will then be displayed on the screen.

Do you use Unreal at AVSimulation ?

G. Lacarriere : Today, yes. Before using Unreal at AVSimulation we used an engine called OpenSceneGraph. It is an old engine that is beginning to become completely obsolete, which limits us on many topics on which we would like to evolve. This is what takes a lot of time, energy, and development to do things that are not native to the OSG engine but are almost native to Unreal. OSG has accumulated a technological gap that, as the years go by, will never catch up with the current state of advances in 3D, photorealism…

The delay in the advance of the OSG engine triggered your need for engine change?

G.Lacarriere : Indeed, our objective is to arrive at the most photorealistic 3D image possible, both in terms of visual rendering, for simulator users, but also on the whole notion of sensor to be the most representative.

What is your ambition for the futur ? 

V. Dognin: Our ambition is to develop a unique and comprehensive toolchain that will be used by Research & Innovation teams and by the engineers all along the V-Cycle. The objective is to accelerate the development of Autonomous Connected Vehicles, whatever the form factor of the test systems: simple workstation, massive simulation in the cloud, driving simulators, HIL / VIL benches.

When you started looking for a new engine. What made you choose Unreal and not another engine?

G. Lacarriere: We made a comparison of the existing engines that would offer more advantages to our users in the 3D industry. Unreal came in the top three of the engines we selected. It had an advantage over other engines of the Unity type: It was coded in C++ and open not the others.

We had both a technical side and a practical side, in the sense that our development teams know about C++ and the open source aspect. This knowledge is of interest to our client because our developments are more stable and fast. We were reassured that we could change or understand how the engine is made in-house. The price of the license has also been a factor in relation to other solutions.

You are on a full use of Unreal? ?

V. Dognin : No, we are still in the phase that will last a few years. For the moment only the visual module and camera sensor are updated under Unreal. In the coming years there will be Terrain and SCANeR Studio. The long-term goal is to have transitioned all OSG to Unreal

What is currently the most challenging part of in this transition phase?

G. Lacarriere: It is the cohabitation with the old engine, for the moment we cannot change from one to the other. We have a whole ecosystem that is still very dependent on the old engine, which requires a cohabitation of the two engines during the transition. This creates constraints. One of the constraints concerns the 3D assets that are built and formatted for the use of the OSG which will have a different formalism for Unreal in order to take full advantage of the performance of the engine. Unreal technology cannot yet be used everywhere, we are obliged to restrict it a little while, for the moment there is a cohabitation of the engines the time of the transition

Can you run any simulations on the Unreal engine ?

G. Lacarriere: Yes, all current driving or other scenarios are compatible. The visual works, the cameras, the sensors work, but for now, for the Unreal part, it stops there. We have warping-static, warping-dynamic is being integrated. There are a lot of little things like that. But yes, it is possible to run a simulation for the need for example a visual rendering with some . In the expectation of being 100% isofunctional with the old visual we recommend using the Unreal visual as a preview. But we already cover more than 80% of the functional scope on this new Unreal visual.

From what you can see and read Unreal is much more than a 3D engine. It’s a development platform. Do you intend to use Unreal to develop your own sensors, cameras, and if so, what would be the advantage?

G. Lacarriere : Unreal contains many tools for managing and analyzing 3D, light, colorimetry, visual effects, rendering passes, optimizers, profiling… and others that will be useful to any new development.

V. Dognin : We are not suppliers of sensors, our goal is not to make the sensors ourselves, but to provide users with the possibility to test the sensors, to have the right configuration parameters that will represent their sensors. It is likely that we offer examples so that users can get a sense of the rendering.

G. Lacarriere : In the 3D engine, there is the goal of displaying a realistic image. The camera sensor will analyze this image which will allow him to have a more Realistic image. For the development of the sensor, it will be more interesting to have more details to analyze. Possibly more weather conditions, disturbance effects, which is very interesting for the development of a sensor that would not have been possible or less possible with the old engine.

In version 2021.1 de SCANeR of SCANeR did you do everything under Unreal ?

G. Lacarriere : No, unfortunately, this is not yet ISO functional. But the goal is to be a complete feature on the equivalent of the old visual and also to propose future improvements and other evolutions that the old engine did not allow.

Can users use Unreal in a simulator ?

 G. Lacarriere : when it’s a simple simulator, using static Warping, yes. But in a simulator with motion no.

When do you think it will be available?

V. Dognin : The warping-dynamic will be for the next release, the 2022.1 of SCANeR, once this is done, we will be able to simulate on bigger simulators.

You mentioned that the headlights function is, for the moment, limited under Unreal. Are you planning to evolve it? If so, when and what would be the benefits for users?

V. Dognin : Ideally, we would like to do that this year, but I don’t think we will have time to finalize everything. To have all the features of SCANeR today ported to Unreal it will be rather next year.

What is the process to update headlight in Unreal? What constraints does this entail?

V. Dognin : Today, we have different tools to configure Headlights, which is not simple. Indeed, it takes a lot of work, and it’s this whole business layer that we have to add to Unreal, in order to have both the quality of rendering, but not also lose all the business knowledge that we have acquired up to now, and for 20 years. The challenge is to bring this whole business layer, into what Unreal is already doing. There are a lot of questions about what already exists in Unreal, what we have already acquired, what we could exploit, what we need to do again.

G. Lacarriere :Today, we need to set the visual to be in headlight mode to enable many more calculation options, light and others.That’s what we’re trying to get native to the Unreal engine. The development of the headlights will allow us to be closer to the realistic rendering without necessarily needing a particular mode. That is, the Headlight mode, eventually, will be the default rendering mode.

Did you encounter any constraints during the Unreal engine transition with the old one?

G. Lacarriere : At the beginning, yes, there were constraints, which the old engine imposed which we could not get away from and which hindered our development. But little by little, we broke the locks, to develop tools that made it possible to abstain from this kind of constraint. In particular on everything concerning the 3D models of vehicles that are available, failing to have graphic designers on hand to be able to make modifications or updates of the 3D models. Another constraint, the need for a realistic rendering in Unreal, and for that we needed graphical improvements. Which we didn’t need in the old engine because it didn’t handle certain effects.

To avoid having a new 3D engine displaying a visual that seems to be already dated, we had to have an update layer. It was necessary to develop small tricks to use the old 3D models to put them in high definition.

Do SCANeR 2021.1 customers have a choice between using the old model and Unreal? Or is this imposed by the software?

V. Dognin : For now, both are still present in 2021.1 and this will probably still be the case for 2021.2. Until we have an ISO functionality between our old engine and the new one, we will not remove the old one. We do not want to penalize users in our transformation choices.

G. Lacarriere :The Unreal visual is shown as a preview.That is, SCANeR users continue to use the current visual, but already have a preview of what the next visual will be.

How does it work for customers who decide to work on Unreal, are the scenes automatically optimized or does a job have to be done to make them photo-realistic?

G. Lacarriere : So I would say both, we have an automatic pass, which will try to improve things. After that, it’s not magic either. It really depends on the source data. It also depends on how the infographic was made and whether the naming rules were respected.

What will happen is that we will try to recognize elements, and when we recognize them, we will improve them with a version, a new generation. For the automatic part, if this detection does not work, we manage to display the element in the state but without the improvements. We give advanced users the opportunity to come back with the Unreal editor to improve their assets and integrate the new ones if they wish and thus be able to display all that as if it were the original.

A word of conclusion?

G. Lacarriere : A keyword on the 3D engine, which is the reason why we are conditioned to this new generation: PBR « Physically based rendering », is the main reason for engine changes, technology and generation change. Before, there was a way to calculate 3D that was not realistic and that used to use certain parameters to try to make it more credible. With PBR, there is a paradigm that has changed, we decided that to arrive at photorealism, we had to base ourselves on physical values. Based on reality to simulate reality. That’s what makes it possible to have this really different aspect in 3D.