Theory of achieving Photo-realism in Unity 5 !

Hi everyone, I was experimenting on creating photo realistic results from a game engine and have been looking at it deeply, especially since Unity 5 came out with the GI support. I will be going through all the possible options in Unity 5 in achieving great looking results 🙁 A quick note that all the opinions and workflow below are my personal opinion, kindly let me know if anything more can be added to this ) .We will consider a simple scene and explore and make it look beautiful. I will also be explaining on how this could have been done in a Pre-rendered VFX scene. The scene we will be working on is a interior of a room with a character sitting.

To start off, in order to get great looking results ( Lets not consider programming here ), the three most important factors are : 1) Cinematic Lighting 2) Great models and 3)Excellent shaders for the models. We will go through each of them in detail. First we need to understand the technical terms and then we will go into the scene. Along with achieving photo realism we will be looking at ways to optimize the scene and make it run at the fastest speeds possible.

1)Cinematic lighting : Lighting is extremely important in giving out great results, ever since humans ventured into making art, they observed the real world a lot and made a number of categories in lights that would enable us to create anything that is needed. Since we will going only into unity 5, there are basically 6 types of lights you can implement in Unity. They are 1)Point lights 2)Area lights 3)Spot Lights 4)Directional light 5)Emissive surfaces 6) Ambient light.

In all the above lights there are defined by 3 things, their color , their intensity and their direction. Now in all the above lights the color and intensity is common to all of them, but the direction of light changes in all the them and hence defines them. So,its the direction of light rays that make a light what it is ! And remember that all lights will fade with distance, this effect is little less seen in sun light ( Usually directional light because of the large size of the sun) and will usually need to be incorporated in the rest of the lights.

Assuming all of us here know what each lights is, let us see examples of where each light is used in real life.

Point lights : Bed lamps, candles , fire bugs , Radio Towers , A bulb

Spot lights : Torch, car lights, All cinematic lights without diffusion, led lights, spot lights in cinema hall.

Directional lights :  Sun (As of now real-time GI is supported only for this )

Area lights : Computer screens, Fluroscent tube lights , interior lighting  (Its important to note that these lights take lot of CPU power and hence they can only be baked to light maps)

*Light map – A light map is a pre-calculated lighting information that is added to the textures before the game starts, doing this will help save the compute power as lighting calculations are expensive to perform in real time. Area lights work like hundreds of points lights at one place ,and hence the light is averaged, which is why area lights need lots of computations.

(A quick note you have to know is, when you add a directional light in a scene, it takes up the whole look of the scene. And when you add other lights on top of it, they don’t matter much. In a real world film shoots, cinematographers often light their subjects with artificial light than depending on sun light all the time. Which is why they build sets in enclosed spaces and arrange lighting however they want. In order to do the same in Unity, a directional light can only be good for environments or large areas where the user is walking around or if you want light to pass through a window and enter the room. But to get complete control its better to use the other lights effectively, the most effective of all lights with the best shadows would be area lights and if you observe this is exactly how real world cinematographers would use, they have a spot light and then have a diffuse paper to spread the light equally in an area rather than coming from single place, this makes all the shadows smooth and because you need bake your area lights as a light map, it makes a good use case especially if you don’t have dynamic objects in the scene )

Now is a good time to go bit more deep about lights, every real world man made light has a lighting profile associated with them, they are called IES lighting profile, a lighting profile will describe how light behaves when its emitted, it explains the shape formed by the light, if you google about IES light profiles Philips has a web page that shows you all the light profiles, now unfortunately Unity 5 does not have these lighting profiles and I am not sure when they would be implemented, some drawbacks of lighting profile is mainly computation time to calculate for real time lights, because a lights shape is Volumetric in nature using IES is  expensive. On the other hand Unreal Engine has IES profiles available in their light shaders. But we have a good alternative inside Unity 5 , there are called COOKIES, cookies are nothing but images with transparency on them that shape the light when being emitted. There are examples where users have created custom cookies and used effectively like in car beams, atmospheric effects , caustics and others.

We have a good idea about light sources, once the light is cast the next important phase to observe are the shadows. Shadows are so crucial in real world film making as well, shadows can show details as intended, the reason why we have ISO, Larger aperture openings is for the sake of  allowing more light so that we can see well in shadow area, in real world when we observe around us, there is nothing that has 0 black, everything has at-least some about of grey in it. Choosing proper shadows will help us create realism. In Unity we mainly have 2 options , soft shadows and hard shadows. Hard shadows are rarely used , they are not real. Soft shadows are the most ideal of all, every real world bends light in such a way that they create soft shadows all the time. So, for cinematic purposes we should always choose soft shadows. And the other setting in Unity shadows that is important is aliasing, in general 3d objects are made up of edges, its not cool if edges show up in the shadows, we always need to have smooth edges, this depends on the quality settings that we set. We have to have highest quality set for cinematic results, I will go more about the quality settings at a later point. And when compared to VFX softwares, there are 2 main options that are not available to us, they are decay rate (for this you have place your lights at proper positions ) and then Penumbra angle ( This is the rate of fall off of the shadow, since this option is not available, its better to use area lights to come close to this effect)

In shadows its important to note that direction lights can light the whole scene and hence there is a possibility that the shadow map does not get enough pixels for the close by objects , which is we have to use cascaded shadow maps in Unity, by using cascaded maps we can have multiple resolution maps for a light, the closer objects will get a higher resolution and smaller size map which is memory efficient and hence works great for real time rendering . To achieve realistic results use a higher resolution map which comes at the cost of compute power.

In Unity, lights are rendered using two techniques which are Vertex lighting and Pixel lighting, vertex lighting just interpolates light values and is not accurate, for cinematic results we have to use Pixel lighting which is calculated for each pixel. For each light , when you set it as important then pixel lighting will take place, make sure you do this to achieve fantastic results.

In terms of memory usage for rendering for lights ,we can classify as Area lights > Point lights > directional lights > Spot lights . In the real world its important to note that an object can be affected by multiple lights and hence to enable this feature we have to use deferred shading . All lights are evaluated per pixel here unlike vertex lighting . Performance can improve by keeping the volume of the lights small, that is we should not use point lights to cover a large area, they use cubic maps for shadows and pixel calculation for large point lights is expensive.

The most important topic in Unity 5 for realistic lighting is Global illumination (GI) : to start off if you observe objects around then you can see that every object has an influence on every other object, a red object close to a white wall bleeds some red color to the white, this is called Global illumination, it can caused by photons bouncing around. It is also named as indirect lighting, meaning not directly influenced by the light. If you observe VFX rendering kits like renderman, arnold or Vray all of them come with GI feature, this is one single thing that makes images look so amazing and  realistic. On a quick note we can consider lighting algorithms into two different categories. 1)Direct lighting 2)Indirect lighting  . Direct lighting will try to mimick real world lighting and will often use ray-tracing tricks to achieve close to GI kind of results. With Indirect lighting (GI) can be achieved by mainly 4 important algorithms a)Path tracking b)Photon mapping c)Stochastic progressive Photon mapping d)Bidirectional path tracking. To categorize the rendering engines,

Photo-realistic

– Any random computation methods with an artists eye thats not physically accurate

– Final Gather approach used in renders is not physically accurate.

Physcially based rendering

– Direct lighting Algorithms

—- Ray tracing (Example vendors : Arnold , vray ) (Arnold uses Monte carlo ray tracing giving good GI )

– Indirect lighting Algorithms = Global illumination = (Direct + Indirect lighting)

—- Path tracing (Octane does this )

—- Photon mapping  (Vray can do this )

—- Stochastic progressive Photon mapping

—- Bidirectional path tracking

All was all about the GI in the current industry now we will see how this works inside Unity 5. GI in Unity 5 will not be using the above algorithms and it will be more of real time GI algorithms because the above algorithms are time consuming. Unity 5 offers us 2 types of GI they are 1)Baked GI and 2)Pre-computed realtime GI. Baked GI is calculated before the game starts and stored in the texture maps , but this will work only for static objects.

I will go over the settings in the lighting tab of unity 5 to get cinematic results :

Go to the object mode and select the lights and make sure it is baked GI for extremely good results, if you have dynamic objects then use mixed GI. Next if you are working on dark rooms then its good to increase the bouse intensity.

Next select the mesh renderers and make sure light map static is selected , only then it will use GI for that object . Next if the camera is close to a object and needs more detailed GI then increse the scale in lightmap. If you have modelled in a 3d package and that have laid out proper Uvs,then make sure you select preserve UV so that Unity does not mess it up when it calculatest he new lightmap.And when a mesh is a scne is sure to be affecting another object with bounced light, then make sure important GI is turned on, by doing this Unity will not work on optimizing that particular light bounce data.

If you have a rendering engine from a VFX software that can export light maps then for particular meshes you can import those light maps rather than the ones generated by Unity. Do this if you light maps generated in a different software are much more accurate.

We will go through the scene tab in the lighting menu, this is a universal place where the over all scene settings can be changed .

If your scene has natural environment, then its best to the kybox and sun that comes with Unity. Also in nnature there is always a little bit of ambient light present, so its good to always add a bit of ambience to set the mood.

Also setting the Ambient GI to baked would be great for your scenes. If you are in a open world then do set your Reflection source for a skybox .
Reflection bousces is a important property, as the name suggests reflections are stored in the reflection probes that you set up in the scene.

If you have dynamic objects then switch on pre-computed GI . Real time resolution should be carefully set based on which platform your playing your game at. Baked resolution is pre-computed and hence since its being pre-computed keep it high if needed. Usually 10 times the real time resolution.

Do not use compressed as it can introduce artifacts ,that is not needed for photoreal rendering.
Indirect resolution is available when we disable precomputed realtime GI and this can be excellent only if we dont have dynamic objects in the scene.

Ambient occulsion is a importasnt facvtor that can show realism, increase the ratio of it , if u need it to be shown more

And switch on final gather to get even better results. It takes more time for baking.

Directional mode is important if we like to give importance for a directional source and this can give us more details diffuse normal
mapped materials, but we will need to store additiona llight maps for this .

If you feel youir scene looks dark even afetr having appropriate lights then just add Indirect intensity and it will scale the light in the whole scene.

Bounce boost will increase the amount of light bounced in the scene , recommeneded if you need light to be bounced and highlighted on
other surfaces

Add fog depending on the requirements you have in each scene.

LIGHT PROBES :
From the above discussion we see that light maps really help creating good GI if the scene is static but for non static objects the scene can look disconnected. This is where light probes will help create a similar effect for dynamic objects.

Create the light probes and place them cleaverly at places where you will need good lighting calculations for a dynamic object. Each light probe is nothing but a spherical panoramic HDR image.

In the lightmap parameters the irradiance budget can make your texel in the lightmap to be blurred or sharp. Higher values will give you a sharper image but at a higher expense .

In the baked GI parameters try increaing the antialiasing samples and also the direct light quality , which gives you softer shadows .
Increasing the Baked AO quality will also increase the quality of your results.

Mkae sure you use Directional lightmapping in your scenes as it adds on top of the normal light maps and the additional data can always give more realistic results.

Also a note that all your calculations are usually stored in a lightmap snapshot.

Make sure you use all the GI visualizations effectively to see how everything is working and see how can optimize the scene.
And one of the main things we have to be sure of linear workflow, see that the whole process in your scene from textures to the end build is using a linear workflow.