Today we will look at several examples of shots from the game and analyze the graphic techniques used in the game.
Foreword
This is an unofficial analysis of the game. I just analyzed the frame grabbing with RenderDoc . If you would like to find out information from the developers themselves, you can study the slides from the talk at SIGGRAPH by Fabian Bayer . Slides (bottom of page), video (starts at 1:58:00).
You can also read the analysis of GTA5 graphics , performed by Adrian Correzhe [ translated into Habré]. Since both RDR2 and GTA5 were created by the same company and use the same engine, some of the tricks from GTA5 are also present here.
Another important point is that I am not an experienced graphics programmer and still a beginner in this field. Therefore, a lot is incomprehensible to me. If you find bugs or things that can be improved,write to me . Let's go!
Parse the frame
Here's our main frame for analysis:
The frame was captured on a PC with medium settings.
In games like RDR2, it's almost impossible to explore all the moves in one shot. Their work is transferred between several frames. Therefore, I captured several frames, but the one shown above will be the main one. It contains many properties, including: spotlight and spotlights, directional lighting (it is very small, but it is), buildings, NPCs, horses, trees, vegetation, clouds, etc. It shows most of the rendering techniques used in the game.
RDR2 is an open world game, so data streaming is always done. Consequently, framing begins with a set of tasks such as creating and removing textures, scans of shader resources, random scans, updating descriptors, buffers, etc.
Earth / Mud Map
Dirt plays a big role. In addition to being a game mechanic, dirt makes environments more realistic. The game renders the textures of human and horse tracks into a displacement map along with the textures of the carriage wheel tracks. This accumulated texture is used for Parallax Occlusion Mapping when rendering terrain.
Mud Map:
R16_UNORM2048x2048
Sky and clouds
After going through a dirt calculation, the game does a lot of work with GPU calculations. They mainly relate to the sky and clouds. Clouds, fog and ambient light are some of the most prominent RDR2 effects. You can learn more about this step from Fabian's slides. He explains all this in much more detail than I can do.
Environment map
Environment maps are the main source of reflections for RDR2 as well as GTA5.
Like GTA5 , RDR2 generates an environment cube map from the camera position. The game engine generates a thin GBuffer for an environment map, similar to that used in Far Cry 4 .
The faces of the cubic map of the environment (albedo):
RGBA8_SRGB
The faces of the cubic map of the environment (normal):
RGBA8_UNORM
Environment cube edges (depth):
D32S8
Generating environment cube maps in each frame can be a very cumbersome task. To reduce computing costs, RDR2 uses optimizations. For example, the game only renders static and opaque objects, performs frustum culling before rendering each face, and renders lower LOD versions of the models. However, I found out that the number of terrain polygons is still very high for environment maps.
After passing G-Buffer, a cubic map of the sky environment is generated using a paraboloid map of the sky and cloud textures. The next step is the convolution. For Image Based Lighting, RDR2 engine uses split sum approximation... This method uses a pre-filtered environment map along with a BRDF environment LUT. For filtering, the game collapses the cubic map of the environment and stores minimized versions in the mip-texture levels of the cubic map.
Before performing a light pass for the cube irradiance map, RDR2 renders the baked large scale ambient occlusion into another cube texture. The game uses screen space ambient occlusion, but SSAO only helps on small scales. The baked ambient occlusion helps to perform darkening on a larger scale, such as darkening terraces and interiors.
Facets of a cubic environment map (baked AO):
R8_UNORM
The game uses delayed tile rendering to calculate the lighting of environment maps . Light culling and lighting are calculated together in one pass of the computational shader for each face of the environment map. (Thanks to @benoitvimont for that tip .) Also for baked lighting, the game uses a top-down world lightmap technique similar to Assassin's Creed III .
For each face of the cube map, RDR2 renders the final color over the sky environment texture. The game then filters the environment cube in the same way as the sky environment cube.
Facets of the environment cube map (final):
R11G11B10_FLOAT
Also, when the player is near a building, RDR2 loads environment maps located in the interiors of buildings. There are also G-Buffers of cube maps that are streamed from disk.
Baked faces of a cubic map of the environment (albedo):
BC3_SRGB(baked AO is stored in the alpha channel)
The baked faces of the cube environment map (normals):
BC3_UNORM
Environment cube baked edges (depth):
R16_UNORM
The game calculates the lighting of these maps and filters them like the previous ones. At a time, it calculates only one baked environment map, and recounts it only when the time of day changes. All environment maps are stored in an array of texture cube maps. No conversion of a cube map to a double paraboloid map is performed.
G-Buffer Pass
This stage begins with a preliminary walk through the depths of the relief, and then the game renders the scene in G-Buffers.
GBuffer 0 RGB |
GBuffer 0 A |
|---|---|
 |
 |
RGBA8_SRGB- this buffer contains albedo (base color) in RGB channels. I don't quite understand what the alpha channel data is for, but it is used during the anti-aliasing stage.
GBuffer 1 RGB |
GBuffer 1 A |
|---|---|
 |
 |
RGBA8_UNORM: RGB channels contain normals and alpha channel contains something related to fabric and hair.
GBuffer 2 RGB |
GBuffer 2 A |
|---|---|
 |
 |
RGBA8_UNORM: this target is used for material properties.
- R: Reflectance (f0)
- G: Smoothness
- B: Metallic
- A: contains shading (this channel will be used as a shadow mask in subsequent steps)
GBuffer 3 R |
GBuffer 3 B |
|---|---|
 |
 |
RGBA8_UNORM: the red channel contains cavities. There are some mysterious data in the blue channel again. And the alpha channel contains data related to hair. I couldn't find anything in the green channel.
GBuffer 4 RG |
|---|
 |
RG16_FLOAT: This buffer contains the screen space velocity for the motion blur implementation.
GBuffer 5 Depth |
GBuffer 5 Stencil |
|---|---|
 |
 |
D32S8: Like GTA5, RDR2 uses inverse z for depth , and stencil buffer to assign values to a specific group of meshes.
Another target is generated from the baked data:
GBuffer 6 R |
GBuffer 6 G |
|---|---|
 |
 |
This buffer contains the same baked ambient occlusion in the red channel as in the environment map stage. But there are other channels in this texture as well. The green channel contains data that resembles the data in the blue channel of GBuffer 3. Again, I do not understand what this data is used for. In the captured frames, I could not find any data in the blue and alpha channel. I will study this in more detail.
Shadow Map Generation
After the G-Buffer stage, the game starts rendering shadow maps. It uses 2D texture arrays for shadow maps of point sources and cubic texture arrays for shadow maps of searchlight sources.
Some games use a large shadow atlas texture for the shadow maps ( like DOOM ). One of the advantages of this method is that the size of the shadow map can vary greatly depending on distance. When using texture arrays, this flexibility is lost because all the textures in the array must be the same size. RDR2 has three different texture arrays for different quality levels. For example, floodlights have:
- 512x768 D16 for distant sources
- 1024x1536 D16 ( — )
- 2048x3072 D16 ( / )
Point sources cast shadows in all directions. To cope with this task, the games use a technique called Omnidirectional Shadow Mapping , in which the scene is rendered into a cubic depth map from the camera position. Using this technique, the shadows from the fire and the shadows from Arthur's lantern are rendered. Like spotlight sources, point sources have three different arrays for different quality settings.
Most static point sources in the game have baked cubic shadow maps.
Therefore, the game uses baked shadows whenever possible and generates a shadow map only when the player is near the volume of light. But it's actually even more interesting.
Most wall lights are spotlights, but the game does not generate an omnidirectional shadow map for them. Instead, it generates a shadowmap and copies the memory of that shadowmap into the cube array of the point source shadowmap.
 |
 |
On the left is a shadow map of searchlight sources of size 1024x1536, on the right is the same image data in a cubic texture format 512x512
Note that linear z maps store linear z.
This explains why the developers did not use a square shadow map for the spotlights. The number of pixels in the spotlight shadow map and the point source cube map must be the same. You can see that the right image is sliced into pieces. This is because the width of the spotlight and point shadow map is different.
Also note that this texture does not cover 360 degrees. However, fortunately, sources on buildings usually have a wall on the back, and baked shadow maps cover it up.
Another interesting point is that the process is reversed. Take for example Saint-Denis - one of the largest cities in the game. The game generates omnidirectional shadow maps for the spotlights and copies this data into an array of spotlights shadow maps. I don’t know why RDR2 does shadow mapping this way. I have not been able to find any such techniques on the Internet.
Directional shadowing in RDR2 is implemented in much the same way as in GTA5 . This is Cascaded Shadow Mapping with four cascades. As a cascade, each tile of size 1024x1024 from the texture 1024x4096 is used (with average graphics settings).
Spotlight Shadow Atlas:
R16_UNORM
Lighting stage
Finally, it is time to combine all of these environment maps, gbuffers, shadow maps, and ao buffers.
This stage consists of two passes: the first for global lighting (sun / moon), the second for local sources.
Global Lighting Pass
The game renders a full-screen quad to compute directional lighting, which is moonlight in our frame. It also uses baked lighting from the above-mentioned "top-down world lightmap".
Local lighting pass
In this pass, the game renders a low poly spherical shape for point source volumes and an octagon for spotlight volumes. Lighting is rendered back to front using additive blending.
To avoid unnecessary shader calls, the game uses depth constrained testing, an optional OpenGL / D3D11 feature that has become native to Vulkan / D3D12. She also uses stencil testing to discard pixels that are absorbed by translucent objects such as window panes. These objects will be rendered during the forward pass.
Water rendering and reflection
In this post I will not talk about rendering water, because it deserves a separate post, but I will say a little about reflections:
- As mentioned above, environment maps are the main source of reflections. For standard reflections. for example, for reflections in windows, the game uses them.
- Mirrors are rendered with planar reflections, in which the scene is rendered again from the direction of reflection. This process is also performed by deferred rendering.
- Reflections in water use screen space reflections in combination with the reflection map generated at the beginning of the frame.
Direct shading stage
One of the drawbacks of the deferred rendering pipeline is that it is not possible to properly render translucent materials with GBuffers. To solve this problem, the game renders back-to-front translucent materials using direct shading, like most deferred rendering.
But a direct pass can be costly:
- , .
, , , . ( ), .
- .
, . - . - (, ) .
- , .
Conveyor states must be changed to be able to toggle between trimming front and back edges. And such changes can be costly.
At this point, there is another render target generated for the bloom effect. This target stores the brightness of the bloom. As you can see in the image, translucent objects are brighter.
Target brightness bloom:
R8_UNORM
Note that the bloom is brighter at long distances to make the foggy areas brighter.
Post-processing
At this stage, time smoothing, bloom, motion blur, depth of field effect and other effects are performed. I plan to devote a separate post to post-processing. Therefore, we will not particularly discuss it here, but I would like to say a few words about bloom. Thanks to volumetric lighting, the main render target already has the glow of the light sources.
The bloom implementation in RDR2 is very similar to the implementation described in Next Generation Post Processing in Call of Duty: Advanced Warfare .
- Target without threshold values is taken as input,
- Render target
R11G11B10_FLOATlevel 7-mip, - 13th order bilinear filter for downsampling, 3x3 tent filter for upscaling.
The game then combines this filtered target of the bloom effect with the main target as well as the luminance target of the bloom.
conclusions
There is a lot more to say, but I don't want this post to be too long. I would like to share my findings about all this, as well as some of the oddities that I discovered during the analysis.
- , — . , . ( , .) bloom: - , , - , . GPU? ?
- , RDR2 . , (, GBuffers). ? ?
- : ( ) . SSAO, SSR, . target SSAO ?
Don't get me wrong, I'm not blaming anyone (although ... I'm a little frustrated with the low frame rate), just trying to figure out how such decisions were made. In the end, a lot of people worked on this game in a sweat . They probably didn't have enough time to further optimize the game.
Afterword
That's all! RDR2 is a great looking game, not only because of the graphics used, but also because of the art and lighting; everything looks just phenomenal. I fell in love with the color palette of this game. Especially at night, it reminds me of “How the cowardly Robert Ford killed Jesse James”, “Old Men Don't Take Place Here” and other westerns filmed with 35 mm cameras.