r.ShaderCompiler.ThreadLocalPreprocessBuffer

r.ShaderCompiler.ThreadLocalPreprocessBuffer

#Overview

name: r.ShaderCompiler.ThreadLocalPreprocessBuffer

This variable is created as a Console Variable (cvar).

It is referenced in 3 C++ source files.

#Summary

#Usage in the C++ source code

The purpose of r.ShaderCompiler.ThreadLocalPreprocessBuffer is to control the amount of memory preallocated for preprocess output per worker thread in the shader compiler. This setting is used to optimize memory allocation and reduce overhead during shader preprocessing.

This setting variable is primarily used in the shader preprocessing system, which is part of the Unreal Engine’s shader compilation pipeline. It is referenced in the ShaderPreprocessor module, specifically in the ShaderPreprocessor.cpp file.

The value of this variable is set using a TAutoConsoleVariable, which means it can be adjusted at runtime through console commands. The default value is set to 1280 * 1024 bytes (1.25 MB).

The associated variable CVarShaderCompilerThreadLocalPreprocessBuffer directly interacts with r.ShaderCompiler.ThreadLocalPreprocessBuffer. They share the same value and purpose.

Developers should be aware that this variable affects memory usage per worker thread in the shader compiler. Setting it too high might lead to excessive memory consumption, while setting it too low could result in more frequent memory reallocations and potential performance impacts.

Best practices when using this variable include:

  1. Monitor shader compilation performance and memory usage to find the optimal value for your project.
  2. Be cautious when adjusting this value, as it affects all worker threads in the shader compiler.
  3. Consider the target hardware specifications when setting this value, especially for projects targeting platforms with limited memory.

Regarding the associated variable CVarShaderCompilerThreadLocalPreprocessBuffer:

The purpose of CVarShaderCompilerThreadLocalPreprocessBuffer is the same as r.ShaderCompiler.ThreadLocalPreprocessBuffer. It’s used to control the preallocated memory for shader preprocessing output.

This variable is defined and used within the ShaderPreprocessor module. It’s implemented as a static TAutoConsoleVariable, which allows for runtime adjustment.

The value of this variable is set in the same place as r.ShaderCompiler.ThreadLocalPreprocessBuffer, with both sharing the same initial value and description.

CVarShaderCompilerThreadLocalPreprocessBuffer interacts directly with the thread-local storage used for preprocessing. It determines the size of the ThreadLocalPreprocessBuffer.

Developers should be aware that this variable is used to initialize thread-local storage, which means changes to its value may not take effect immediately for all threads.

Best practices for using CVarShaderCompilerThreadLocalPreprocessBuffer include:

  1. Use the GetValueOnAnyThread() method when accessing the value to ensure thread-safety.
  2. Be mindful of the clamping applied to the value (between 64 KB and 4 MB) to prevent potential issues with extremely small or large values.
  3. Consider profiling shader compilation performance with different values to find the optimal setting for your specific use case.

#References in C++ code

#Callsites

This variable is referenced in the following C++ source code:

#Loc: <Workspace>/Engine/Source/Developer/ShaderPreprocessor/Private/ShaderPreprocessor.cpp:12

Scope: file

Source code excerpt:


static TAutoConsoleVariable<int32> CVarShaderCompilerThreadLocalPreprocessBuffer(
	TEXT("r.ShaderCompiler.ThreadLocalPreprocessBuffer"),
	1280 * 1024,
	TEXT("Amount to preallocate for preprocess output per worker thread, to save reallocation overhead in the preprocessor."),
	ECVF_Default
);

namespace

#Associated Variable and Callsites

This variable is associated with another variable named CVarShaderCompilerThreadLocalPreprocessBuffer. They share the same value. See the following C++ source code.

#Loc: <Workspace>/Engine/Source/Developer/ShaderPreprocessor/Private/ShaderPreprocessor.cpp:11

Scope: file

Source code excerpt:

#include "stb_preprocess/stb_ds.h"

static TAutoConsoleVariable<int32> CVarShaderCompilerThreadLocalPreprocessBuffer(
	TEXT("r.ShaderCompiler.ThreadLocalPreprocessBuffer"),
	1280 * 1024,
	TEXT("Amount to preallocate for preprocess output per worker thread, to save reallocation overhead in the preprocessor."),
	ECVF_Default
);

#Loc: <Workspace>/Engine/Source/Developer/ShaderPreprocessor/Private/ShaderPreprocessor.cpp:730

Scope (from outer to inner):

file
function     bool PreprocessShader

Source code excerpt:

	pp_diagnostic* Diagnostics = nullptr;

	static const int32 ThreadLocalPreprocessBufferSize = CVarShaderCompilerThreadLocalPreprocessBuffer.GetValueOnAnyThread();
	static thread_local char* ThreadLocalPreprocessBuffer = nullptr;

	// Sanity check the buffer size so it won't OOM if a bad value is entered.
	int32 ClampedPreprocessBufferSize = ThreadLocalPreprocessBufferSize ? FMath::Clamp(ThreadLocalPreprocessBufferSize, 64 * 1024, 4 * 1024 * 1024) : 0;
	if (ClampedPreprocessBufferSize && !ThreadLocalPreprocessBuffer)
	{