MaxShaderJobBatchSize

MaxShaderJobBatchSize

#Overview

name: MaxShaderJobBatchSize

The value of this variable can be defined or overridden in .ini config files. 2 .ini config files referencing this setting variable.

It is referenced in 3 C++ source files.

#Summary

#Usage in the C++ source code

The purpose of MaxShaderJobBatchSize is to control the maximum number of shader compilation jobs that can be processed in a single batch. This variable is primarily used in the shader compilation system of Unreal Engine 5.

Based on the Callsites section, the MaxShaderJobBatchSize variable is utilized in the Engine module, specifically within the ShaderCompiler subsystem. It’s referenced in the ShaderCompiler.cpp and ShaderCompiler.h files, which are part of the shader compilation management system.

The value of this variable is set in the FShaderCompilingManager constructor. It’s loaded from the engine configuration file (GEngineIni) using the GConfig->GetInt function, with the section “DevOptions.Shaders” and the key “MaxShaderJobBatchSize”.

MaxShaderJobBatchSize interacts with other variables in the shader compilation process, such as NumJobsPerWorker and NumPendingJobs. It’s used to limit the number of jobs that can be assigned to a worker thread in a single batch.

Developers should be aware that this variable directly impacts the shader compilation performance and resource utilization. A larger value may improve compilation speed but could also increase memory usage and potentially impact system responsiveness.

Best practices when using this variable include:

  1. Adjusting it based on the target hardware capabilities. More powerful systems may benefit from larger batch sizes.
  2. Balancing it with other shader compilation settings for optimal performance.
  3. Monitoring its impact on compilation times and system resources when modifying its value.
  4. Considering different values for development and shipping builds, as compilation needs may vary.
  5. Documenting any custom values used in project settings for team awareness and consistency.

#Setting Variables

#References In INI files

Location: <Workspace>/Engine/Config/BaseEngine.ini:1962, section: [DevOptions.Shaders]

Location: <Workspace>/Engine/Config/Mac/BaseMacEngine.ini:12, section: [DevOptions.Shaders]

#References in C++ code

#Callsites

This variable is referenced in the following C++ source code:

#Loc: <Workspace>/Engine/Source/Runtime/Engine/Private/ShaderCompiler/ShaderCompiler.cpp:3624

Scope (from outer to inner):

file
function     int32 FShaderCompileThreadRunnable::PullTasksFromQueue

Source code excerpt:

						if (PriorityIndex < (int32)EShaderCompileJobPriority::High || Manager->IgnoreAllThrottling())
						{
							MaxNumJobs = FMath::Min3(NumJobsPerWorker, NumPendingJobs, Manager->MaxShaderJobBatchSize);
						}

						NumJobsStarted[PriorityIndex] += Manager->AllJobs.GetPendingJobs(EShaderCompilerWorkerType::LocalThread, (EShaderCompileJobPriority)PriorityIndex, 1, MaxNumJobs, CurrentWorkerInfo.QueuedJobs);

						// Update the worker state as having new tasks that need to be issued					
						// don't reset worker app ID, because the shadercompileworkers don't shutdown immediately after finishing a single job queue.

#Loc: <Workspace>/Engine/Source/Runtime/Engine/Private/ShaderCompiler/ShaderCompiler.cpp:5628

Scope (from outer to inner):

file
function     FShaderCompilingManager::FShaderCompilingManager

Source code excerpt:

	}

	verify(GConfig->GetInt( TEXT("DevOptions.Shaders"), TEXT("MaxShaderJobBatchSize"), MaxShaderJobBatchSize, GEngineIni ));
	verify(GConfig->GetBool( TEXT("DevOptions.Shaders"), TEXT("bPromptToRetryFailedShaderCompiles"), bPromptToRetryFailedShaderCompiles, GEngineIni ));
	verify(GConfig->GetBool(TEXT("DevOptions.Shaders"), TEXT("bDebugBreakOnPromptToRetryShaderCompile"), bDebugBreakOnPromptToRetryShaderCompile, GEngineIni));
	verify(GConfig->GetBool( TEXT("DevOptions.Shaders"), TEXT("bLogJobCompletionTimes"), bLogJobCompletionTimes, GEngineIni ));
	GConfig->GetFloat(TEXT("DevOptions.Shaders"), TEXT("WorkerTimeToLive"), GRegularWorkerTimeToLive, GEngineIni);
	GConfig->GetFloat(TEXT("DevOptions.Shaders"), TEXT("BuildWorkerTimeToLive"), GBuildWorkerTimeToLive, GEngineIni);

#Loc: <Workspace>/Engine/Source/Runtime/Engine/Public/ShaderCompiler.h:703

Scope (from outer to inner):

file
class        class FShaderCompilingManager : IAssetCompilingManager

Source code excerpt:

	uint32 NumShaderCompilingThreadsDuringGame;
	/** Largest number of jobs that can be put in the same batch. */
	int32 MaxShaderJobBatchSize;
	/** Number of runs through single-threaded compiling before we can retry to compile through workers. -1 if not used. */
	int32 NumSingleThreadedRunsBeforeRetry;
	/** Number of preprocessed shader sources that are dumped due to a crash of the shader compiler. */
	/** This state is updated when a job is completed, which happens on a worker thread.  The state is read on the Game Thread to see if we should dump the source to begin with. */
	std::atomic<int32> NumDumpedShaderSources = 0;
	/** Process Id of UE. */