MaxConcurrentShaderJobs

MaxConcurrentShaderJobs

#Overview

name: MaxConcurrentShaderJobs

The value of this variable can be defined or overridden in .ini config files. 1 .ini config file referencing this setting variable.

It is referenced in 7 C++ source files.

#Summary

#Usage in the C++ source code

The purpose of MaxConcurrentShaderJobs is to control the maximum number of concurrent shader compilation jobs during the cooking process in Unreal Engine 5. This setting is primarily used in the rendering system, specifically for managing shader compilation workload.

The Unreal Engine subsystem that relies on this setting variable is the Cook On The Fly Server, which is part of the UnrealEd module. This can be seen from the references in the UCookOnTheFlyServer class and related functions.

The value of this variable is set in the FInitializeConfigSettings::LoadLocal function, which reads the value from the GEditorIni configuration file. If not specified in the config file, it defaults to four times the number of CPU cores.

MaxConcurrentShaderJobs interacts with the GShaderCompilingManager, as seen in the CallBeginCacheOnObjects function. It’s used to limit the number of concurrent shader compilation jobs.

Developers must be aware that this variable directly affects the cooking performance and resource usage. Setting it too low can increase cook time, while setting it too high might overload the system.

Best practices when using this variable include:

  1. Adjusting it based on the available system resources and the complexity of shaders in the project.
  2. Monitoring the cooking performance and adjusting the value accordingly.
  3. Considering the trade-off between cooking speed and system resource usage.
  4. Documenting any changes made to this setting in the project configuration.

Remember that the default value (number of cores * 4) is generally a good starting point, but optimal values may vary depending on the specific project and hardware.

#Setting Variables

#References In INI files

Location: <Workspace>/Engine/Config/BaseEditor.ini:369, section: [CookSettings]

#References in C++ code

#Callsites

This variable is referenced in the following C++ source code:

#Loc: <Workspace>/Engine/Source/Editor/UnrealEd/Classes/CookOnTheSide/CookOnTheFlyServer.h:335

Scope (from outer to inner):

file
class        class UCookOnTheFlyServer : public UObject, public FTickableEditorObject, public FExec, public UE::Cook::ICookInfo

Source code excerpt:

	int32 MaxNumPackagesBeforePartialGC;
	/** Max number of concurrent shader jobs reducing this too low will increase cook time */
	int32 MaxConcurrentShaderJobs;
	/** Min number of free UObject indices before the cooker should partial gc */
	int32 MinFreeUObjectIndicesBeforeGC;
	double LastGCTime = 0.;
	double LastFullGCTime = 0.;
	double LastSoftGCTime = 0.;
	int64 SoftGCNextAvailablePhysicalTarget = -1;

#Loc: <Workspace>/Engine/Source/Editor/UnrealEd/Private/CookOnTheFlyServer.cpp:4065

Scope (from outer to inner):

file
function     UE::Cook::EPollStatus UCookOnTheFlyServer::CallBeginCacheOnObjects

Source code excerpt:

			if (Obj->IsA(UMaterialInterface::StaticClass()))
			{
				if (GShaderCompilingManager->GetNumRemainingJobs() + 1 > MaxConcurrentShaderJobs)
				{
#if DEBUG_COOKONTHEFLY
					UE_LOG(LogCook, Display, TEXT("Delaying shader compilation of material %s"), *Obj->GetFullName());
#endif
					return EPollStatus::Incomplete;
				}

#Loc: <Workspace>/Engine/Source/Editor/UnrealEd/Private/CookOnTheFlyServer.cpp:6872

Scope (from outer to inner):

file
namespace    UE::Cook
function     void FInitializeConfigSettings::LoadLocal

Source code excerpt:

	GConfig->GetInt(TEXT("CookSettings"), TEXT("MaxPrecacheShaderJobs"), MaxPrecacheShaderJobs, GEditorIni);

	MaxConcurrentShaderJobs = FPlatformMisc::NumberOfCores() * 4; // TODO: document why number of cores * 4 is a good default
	GConfig->GetInt(TEXT("CookSettings"), TEXT("MaxConcurrentShaderJobs"), MaxConcurrentShaderJobs, GEditorIni);

	PackagesPerGC = 500;
	int32 ConfigPackagesPerGC = 0;
	if (GConfig->GetInt( TEXT("CookSettings"), TEXT("PackagesPerGC"), ConfigPackagesPerGC, GEditorIni ))
	{
		// Going unsigned. Make negative values 0

#Loc: <Workspace>/Engine/Source/Editor/UnrealEd/Private/Cooker/CookTypes.cpp:617

Scope (from outer to inner):

file
function     FCbWriter& operator<<

Source code excerpt:

	Writer << "OutputDirectoryOverride" << Value.OutputDirectoryOverride;
	Writer << "MaxPrecacheShaderJobs" << Value.MaxPrecacheShaderJobs;
	Writer << "MaxConcurrentShaderJobs" << Value.MaxConcurrentShaderJobs;
	Writer << "PackagesPerGC" << Value.PackagesPerGC;
	Writer << "MemoryExpectedFreedToSpreadRatio" << Value.MemoryExpectedFreedToSpreadRatio;
	Writer << "IdleTimeToGC" << Value.IdleTimeToGC;
	Writer << "MemoryMaxUsedVirtual" << Value.MemoryMaxUsedVirtual;
	Writer << "MemoryMaxUsedPhysical" << Value.MemoryMaxUsedPhysical;
	Writer << "MemoryMinFreeVirtual" << Value.MemoryMinFreeVirtual;

#Loc: <Workspace>/Engine/Source/Editor/UnrealEd/Private/Cooker/CookTypes.cpp:643

Scope (from outer to inner):

file
function     bool LoadFromCompactBinary

Source code excerpt:

	bOk = LoadFromCompactBinary(Field["OutputDirectoryOverride"], OutValue.OutputDirectoryOverride) & bOk;
	bOk = LoadFromCompactBinary(Field["MaxPrecacheShaderJobs"], OutValue.MaxPrecacheShaderJobs) & bOk;
	bOk = LoadFromCompactBinary(Field["MaxConcurrentShaderJobs"], OutValue.MaxConcurrentShaderJobs) & bOk;
	bOk = LoadFromCompactBinary(Field["PackagesPerGC"], OutValue.PackagesPerGC) & bOk;
	bOk = LoadFromCompactBinary(Field["MemoryExpectedFreedToSpreadRatio"], OutValue.MemoryExpectedFreedToSpreadRatio) & bOk;
	bOk = LoadFromCompactBinary(Field["IdleTimeToGC"], OutValue.IdleTimeToGC) & bOk;
	bOk = LoadFromCompactBinary(Field["MemoryMaxUsedVirtual"], OutValue.MemoryMaxUsedVirtual) & bOk;
	bOk = LoadFromCompactBinary(Field["MemoryMaxUsedPhysical"], OutValue.MemoryMaxUsedPhysical) & bOk;
	bOk = LoadFromCompactBinary(Field["MemoryMinFreeVirtual"], OutValue.MemoryMinFreeVirtual) & bOk;

#Loc: <Workspace>/Engine/Source/Editor/UnrealEd/Private/Cooker/CookTypes.cpp:680

Scope (from outer to inner):

file
namespace    UE::Cook
function     void FInitializeConfigSettings::MoveOrCopy

Source code excerpt:

	Target.OutputDirectoryOverride = MoveTempIfPossible(Source.OutputDirectoryOverride);
	Target.MaxPrecacheShaderJobs = Source.MaxPrecacheShaderJobs;
	Target.MaxConcurrentShaderJobs = Source.MaxConcurrentShaderJobs;
	Target.PackagesPerGC = Source.PackagesPerGC;
	Target.MemoryExpectedFreedToSpreadRatio = Source.MemoryExpectedFreedToSpreadRatio;
	Target.IdleTimeToGC = Source.IdleTimeToGC;
	Target.MemoryMaxUsedVirtual = Source.MemoryMaxUsedVirtual;
	Target.MemoryMaxUsedPhysical = Source.MemoryMaxUsedPhysical;
	Target.MemoryMinFreeVirtual = Source.MemoryMinFreeVirtual;

#Loc: <Workspace>/Engine/Source/Editor/UnrealEd/Private/Cooker/CookTypes.h:312

Scope (from outer to inner):

file
namespace    UE::Cook

Source code excerpt:

		FString OutputDirectoryOverride;
		int32 MaxPrecacheShaderJobs = 0;
		int32 MaxConcurrentShaderJobs = 0;
		uint32 PackagesPerGC = 0;
		float MemoryExpectedFreedToSpreadRatio = 0.f;
		double IdleTimeToGC = 0.;
		uint64 MemoryMaxUsedVirtual;
		uint64 MemoryMaxUsedPhysical;
		uint64 MemoryMinFreeVirtual;