bAllowCompilingThroughWorkers
bAllowCompilingThroughWorkers
#Overview
name: bAllowCompilingThroughWorkers
The value of this variable can be defined or overridden in .ini config files. 1
.ini config file referencing this setting variable.
It is referenced in 10
C++ source files.
#Summary
#Usage in the C++ source code
The purpose of bAllowCompilingThroughWorkers is to control whether shader compilation can be performed through worker processes, allowing for multi-core utilization during shader compilation.
This setting variable is primarily used in the shader compilation system of Unreal Engine. It is referenced in the ShaderCompiler module, specifically in the FShaderCompilingManager and FShaderCompileThreadRunnable classes.
The value of this variable is typically set in the engine configuration file (GEngineIni) under the [DevOptions.Shaders] section. It can also be overridden through command-line parameters or console variables.
bAllowCompilingThroughWorkers interacts with other variables and systems, such as:
- CVarCompileParallelInProcess
- CVarAllowCompilingThroughWorkers
- The number of virtual cores available on the system
- The ability to support multi-threading on the platform
Developers should be aware of the following when using this variable:
- Setting it to false will force shader compilation to occur in-process and single-threaded.
- It can be automatically disabled if the platform doesn’t support multi-threading or if specific command-line arguments are used.
- It affects the number of shader compiling threads used, both during normal operation and game execution.
Best practices when using this variable include:
- Ensure it’s enabled for better performance on multi-core systems.
- Consider disabling it for debugging purposes or when investigating shader compilation issues.
- Be aware of its interaction with other shader compilation settings and adjust accordingly based on the specific needs of your project and development environment.
- Monitor shader compilation performance and adjust the setting if necessary, especially on machines with few cores where compile latency might be prioritized over editor performance.
#Setting Variables
#References In INI files
Location: <Workspace>/Engine/Config/BaseEngine.ini:1949, section: [DevOptions.Shaders]
- INI Section:
DevOptions.Shaders
- Raw value:
True
- Is Array:
False
#References in C++ code
#Callsites
This variable is referenced in the following C++ source code:
#Loc: <Workspace>/Engine/Source/Runtime/Engine/Private/ShaderCompiler/ShaderCompiler.cpp:4320
Scope (from outer to inner):
file
function int32 FShaderCompileThreadRunnable::CompilingLoop
Source code excerpt:
int32 FShaderCompileThreadRunnable::CompilingLoop()
{
if (!Manager->bAllowCompilingThroughWorkers && CVarCompileParallelInProcess.GetValueOnAnyThread())
{
int32 NumJobs = Manager->GetNumPendingJobs();
if (NumJobs == 0)
{
return 0;
}
#Loc: <Workspace>/Engine/Source/Runtime/Engine/Private/ShaderCompiler/ShaderCompiler.cpp:4384
Scope (from outer to inner):
file
function int32 FShaderCompileThreadRunnable::CompilingLoop
Source code excerpt:
return 0;
}
else // compile either through worker processes or single-threaded in-process (depending on Manager->bAllowCompilingThroughWorkers)
{
// push completed jobs to Manager->ShaderMapJobs before asking for new ones, so we can free the workers now and avoid them waiting a cycle
PushCompletedJobsToManager();
// Grab more shader compile jobs from the input queue
const int32 NumActiveThreads = PullTasksFromQueue();
#Loc: <Workspace>/Engine/Source/Runtime/Engine/Private/ShaderCompiler/ShaderCompiler.cpp:4400
Scope (from outer to inner):
file
function int32 FShaderCompileThreadRunnable::CompilingLoop
Source code excerpt:
}
if (Manager->bAllowCompilingThroughWorkers)
{
// Write out the files which are input to the shader compile workers
WriteNewTasks();
// Launch shader compile workers if they are not already running
// Workers can time out when idle so they may need to be relaunched
#Loc: <Workspace>/Engine/Source/Runtime/Engine/Private/ShaderCompiler/ShaderCompiler.cpp:4413
Scope (from outer to inner):
file
function int32 FShaderCompileThreadRunnable::CompilingLoop
Source code excerpt:
// Fall back to local compiles if the SCW crashed.
// This is nasty but needed to work around issues where message passing through files to SCW is unreliable on random PCs
Manager->bAllowCompilingThroughWorkers = false;
// Try to recover from abandoned workers after a certain amount of single-threaded compilations
if (Manager->NumSingleThreadedRunsBeforeRetry == GSingleThreadedRunsIdle)
{
// First try to recover, only run single-threaded approach once
Manager->NumSingleThreadedRunsBeforeRetry = 1;
#Loc: <Workspace>/Engine/Source/Runtime/Engine/Private/ShaderCompiler/ShaderCompiler.cpp:4455
Scope (from outer to inner):
file
function int32 FShaderCompileThreadRunnable::CompilingLoop
Source code excerpt:
{
UE_LOG(LogShaderCompilers, Display, TEXT("Retry shader compiling through workers."));
Manager->bAllowCompilingThroughWorkers = true;
}
}
}
return NumActiveThreads;
}
#Loc: <Workspace>/Engine/Source/Runtime/Engine/Private/ShaderCompiler/ShaderCompiler.cpp:5610
Scope (from outer to inner):
file
function FShaderCompilingManager::FShaderCompilingManager
Source code excerpt:
// Read values from the engine ini
verify(GConfig->GetBool( TEXT("DevOptions.Shaders"), TEXT("bAllowCompilingThroughWorkers"), bAllowCompilingThroughWorkers, GEngineIni ));
verify(GConfig->GetBool( TEXT("DevOptions.Shaders"), TEXT("bAllowAsynchronousShaderCompiling"), bAllowAsynchronousShaderCompiling, GEngineIni ));
// Explicitly load ShaderPreprocessor module so it will run its initialization step
FModuleManager::LoadModuleChecked<IModuleInterface>(TEXT("ShaderPreprocessor"));
// override the use of workers, can be helpful for debugging shader compiler code
#Loc: <Workspace>/Engine/Source/Runtime/Engine/Private/ShaderCompiler/ShaderCompiler.cpp:5620
Scope (from outer to inner):
file
function FShaderCompilingManager::FShaderCompilingManager
Source code excerpt:
if (!FPlatformProcess::SupportsMultithreading() || FParse::Param(FCommandLine::Get(), TEXT("noshaderworker")) || (CVarAllowCompilingThroughWorkers && CVarAllowCompilingThroughWorkers->GetInt() == 0))
{
bAllowCompilingThroughWorkers = false;
}
if (!FPlatformProcess::SupportsMultithreading())
{
bAllowAsynchronousShaderCompiling = false;
}
#Loc: <Workspace>/Engine/Source/Runtime/Engine/Private/ShaderCompiler/ShaderCompiler.cpp:5689
Scope (from outer to inner):
file
function FShaderCompilingManager::FShaderCompilingManager
Source code excerpt:
TUniquePtr<FShaderCompileThreadRunnableBase> RemoteCompileThread;
const bool bCanUseRemoteCompiling = bAllowCompilingThroughWorkers && ShaderCompiler::IsRemoteCompilingAllowed() && AllTargetPlatformSupportsRemoteShaderCompiling();
BuildDistributionController = bCanUseRemoteCompiling ? FindRemoteCompilerController() : nullptr;
if (BuildDistributionController)
{
UE_LOG(LogShaderCompilers, Display, TEXT("Using %s for Shader Compilation."), *BuildDistributionController->GetName());
RemoteCompileThread = MakeUnique<FShaderCompileDistributedThreadRunnable_Interface>(this, *BuildDistributionController);
#Loc: <Workspace>/Engine/Source/Runtime/Engine/Private/ShaderCompiler/ShaderCompiler.cpp:5894
Scope (from outer to inner):
file
function void FShaderCompilingManager::CalculateNumberOfCompilingThreads
Source code excerpt:
}
NumShaderCompilingThreads = (bAllowCompilingThroughWorkers && NumVirtualCores > NumUnusedShaderCompilingThreads) ? (NumVirtualCores - NumUnusedShaderCompilingThreads) : 1;
// Make sure there's at least one worker allowed to be active when compiling during the game
NumShaderCompilingThreadsDuringGame = (bAllowCompilingThroughWorkers && NumVirtualCores > NumUnusedShaderCompilingThreadsDuringGame) ? (NumVirtualCores - NumUnusedShaderCompilingThreadsDuringGame) : 1;
// On machines with few cores, each core will have a massive impact on compile time, so we prioritize compile latency over editor performance during the build
if (NumVirtualCores <= 4)
{
NumShaderCompilingThreads = NumVirtualCores - 1;
NumShaderCompilingThreadsDuringGame = NumVirtualCores - 1;
#Loc: <Workspace>/Engine/Source/Runtime/Engine/Public/ShaderCompiler.h:712
Scope (from outer to inner):
file
class class FShaderCompilingManager : IAssetCompilingManager
Source code excerpt:
uint32 ProcessId;
/** Whether to allow compiling shaders through the worker application, which allows multiple cores to be used. */
bool bAllowCompilingThroughWorkers;
/** Whether to allow shaders to compile in the background or to block after each material. */
bool bAllowAsynchronousShaderCompiling;
/** Whether to ask to retry a failed shader compile error. */
bool bPromptToRetryFailedShaderCompiles;
/** If enabled when we enter the prompt to retry we will break in the debugger if one is attached rather than prompting. */
bool bDebugBreakOnPromptToRetryShaderCompile = false;