AddWork
AddWork
#Overview
name: AddWork
This variable is created as a Console Variable (cvar).
- type:
Cmd
- help: ``
It is referenced in 29
C++ source files.
#Summary
#Usage in the C++ source code
The purpose of AddWork is to add new tasks or work items to a work queue or job system within the Unreal Engine. This function is used in various subsystems to schedule work that needs to be executed, often in a parallel or asynchronous manner.
Based on the provided callsites, AddWork is utilized in several Unreal Engine subsystems, plugins, and modules:
- UnrealEd: Used for testing and debugging purposes in the editor.
- VirtualHeightfieldMesh: A plugin for rendering virtual heightfield meshes.
- UnrealBuildAccelerator: A tool for speeding up build processes.
- Networking: Used in the NetworkServer class for managing network-related tasks.
- Storage: Used in various storage-related operations, including file compression and decompression.
- Work Management: Part of a general work management system in the engine.
The value of this variable is typically set when calling the AddWork function, which takes a work item (usually a lambda function or a function object) and a count of how many times to add that work item.
AddWork often interacts with other variables and systems:
- It may use mutex locks for thread safety.
- It often works with a WorkManager or similar system that manages the execution of work items.
- In some cases, it interacts with event systems for signaling when work is complete.
Developers should be aware of the following when using AddWork:
- Thread safety: Ensure proper synchronization when adding work from multiple threads.
- Work item lifecycle: Be cautious about capturing variables in lambdas, ensuring they remain valid for the duration of the work item’s execution.
- Performance impact: Adding too many small work items can lead to overhead; consider batching work when appropriate.
Best practices for using AddWork include:
- Use descriptive names for work items to aid in debugging and profiling.
- Balance work distribution to avoid overloading the work queue or creating too fine-grained tasks.
- Consider the priority and dependencies of work items when adding them.
- Use the appropriate WorkManager for the context (e.g., rendering, networking, etc.).
- Monitor and profile the work queue to ensure efficient execution and identify bottlenecks.
#References in C++ code
#Callsites
This variable is referenced in the following C++ source code:
#Loc: <Workspace>/Engine/Source/Editor/UnrealEd/Private/GlobalEditorNotification.cpp:165
Scope: file
Source code excerpt:
}
FAutoConsoleCommand StartWorkTestCommand(TEXT("StartWorkTest"), TEXT(""), FConsoleCommandDelegate::CreateStatic(&StartWorkTest));
FAutoConsoleCommand AddWorkTestCommand(TEXT("AddWork"), TEXT(""), FConsoleCommandDelegate::CreateStatic(&AddWork));
FAutoConsoleCommand StopWorkTestCommand(TEXT("StopWorkTest"), TEXT(""), FConsoleCommandDelegate::CreateStatic(&StopWorkTest));
#Loc: <Workspace>/Engine/Plugins/Experimental/VirtualHeightfieldMesh/Source/VirtualHeightfieldMesh/Private/VirtualHeightfieldMeshSceneProxy.cpp:184
Scope (from outer to inner):
file
class class FVirtualHeightfieldMeshRendererExtension : public FRenderResource
Source code excerpt:
/** Call once per frame for each mesh/view that has relevance. This allocates the buffers to use for the frame and adds the work to fill the buffers to the queue. */
VirtualHeightfieldMesh::FDrawInstanceBuffers& AddWork(FRHICommandListBase& RHICmdList, FVirtualHeightfieldMeshSceneProxy const* InProxy, FSceneView const* InMainView, FSceneView const* InCullView);
/** Submit all the work added by AddWork(). The work fills all of the buffers ready for use by the referencing mesh batches. */
void SubmitWork(FRDGBuilder& GraphBuilder);
protected:
//~ Begin FRenderResource Interface
virtual void ReleaseRHI() override;
//~ End FRenderResource Interface
#Loc: <Workspace>/Engine/Plugins/Experimental/VirtualHeightfieldMesh/Source/VirtualHeightfieldMesh/Private/VirtualHeightfieldMeshSceneProxy.cpp:265
Scope (from outer to inner):
file
function VirtualHeightfieldMesh::FDrawInstanceBuffers& FVirtualHeightfieldMeshRendererExtension::AddWork
Source code excerpt:
}
VirtualHeightfieldMesh::FDrawInstanceBuffers& FVirtualHeightfieldMeshRendererExtension::AddWork(FRHICommandListBase& RHICmdList, FVirtualHeightfieldMeshSceneProxy const* InProxy, FSceneView const* InMainView, FSceneView const* InCullView)
{
UE::TScopeLock Lock(Mutex);
// If we hit this then BegineFrame()/EndFrame() logic needs fixing in the Scene Renderer.
if (!ensure(!bInFrame))
{
#Loc: <Workspace>/Engine/Plugins/Experimental/VirtualHeightfieldMesh/Source/VirtualHeightfieldMesh/Private/VirtualHeightfieldMeshSceneProxy.cpp:549
Scope (from outer to inner):
file
function void FVirtualHeightfieldMeshSceneProxy::GetDynamicMeshElements
Source code excerpt:
{
// Can't add new work while bInFrame.
// In UE5 we need to AddWork()/SubmitWork() in two phases: InitViews() and InitViewsAfterPrepass()
// The main renderer hooks for that don't exist in UE5.0 and are only added in UE5.1
// That means that for UE5.0 we always hit this for shadow drawing and shadows will not be rendered.
// Not earlying out here can lead to crashes from buffers being released too soon.
return;
}
#Loc: <Workspace>/Engine/Plugins/Experimental/VirtualHeightfieldMesh/Source/VirtualHeightfieldMesh/Private/VirtualHeightfieldMeshSceneProxy.cpp:560
Scope (from outer to inner):
file
function void FVirtualHeightfieldMeshSceneProxy::GetDynamicMeshElements
Source code excerpt:
if (VisibilityMap & (1 << ViewIndex))
{
VirtualHeightfieldMesh::FDrawInstanceBuffers& Buffers = GVirtualHeightfieldMeshViewRendererExtension.AddWork(Collector.GetRHICommandList(), this, ViewFamily.Views[0], Views[ViewIndex]);
FMeshBatch& Mesh = Collector.AllocateMesh();
Mesh.bWireframe = AllowDebugViewmodes() && ViewFamily.EngineShowFlags.Wireframe;
Mesh.bUseWireframeSelectionColoring = IsSelected();
Mesh.VertexFactory = VertexFactory;
Mesh.MaterialRenderProxy = Material;
#Loc: <Workspace>/Engine/Source/Editor/UnrealEd/Private/GlobalEditorNotification.cpp:155
Scope (from outer to inner):
file
function static void AddWork
Source code excerpt:
}
static void AddWork()
{
int32 TaskIndex = FMath::Rand() % TestProgressNotificationTasks.Num();
TUniquePtr<FTestProgressNotification>& TestProgressNotificationTask = TestProgressNotificationTasks[TaskIndex];
//if (TestProgressNotificationTask)
{
TestProgressNotificationTask->RemainingJobs += 100;
#Loc: <Workspace>/Engine/Source/Programs/UnrealBuildAccelerator/Cli/Private/UbaCli.cpp:404
Scope (from outer to inner):
file
namespace uba
function int WrappedMain
lambda-function
Source code excerpt:
storageServer.TraverseAllCasFiles([&](const CasKey& casKey)
{
workManager.AddWork([&, casKey]()
{
Storage::RetrieveResult res;
storageServer.EnsureCasFile(casKey, TC("Dummy"));
CasKey casKey2 = AsCompressed(casKey, false);
if (!storageClient.RetrieveCasFile(res, casKey2, TC("")))
success = false;
#Loc: <Workspace>/Engine/Source/Programs/UnrealBuildAccelerator/Common/Private/UbaFileMapping.cpp:566
Scope (from outer to inner):
file
namespace uba
function void FileMappingBuffer::UnmapView
Source code excerpt:
};
if (m_workManager && newSize == InvalidValue)
m_workManager->AddWork([=, h = TString(hint_)]() { unmap(h.c_str()); }, 1, TC("UnmapView"));
else
unmap(hint_);
}
void FileMappingBuffer::GetSizeAndCount(FileMappingType type, u64& outSize, u32& outCount)
{
#Loc: <Workspace>/Engine/Source/Programs/UnrealBuildAccelerator/Common/Private/UbaNetworkServer.cpp:536
Scope (from outer to inner):
file
namespace uba
function void NetworkServer::Worker::DoAdditionalWorkAndSignalAvailable
Source code excerpt:
// Both locks needs to be taken to verify if additional work
// is present before making ourself available to avoid
// a race where AddWork would not see this thread in the
// available list after adding some work.
SCOPED_WRITE_LOCK(server.m_availableWorkersLock, lock1);
SCOPED_READ_LOCK(server.m_additionalWorkLock, lock2);
// Verify there is not additional work while we hold both lock
// and only add ourself as available if no additional work is present.
if (!server.m_additionalWork.empty())
#Loc: <Workspace>/Engine/Source/Programs/UnrealBuildAccelerator/Common/Private/UbaNetworkServer.cpp:779
Scope (from outer to inner):
file
namespace uba
function void NetworkServer::AddWork
Source code excerpt:
}
void NetworkServer::AddWork(const Function<void()>& work, u32 count, const tchar* desc)
{
SCOPED_WRITE_LOCK(m_additionalWorkLock, lock);
for (u32 i = 0; i != count; ++i)
{
m_additionalWork.push_back({ work });
if (m_workTracker)
#Loc: <Workspace>/Engine/Source/Programs/UnrealBuildAccelerator/Common/Private/UbaSession.cpp:2026
Scope (from outer to inner):
file
function bool Session::WriteFilesToDisk
lambda-function
Source code excerpt:
if (IsRarelyReadAfterWritten(process, file.name.c_str(), file.name.size()) || file.mappingWritten > m_keepOutputFileMemoryMapsThreshold)
{
m_workManager->AddWork([mh = file.mappingHandle]()
{
CloseFileMapping(mh);
}, 1, TC("CFM"));
//CloseFileMapping(file.mappingHandle);
}
else
#Loc: <Workspace>/Engine/Source/Programs/UnrealBuildAccelerator/Common/Private/UbaSession.cpp:2072
Scope (from outer to inner):
file
function bool Session::WriteFilesToDisk
Source code excerpt:
events[i].Create(true);
m_workManager->AddWork([&, ii = i]()
{
writeFile(*files[ii]);
events[ii].Set();
}, 1, TC(""));
}
for (u32 i=0; i!=fileCount; ++i)
events[i].IsSet();
*/
return true;
}
bool Session::AllocFailed(Process& process, const tchar* allocType, u32 error)
{
m_logger.Warning(TC("Allocation failed in %s (%s).. process will sleep and try again"), allocType, LastErrorToText(error).data);
return true;
}
bool Session::GetNextProcess(Process& process, bool& outNewProcess, NextProcessInfo& outNextProcess, u32 prevExitCode, BinaryReader& statsReader)
{
if (!m_getNextProcessFunction)
{
outNewProcess = false;
return true;
#Loc: <Workspace>/Engine/Source/Programs/UnrealBuildAccelerator/Common/Private/UbaSessionClient.cpp:307
Scope (from outer to inner):
file
namespace uba
function bool SessionClient::EnsureApplicationEnvironment
Source code excerpt:
for (auto& m : modules)
{
m_client.AddWork([&handledCount, &m, this, &success, &keyStr]()
{
++handledCount;
auto g = MakeGuard([&]() { m.done.Set(); });
CasKey newCasKey;
bool storeUncompressed = true;
u64 fileSize;
#Loc: <Workspace>/Engine/Source/Programs/UnrealBuildAccelerator/Common/Private/UbaSessionClient.cpp:1181
Scope (from outer to inner):
file
namespace uba
function bool SessionClient::SendProcessAvailable
Source code excerpt:
knownInputKey = AsCompressed(knownInputKey, false);
m_client.AddWork([knownInputKey, this]()
{
Storage::RetrieveResult result;
bool allowProxy = true;
bool res = m_storage.RetrieveCasFile(result, knownInputKey, TC("KnownInput"), nullptr, 1, allowProxy);
(void)res;
}, 1, TC("KnownInput"));
#Loc: <Workspace>/Engine/Source/Programs/UnrealBuildAccelerator/Common/Private/UbaStorage.cpp:261
Scope (from outer to inner):
file
namespace uba
function bool StorageImpl::WriteCompressed
lambda-function
Source code excerpt:
{
#if UBA_EXPERIMENTAL
m_workManager->AddWork([=, f = TString(from)]() { UnmapViewOfFile(uncompressedData, fileSize, f.c_str()); }, 1, TC("UnmapFile"));
#else
UnmapViewOfFile(uncompressedData, fileSize, from);
#endif
});
if (!WriteMemToCompressedFile(destinationFile, workCount, uncompressedData, fileSize, maxUncompressedBlock, totalWritten))
#Loc: <Workspace>/Engine/Source/Programs/UnrealBuildAccelerator/Common/Private/UbaStorage.cpp:450
Scope (from outer to inner):
file
namespace uba
function bool StorageImpl::WriteMemToCompressedFile
Source code excerpt:
rec->refCount = workerCount + 1; // We need to keep refcount up 1 to make sure it is not deleted before we read rec->written
m_workManager->AddWork(work, workerCount-1, TC("Compress")); // We are a worker ourselves
work();
rec->events[rec->workCount - 1].IsSet();
totalWritten += rec->written;
bool error = rec->error;
#Loc: <Workspace>/Engine/Source/Programs/UnrealBuildAccelerator/Common/Private/UbaStorage.cpp:636
Scope (from outer to inner):
file
namespace uba
function bool StorageImpl::DecompressMemoryToMemory
Source code excerpt:
workerCount = Min(workerCount, MaxWorkItemsPerAction); // Cap this to not starve other things
rec->refCount += workerCount;
m_workManager->AddWork(work, workerCount, TC("DecompressMemToMem"));
}
TimerScope ts(stats.decompressToMem);
work();
rec->done.IsSet();
bool success = !rec->error;
#Loc: <Workspace>/Engine/Source/Programs/UnrealBuildAccelerator/Common/Private/UbaStorage.cpp:1464
Scope (from outer to inner):
file
namespace uba
function bool StorageImpl::CheckCasContent
lambda-function
Source code excerpt:
{
++entryCount;
workManager.AddWork([&, filePath = TString(fullPath.data), name = TString(e.name), lastWritten = e.lastWritten]()
{
StringBuffer<> timeStr;
writeTimeAgo(timeStr, lastWritten);
//m_logger.Info(TC("Validating %s (%s ago)"), name.c_str(), timeStr.data);
#Loc: <Workspace>/Engine/Source/Programs/UnrealBuildAccelerator/Common/Private/UbaStorage.cpp:1572
Scope (from outer to inner):
file
namespace uba
function bool StorageImpl::DeleteAllCas
lambda-function
Source code excerpt:
if (!IsDirectory(e.attributes))
return;
workManager.AddWork([&, name = TString(e.name)]()
{
StringBuffer<> fullPath;
fullPath.Append(m_rootDir).Append(name);
u32 deleteCountTemp = 0;
DeleteAllFiles(m_logger, fullPath.data, true, &deleteCountTemp);
atomicDeleteCount += deleteCountTemp;
#Loc: <Workspace>/Engine/Source/Programs/UnrealBuildAccelerator/Common/Private/UbaStorage.cpp:2266
Scope (from outer to inner):
file
namespace uba
function CasKey StorageImpl::CalculateCasKey
Source code excerpt:
workerCount = Min(workerCount, MaxWorkItemsPerAction); // Cap this to not starve other things
rec->refCount += workerCount;
m_workManager->AddWork(work, workerCount, TC("CalculateKey"));
}
work();
rec->done.IsSet();
hasher.Update(rec->keys.data(), rec->keys.size()*sizeof(CasKey));
#Loc: <Workspace>/Engine/Source/Programs/UnrealBuildAccelerator/Common/Private/UbaStorage.cpp:2319
Scope (from outer to inner):
file
namespace uba
function CasKey StorageImpl::CalculateCasKey
lambda-function
Source code excerpt:
auto udg = MakeGuard([&]() {
#if UBA_EXPERIMENTAL
m_workManager->AddWork([=, fn = TString(fileName)]() { UnmapViewOfFile(fileData, fileSize, fn.c_str()); }, 1, TC("UnmapFile"));
#else
UnmapViewOfFile(fileData, fileSize, fileName);
#endif
});
struct WorkRec
#Loc: <Workspace>/Engine/Source/Programs/UnrealBuildAccelerator/Common/Private/UbaStorage.cpp:2378
Scope (from outer to inner):
file
namespace uba
function CasKey StorageImpl::CalculateCasKey
Source code excerpt:
rec->refCount = workerCount + 1; // We need to keep refcount up 1 to make sure it is not deleted before we read rec->written
m_workManager->AddWork(work, workerCount-1, TC("CalculateKey")); // We are a worker ourselves
work();
rec->done.IsSet();
hasher.Update(rec->keys.data(), rec->keys.size()*sizeof(CasKey));
bool error = rec->error;
#Loc: <Workspace>/Engine/Source/Programs/UnrealBuildAccelerator/Common/Private/UbaStorageClient.cpp:1039
Scope (from outer to inner):
file
namespace uba
function bool StorageClient::PopulateCasFromDirsRecursive
lambda-function
Source code excerpt:
}
workManager.AddWork([&, filePath = TString(fullPath.data), name = TString(e.name)]()
{
FileInformation info;
if (!GetFileInformation(info, m_logger, filePath.c_str()))
{
m_logger.Error(TC("Failed to get information for file %s"), filePath.c_str());
return;
#Loc: <Workspace>/Engine/Source/Programs/UnrealBuildAccelerator/Common/Private/UbaStorageProxy.cpp:246
Scope (from outer to inner):
file
namespace uba
function bool StorageProxy::HandleMessage
Source code excerpt:
// Move the additional messages to a job to be able to return this one quickly.
m_server.AddWork([f = &file, segmentCount, this]()
{
SCOPED_WRITE_LOCK(m_largeFileLock, lock);
//TrackWorkScope tws(m_client, TC("SEGMENTS"));
auto& file = *f;
for (u32 i=0; i!=segmentCount; ++i)
{
#Loc: <Workspace>/Engine/Source/Programs/UnrealBuildAccelerator/Common/Private/UbaStorageProxy.cpp:259
Scope (from outer to inner):
file
lambda-function
lambda-function
Source code excerpt:
auto mif = (MessageInFlight*)userData;
mif->error = error;
mif->proxy.m_server.AddWork([mif]() { mif->proxy.HandleReceivedData(*mif); }, 1, TC(""));
}, mif);
if (!res)
{
// TODO: Don't leak mif
mif->error = true;
}
#Loc: <Workspace>/Engine/Source/Programs/UnrealBuildAccelerator/Common/Private/UbaWorkManager.cpp:99
Scope (from outer to inner):
file
namespace uba
function void WorkManagerImpl::AddWork
Source code excerpt:
void WorkManagerImpl::AddWork(const Function<void()>& work, u32 count, const tchar* desc)
{
SCOPED_WRITE_LOCK(m_workLock, lock);
for (u32 i = 0; i != count; ++i)
{
m_work.push_back({ work });
if (m_workTracker.load())
#Loc: <Workspace>/Engine/Source/Programs/UnrealBuildAccelerator/Common/Public/UbaNetworkServer.h:72
Scope (from outer to inner):
file
namespace uba
class class NetworkServer : public WorkManager
Source code excerpt:
//void UnregisterOnConnection(u8 id);
virtual void AddWork(const Function<void()>& work, u32 count, const tchar* desc) override final;
virtual u32 GetWorkerCount() override final;
struct ClientStats
{
u64 send = 0;
u64 recv = 0;
#Loc: <Workspace>/Engine/Source/Programs/UnrealBuildAccelerator/Common/Public/UbaWorkManager.h:16
Scope (from outer to inner):
file
namespace uba
class class WorkManager
Source code excerpt:
{
public:
virtual void AddWork(const Function<void()>& work, u32 count, const tchar* desc) = 0;
virtual u32 GetWorkerCount() = 0;
u32 TrackWorkStart(const tchar* desc);
void TrackWorkEnd(u32 id);
void SetWorkTracker(WorkTracker* workTracker) { m_workTracker = workTracker; }
#Loc: <Workspace>/Engine/Source/Programs/UnrealBuildAccelerator/Common/Public/UbaWorkManager.h:35
Scope (from outer to inner):
file
namespace uba
class class WorkManagerImpl : public WorkManager
Source code excerpt:
WorkManagerImpl(u32 workerCount);
virtual ~WorkManagerImpl();
virtual void AddWork(const Function<void()>& work, u32 count, const tchar* desc) override;
virtual u32 GetWorkerCount() override;
void DoWork(u32 count = 1);
void FlushWork();
private:
struct Worker;