r.Vulkan.WaitForIdleOnSubmit

r.Vulkan.WaitForIdleOnSubmit

#Overview

name: r.Vulkan.WaitForIdleOnSubmit

This variable is created as a Console Variable (cvar).

It is referenced in 6 C++ source files.

#Summary

#Usage in the C++ source code

The purpose of r.Vulkan.WaitForIdleOnSubmit is to control the GPU synchronization behavior in the Vulkan rendering backend of Unreal Engine 5. It allows developers to wait for the GPU to be idle after submitting a command buffer, which can be useful for tracking GPU hangs and debugging graphics-related issues.

This setting variable is primarily used in the Vulkan RHI (Rendering Hardware Interface) subsystem of Unreal Engine 5. It affects the behavior of command buffer submission and GPU synchronization in the Vulkan renderer.

The value of this variable is set through the Unreal Engine console system, using the r.Vulkan.WaitForIdleOnSubmit console variable. It can be set to different values: 0: Do not wait (default) 1: Wait on every submit 2: Wait when submitting an upload buffer 3: Wait when submitting an active buffer (one that has graphics commands)

This variable interacts closely with the associated variable GWaitForIdleOnSubmit, which is an integer that holds the actual value used in the C++ code.

Developers must be aware that enabling this variable (setting it to non-zero values) can significantly impact performance, as it introduces additional GPU synchronization points. It should primarily be used for debugging purposes and not in production builds.

Best practices when using this variable include:

  1. Keep it disabled (set to 0) for normal operation and performance.
  2. Use it temporarily when investigating GPU hangs or rendering issues.
  3. Be aware of the performance impact when enabled, especially when set to wait on every submit (value 1).
  4. Consider using values 2 or 3 for more targeted synchronization if needed.

Regarding the associated variable GWaitForIdleOnSubmit: This is the C++ integer variable that directly corresponds to the console variable r.Vulkan.WaitForIdleOnSubmit. It is used throughout the Vulkan RHI code to determine when to wait for GPU idle states. The variable is checked in various parts of the code, such as in FVulkanCommandListContext::PrepareForCPURead and FVulkanQueue::Submit, to decide whether to perform additional GPU synchronization.

Developers should not modify GWaitForIdleOnSubmit directly in code, but instead use the console variable r.Vulkan.WaitForIdleOnSubmit to control its value. This ensures that the setting can be changed dynamically during runtime and maintains consistency with the engine’s configuration system.

#References in C++ code

#Callsites

This variable is referenced in the following C++ source code:

#Loc: <Workspace>/Engine/Source/Runtime/VulkanRHI/Private/VulkanQueue.cpp:11

Scope: file

Source code excerpt:

int32 GWaitForIdleOnSubmit = 0;
FAutoConsoleVariableRef CVarVulkanWaitForIdleOnSubmit(
	TEXT("r.Vulkan.WaitForIdleOnSubmit"),
	GWaitForIdleOnSubmit,
	TEXT("Waits for the GPU to be idle after submitting a command buffer. Useful for tracking GPU hangs.\n")
	TEXT(" 0: Do not wait(default)\n")
	TEXT(" 1: Wait on every submit\n")
	TEXT(" 2: Wait when submitting an upload buffer\n")
	TEXT(" 3: Wait when submitting an active buffer (one that has gfx commands)\n"),

#Associated Variable and Callsites

This variable is associated with another variable named GWaitForIdleOnSubmit. They share the same value. See the following C++ source code.

#Loc: <Workspace>/Engine/Source/Runtime/VulkanRHI/Private/VulkanCommands.cpp:866

Scope (from outer to inner):

file
function     void FVulkanCommandListContext::PrepareForCPURead

Source code excerpt:


		CommandBufferManager->SubmitActiveCmdBuffer();
		if (!GWaitForIdleOnSubmit)
		{
			// The wait has already happened if GWaitForIdleOnSubmit is set
			CommandBufferManager->WaitForCmdBuffer(CmdBuffer);
		}
	}
}

void FVulkanCommandListContext::RHISubmitCommandsHint()

#Loc: <Workspace>/Engine/Source/Runtime/VulkanRHI/Private/VulkanQueue.cpp:9

Scope: file

Source code excerpt:

#include "VulkanContext.h"

int32 GWaitForIdleOnSubmit = 0;
FAutoConsoleVariableRef CVarVulkanWaitForIdleOnSubmit(
	TEXT("r.Vulkan.WaitForIdleOnSubmit"),
	GWaitForIdleOnSubmit,
	TEXT("Waits for the GPU to be idle after submitting a command buffer. Useful for tracking GPU hangs.\n")
	TEXT(" 0: Do not wait(default)\n")
	TEXT(" 1: Wait on every submit\n")
	TEXT(" 2: Wait when submitting an upload buffer\n")
	TEXT(" 3: Wait when submitting an active buffer (one that has gfx commands)\n"),
	ECVF_Default

#Loc: <Workspace>/Engine/Source/Runtime/VulkanRHI/Private/VulkanQueue.cpp:80

Scope (from outer to inner):

file
function     void FVulkanQueue::Submit

Source code excerpt:

	bool bShouldStall = false;

	if (GWaitForIdleOnSubmit != 0)
	{
		FVulkanCommandBufferManager* CmdBufferMgr = &CmdBuffer->GetOwner()->GetMgr();

		switch(GWaitForIdleOnSubmit)
		{
			default:
				// intentional fall-through
			case 1:
				bShouldStall = true;
				break;

#Loc: <Workspace>/Engine/Source/Runtime/VulkanRHI/Private/VulkanRHIPrivate.h:927

Scope: file

Source code excerpt:


extern int32 GVulkanSubmitAfterEveryEndRenderPass;
extern int32 GWaitForIdleOnSubmit;

// Vendor-specific GPU crash dumps
extern bool GGPUCrashDebuggingEnabled;

#if VULKAN_HAS_DEBUGGING_ENABLED
extern bool GRenderDocFound;

#Loc: <Workspace>/Engine/Source/Runtime/VulkanRHI/Private/VulkanViewport.cpp:272

Scope (from outer to inner):

file
function     void FVulkanViewport::WaitForFrameEventCompletion

Source code excerpt:

			if (LastFrameFenceCounter == LastFrameCommandBuffer->GetFenceSignaledCounter())
			{
				if (!GWaitForIdleOnSubmit)
				{
					// The wait has already happened if GWaitForIdleOnSubmit is set
					LastFrameCommandBuffer->GetOwner()->GetMgr().WaitForCmdBuffer(LastFrameCommandBuffer);
				}
			}
		}
	}
}