BackendGraph

BackendGraph

#Overview

name: BackendGraph

The value of this variable can be defined or overridden in .ini config files. 1 .ini config file referencing this setting variable.

It is referenced in 3 C++ source files.

#Summary

#Usage in the C++ source code

The purpose of BackendGraph is to configure the backend hierarchy for the content virtualization system in Unreal Engine 5. It is used to define which backends should be mounted and in what order for handling virtualized content.

This setting variable is primarily used by the Virtualization subsystem of Unreal Engine 5, specifically within the VirtualizationManager module.

The value of this variable is set in the configuration files, typically in the [Core.ContentVirtualization] section. It can also be overridden from the command line using the ‘BackendGraph=FooBar’ syntax, where FooBar is the name of the desired graph.

BackendGraph interacts with other variables in the backend graph configuration, such as ‘Hierarchy’, which describes the order of backend mounting, and individual backend entries that specify their type and additional customization parameters.

Developers must be aware that:

  1. If not set, the default value ‘ContentVirtualizationBackendGraph_None’ will be used, which effectively disables the virtualization system.
  2. The graph name specified by BackendGraph should have a corresponding configuration section defining the hierarchy and individual backend details.
  3. Each backend in the hierarchy requires its own entry in the graph with at least a ‘Type’ parameter.

Best practices when using this variable include:

  1. Defining a project-specific backend graph that suits the project’s needs.
  2. Ensuring that all referenced backends in the hierarchy have proper configurations.
  3. Using meaningful names for the backend graph to easily identify its purpose.
  4. Considering the order of backends in the hierarchy, as it affects the priority of content retrieval.
  5. Properly configuring each backend with necessary parameters (e.g., providing a path for FileSystem backends).

#Setting Variables

#References In INI files

Location: <Workspace>/Engine/Config/BaseEngine.ini:1469, section: [Core.VirtualizationModule]

#References in C++ code

#Callsites

This variable is referenced in the following C++ source code:

#Loc: <Workspace>/Engine/Source/Developer/Virtualization/Private/VirtualizationManager.cpp:1062

Scope (from outer to inner):

file
namespace    UE::Virtualization
function     void FVirtualizationManager::ApplySettingsFromConfigFiles

Source code excerpt:

		if (!ConfigFile.GetString(LegacyConfigSection, TEXT("BackendGraph"), BackendGraphName))
		{
			ConfigFile.GetString(ConfigSection, TEXT("BackendGraph"), BackendGraphName);
		}

		UE_LOG(LogVirtualization, Display, TEXT("\tBackendGraphName : %s"), *BackendGraphName);
	}

	{

#Loc: <Workspace>/Engine/Source/Developer/Virtualization/Private/VirtualizationManager.h:16

Scope: file

Source code excerpt:

 * Configuring the backend hierarchy
 * 
 * The [Core.ContentVirtualization] section can contain a string 'BackendGraph' which will set with the name of  
 * the backend graph, if not set then the default 'ContentVirtualizationBackendGraph_None' will be used instead. 
 * This value can also be overridden from the command line by using 'BackendGraph=FooBar' where FooBar is the 
 * name of the graph.
 * 
 * The first entry in the graph to be parsed will be the 'Hierarchy' which describes which backends should be
 * mounted and in which order. For example 'Hierarchy=(Entry=Foo, Entry=Bar)' which should mount two backends
 * 'Foo' and 'Bar' in that order. 
 * 
 * Each referenced backend in the hierarchy will then require it's own entry in the graph where the key will be
 * it's name in the hierarchy and the value a string describing how to set it up. 
 * The value must contain 'Type=X' where X is the name used to find the correct IVirtualizationBackendFactory 
 * to create the backend with. 
 * Once the backend is created then reset of the string will be passed to it, so that additional customization
 * can be extracted. Depending on the backend implementation these values may or may not be required.
 * 
 * Example graph:
 * [ContentVirtualizationBackendGraph_Example] 
 * Hierarchy=(Entry=MemoryCache, Entry=NetworkShare)
 * MemoryCache=(Type=InMemory)
 * NetworkShare=(Type=FileSystem, Path="\\path\to\somewhere")
 * 
 * The graph is named 'ContentVirtualizationBackendGraph_Example'.
 * The hierarchy contains two entries 'InMemory' and 'NetworkShare' to be mounted in that order
 * MemoryCache creates a backend of type 'InMemory' and has no additional customization
 * NetworkShare creates a backend of type 'FileSystem' and provides an additional path, the filesystem backend would 
 * fatal error without this value.
 */

/**
 * Filtering
 * 
 * By default all packages in a project can be virtualized once the system has been enabled. The can be
 * overridden by the filtering system, either by excluding specific packages/directories, or by changing
 * the default so that no package will be virtualized except packages/directories that have been 
 * specifically marked as to be virtualized.
 * 
 * Note that the filtering is applied when a package is saved and stored as meta data in the FPackageTrailer.
 * This means that you can scan your package files and reason about the behavior of the payloads but also

#Loc: <Workspace>/Engine/Source/Developer/Virtualization/Private/VirtualizationManager.h:100

Scope: file

Source code excerpt:

 *												virtualization. Use this to strike a balance between disk space and the number of smaller
												payloads in your project being virtualized. [Default=0]
 * BackendGraph [string]:						The name of the backend graph to use. The default graph has no backends and effectively
												disables the system. It is expected that a project will define the graph that it wants
												and then set this option [Default=ContentVirtualizationBackendGraph_None]
 * VirtualizationProcessTag [string]:			The tag to be applied to any set of packages that have had  the virtualization process run
 *												on them. Typically this means appending the tag to the description of a changelist of 
 *												packages. This value can be set to an empty string. [Default="#virtualized"]
 * AllowSubmitIfVirtualizationFailed [bool]:	Revision control submits that trigger the virtualization system can either allow or block
 *												the submit if the virtualization process fails based on this value. True will allow a
 *												submit with an error to continue and false will block the submit. Note that by error we mean
 *												that the packages were not virtualized, not that bad data was produced. [Default=false]
 * LazyInitConnections [bool]:					When true, backends will not attempt to connect to their services until actually required.
 *												This can remove lengthy connection steps from the process init phase and then only connect
 *												if we actually need that service. Note that if this is true then the connection can come from
 *												any thread, so custom backend code will need to take that into account. [Default=false]
 * DisableLazyInitIfInteractive [bool]			When true 'LazyInitConnections' will be forced to false if slate is enabled. This exists because
 *												some backends can show slate dialogs when their initial connection fails to prompt for the 
 *												correct login values. When 'LazyInitConnections' is true, this request can come on any thread and
 *												trying to marshal the slate request to the gamethread can often introduce thread locks. Setting
 *												both this and 'LazyInitConnections' to true will allow tools that do not use slate to initialize
 *												the VA connections on use, but force tools that can display the slate dialog to initialized 
 *												during preinit on the game thread so that the dialog can be shown. Note that this only overrides
												the setting of 'LazyInitConnections' via the config file, not cvar, cmdline or code [Default=false]
 * UseLegacyErrorHandling [bool]:				Controls how we deal with errors encountered when pulling payloads. When true a failed payload 
 *												pull will return an error and allow the process to carry on (the original error handling logic)
 *												and when false a dialog will be displayed to the user warning them about the failed pull and 
 *												prompting them to retry the pull or to quit the process. [Default=true]
 * PullErrorAdditionalMsg [string]				An additional message that will be added to the error dialog presented on payload pull failure.
 *												This allows you to add custom information, such as links to internal help docs without editing
 *												code. Note that this additional message only works with the error dialog and will do nothing
 *												if 'UseLegacyErrorHandling' is true. [Default=""]
 * ForceCachingOnPull [bool]:					When true backends will be told to always upload the payload when a caching as a result of 
 *												a payload pull as in this scenario we already know that the backend failed to pull the payload
 *												before it was pulled from a backend later in the hierarchy. Can be used to try and skip
 *												expensive existence checks, or if a backend is in a bad state where it believes it has the payload
 *												but is unable to actually return the data. [Default=false]
 * UnattendedRetryCount [int32]:				How many times the process should retry pulling payloads after a failure is encountered if the
												process is unattended. Usually when a payload pull fails we ask the user to try and fix the issue
												and retry, but in unattended mode we just log an error and terminate the process. In some cases
												such as build machines with unreliable internet it is possible that the process could recover in
												which case setting this value might help. Zero or negative values will disable the system. 
												Note: If many pulls are occurring at the same time on many threads a very short network outage might
												spawn many errors in which case we try to group these errors into a single 'try' so 32 errors on 32
												threads would not immediately blow past a retry count of 30 for example. [Default=0]
 * UnattendedRetryTimer [int32]					If 'UnattendedRetryCount' is set to a positive value then this value sets how long (in seconds)
 *												the process should wait after a failure is encountered before retrying the pull. Depending on the
												likely cause of the failure you may want to set this value to several minutes. [Default=0]
 */

namespace UE::Virtualization
{
class FPullRequestCollection;

/** The default mode of filtering to use with package paths that do not match entries in UVirtualizationFilterSettings */
enum class EPackageFilterMode : uint8
{
	/** Packages will be virtualized by default and must be opted out by the use of UVirtualizationFilterSettings::ExcludePackagePaths */
	OptOut = 0,
	/** Packages will not be virtualized by default and must be opted in by the user of UVirtualizationFilterSettings::IncludePackagePaths */
	OptIn
};

/** Attempt to convert a string buffer to EPackageFilterMode */
bool LexTryParseString(EPackageFilterMode& OutValue, FStringView Buffer);

/** This is used as a wrapper around the various potential back end implementations. 
	The calling code shouldn't need to care about which back ends are actually in use. */
class FVirtualizationManager final : public IVirtualizationSystem