MaxActiveDownloads

MaxActiveDownloads

#Overview

name: MaxActiveDownloads

The value of this variable can be defined or overridden in .ini config files. 1 .ini config file referencing this setting variable.

It is referenced in 15 C++ source files.

#Summary

#Usage in the C++ source code

The purpose of MaxActiveDownloads is to control the maximum number of concurrent background HTTP downloads that can be active at any given time in Unreal Engine 5. This setting is used to manage system resources and network bandwidth by limiting the number of simultaneous download operations.

MaxActiveDownloads is primarily used by the Background HTTP subsystem, which is part of the Online module in Unreal Engine 5. It is specifically utilized in the BackgroundHTTP plugin and the AndroidFetchBackgroundDownload plugin for Android platforms.

The value of this variable is typically set in the following ways:

  1. It is initialized with a default value of 4 in the FBackgroundHttpManagerImpl constructor.
  2. It can be configured in the engine configuration file (GEngineIni) under the [BackgroundHttp] section.
  3. It can be dynamically set using the SetMaxActiveDownloads function.

MaxActiveDownloads interacts with other variables such as NumCurrentlyActiveRequests and PendingStartRequests to manage the queue of download requests.

Developers should be aware of the following when using this variable:

  1. Setting it too high may overload system resources or network bandwidth.
  2. Setting it too low may cause downloads to queue up and take longer to complete.
  3. The actual number of active downloads may temporarily exceed MaxActiveDownloads if the limit is lowered while downloads are in progress.

Best practices for using MaxActiveDownloads include:

  1. Consider the target platform’s capabilities when setting this value.
  2. Monitor performance and adjust the value as needed for optimal balance between download speed and system resource usage.
  3. Use the SetMaxActiveDownloads function to dynamically adjust the value based on current network conditions or application state.
  4. Be aware that changing this value may affect the behavior of ongoing downloads and the processing of the download queue.

#Setting Variables

#References In INI files

Location: <Workspace>/Engine/Config/BaseEngine.ini:54, section: [BackgroundHttp]

#References in C++ code

#Callsites

This variable is referenced in the following C++ source code:

#Loc: <Workspace>/Engine/Plugins/Online/Android/AndroidFetchBackgroundDownload/Source/AndroidFetchBackgroundDownload/Private/AndroidPlatformBackgroundHttpManager.cpp:348

Scope (from outer to inner):

file
function     void FAndroidPlatformBackgroundHttpManager::ActivatePendingRequests

Source code excerpt:

				WorkParams.AddDataToWorkerParameters(FAndroidNativeDownloadWorkerParameterKeys::DOWNLOAD_DESCRIPTION_LIST_KEY, FileNameForDownloadDescList);
					
				//Set our MaxActiveDownloads in the underlying java layer to match our expectation
				WorkParams.AddDataToWorkerParameters(FAndroidNativeDownloadWorkerParameterKeys::DOWNLOAD_MAX_CONCURRENT_REQUESTS_KEY, MaxActiveDownloads);

				WorkParams.AddDataToWorkerParameters(FAndroidNativeDownloadWorkerParameterKeys::CELLULAR_HANDLE_KEY, CVarEnableCellularHandling.GetValueOnGameThread());

				//Make sure we pass in localized notification text bits for the important worker keys
				WorkParams.AddDataToWorkerParameters(FAndroidNativeDownloadWorkerParameterKeys::NOTIFICATION_CONTENT_TITLE_KEY, AndroidBackgroundHTTPManagerDefaultLocalizedText.DefaultNotificationText_Title.GetText());
				WorkParams.AddDataToWorkerParameters(FAndroidNativeDownloadWorkerParameterKeys::NOTIFICATION_CONTENT_COMPLETE_TEXT_KEY, AndroidBackgroundHTTPManagerDefaultLocalizedText.DefaultNotificationText_Complete.GetText());

#Loc: <Workspace>/Engine/Source/Runtime/Online/BackgroundHTTP/Private/BackgroundHttpManagerImpl.cpp:29

Scope (from outer to inner):

file
function     FBackgroundHttpManagerImpl::FBackgroundHttpManagerImpl

Source code excerpt:

	, ActiveRequestLock()
	, NumCurrentlyActiveRequests(0)
	, MaxActiveDownloads(4)
	, FileHashHelper(MakeShared<FBackgroundHttpFileHashHelper, ESPMode::ThreadSafe>())
{
}

FBackgroundHttpManagerImpl::~FBackgroundHttpManagerImpl()
{

#Loc: <Workspace>/Engine/Source/Runtime/Online/BackgroundHTTP/Private/BackgroundHttpManagerImpl.cpp:47

Scope (from outer to inner):

file
function     void FBackgroundHttpManagerImpl::Initialize

Source code excerpt:

	// Can't read into an atomic int directly
	int MaxActiveDownloadsConfig = 4;
	ensureAlwaysMsgf(GConfig->GetInt(TEXT("BackgroundHttp"), TEXT("MaxActiveDownloads"), MaxActiveDownloadsConfig, GEngineIni), TEXT("No value found for MaxActiveDownloads! Defaulting to 4!"));
	MaxActiveDownloads = MaxActiveDownloadsConfig;
}

void FBackgroundHttpManagerImpl::Shutdown()
{
	//Pending Requests Clear
	{

#Loc: <Workspace>/Engine/Source/Runtime/Online/BackgroundHTTP/Private/BackgroundHttpManagerImpl.cpp:288

Scope (from outer to inner):

file
function     int FBackgroundHttpManagerImpl::GetMaxActiveDownloads

Source code excerpt:

int FBackgroundHttpManagerImpl::GetMaxActiveDownloads() const
{
	return MaxActiveDownloads;
}

void FBackgroundHttpManagerImpl::SetMaxActiveDownloads(int InMaxActiveDownloads)
{
	MaxActiveDownloads = InMaxActiveDownloads;
}

void FBackgroundHttpManagerImpl::AddRequest(const FBackgroundHttpRequestPtr Request)
{
	UE_LOG(LogBackgroundHttpManager, VeryVerbose, TEXT("AddRequest Called - RequestID:%s"), *Request->GetRequestID());

#Loc: <Workspace>/Engine/Source/Runtime/Online/BackgroundHTTP/Private/BackgroundHttpManagerImpl.cpp:411

Scope (from outer to inner):

file
function     bool FBackgroundHttpManagerImpl::Tick

Source code excerpt:


	ensureAlwaysMsgf(IsInGameThread(), TEXT("Called from un-expected thread! Potential error in an implementation of background downloads!"));
	CSV_CUSTOM_STAT(BackgroundDownload, MaxActiveDownloads, MaxActiveDownloads, ECsvCustomStatOp::Set);
	CSV_CUSTOM_STAT(BackgroundDownload, PendingStartRequests, PendingStartRequests.Num(), ECsvCustomStatOp::Set);
	CSV_CUSTOM_STAT(BackgroundDownload, NumCurrentlyActiveRequests, NumCurrentlyActiveRequests, ECsvCustomStatOp::Set);
	ActivatePendingRequests();
	
	//for now we are saving data every tick, could change this to be on an interval later if required
	GetFileHashHelper()->SaveData();

#Loc: <Workspace>/Engine/Source/Runtime/Online/BackgroundHTTP/Private/BackgroundHttpManagerImpl.cpp:432

Scope (from outer to inner):

file
function     void FBackgroundHttpManagerImpl::ActivatePendingRequests

Source code excerpt:

		FRWScopeLock ActiveScopeLock(ActiveRequestLock, SLT_ReadOnly);
		FRWScopeLock PendingScopeLock(PendingRequestLock, SLT_ReadOnly);
		const int NumRequestsWeCanProcess = (MaxActiveDownloads - NumCurrentlyActiveRequests);
		if (NumRequestsWeCanProcess > 0)
		{
			if (PendingStartRequests.Num() > 0)
			{
				UE_LOG(LogBackgroundHttpManager, Verbose, TEXT("Populating Requests to Start from PendingStartRequests - MaxActiveDownloads:%d | NumCurrentlyActiveRequests:%d | NumPendingStartRequests:%d"), MaxActiveDownloads.Load(), NumCurrentlyActiveRequests, PendingStartRequests.Num());

				HighestPriorityRequestToStart = PendingStartRequests[0];
				HighestRequestPriority = HighestPriorityRequestToStart.IsValid() ? HighestPriorityRequestToStart->GetRequestPriority() : EBackgroundHTTPPriority::Num;

				//See how many more requests we can process and only do anything if we can still process more
				{

#Loc: <Workspace>/Engine/Source/Runtime/Online/BackgroundHTTP/Private/IOS/ApplePlatformBackgroundHttpManager.cpp:42

Scope (from outer to inner):

file
function     FApplePlatformBackgroundHttpManager::FApplePlatformBackgroundHttpManager

Source code excerpt:

    , RequestsPendingRemove()
    , NumCurrentlyActiveTasks(0)
	, MaxNumActualTasks(MaxActiveDownloads)
{
}

void FApplePlatformBackgroundHttpManager::Initialize()
{
	//Initialize UnAssociatedTasks

#Loc: <Workspace>/Engine/Source/Runtime/Online/BackgroundHTTP/Private/IOS/ApplePlatformBackgroundHttpManager.cpp:265

Scope (from outer to inner):

file
function     void FApplePlatformBackgroundHttpManager::PopulateUnAssociatedTasks

Source code excerpt:

	FBackgroundHttpManagerImpl::SetMaxActiveDownloads(InMaxActiveDownloads);

	// It's possible we have more than the new maximum active right now, so we gracefully reduce MaxNumActualTasks down to MaxActiveDownloads as the extra tasks finish.
	MaxNumActualTasks = FMath::Max(MaxActiveDownloads.Load(), FPlatformAtomics::AtomicRead(&NumCurrentlyActiveTasks));
}

bool FApplePlatformBackgroundHttpManager::AssociateWithAnyExistingUnAssociatedTasks(const FBackgroundHttpRequestPtr Request)
{
    bool bDidAssociateWithUnAssociatedTask = false;
    

#Loc: <Workspace>/Engine/Source/Runtime/Online/BackgroundHTTP/Private/IOS/ApplePlatformBackgroundHttpManager.cpp:608

Scope (from outer to inner):

file
function     void FApplePlatformBackgroundHttpManager::FinishRequest

Source code excerpt:

                    int NumActualTasks = FPlatformAtomics::InterlockedDecrement(&NumCurrentlyActiveTasks);

					// Handle the case that SetMaxActiveDownloads reduced the MaxActiveDownloads while we had more than the new maximum in progress.
					MaxNumActualTasks = FMath::Max(MaxNumActualTasks - 1, MaxActiveDownloads.Load());

                    //Sanity check that our data is valid. Shouldn't ever trip if everything is working as intended.
                    const bool bNumActualTasksIsValid = ((NumActualTasks >= 0) && (NumActualTasks <= MaxNumActualTasks));
                    
                    UE_LOG(LogBackgroundHttpManager,Display, TEXT("Finishing Request lowering Task Count: %d"), NumActualTasks);
                    

#Loc: <Workspace>/Engine/Source/Runtime/Online/BackgroundHTTP/Private/IOS/ApplePlatformBackgroundHttpManager.cpp:843

Scope (from outer to inner):

file
function     void FApplePlatformBackgroundHttpManager::TickRequests

Source code excerpt:

		//Check to make sure we have room for more tasks to be active first
		int CurrentCount = FPlatformAtomics::AtomicRead(&NumCurrentlyActiveTasks);
		bool bNeedsMoreTasks = (CurrentCount < MaxActiveDownloads);
		FAppleBackgroundHttpRequestPtr FoundRequestToStart = nullptr;
		
        FRWScopeLock ScopeLock(ActiveRequestLock, SLT_ReadOnly);
        for (FBackgroundHttpRequestPtr& Request : ActiveRequests)
        {
            FAppleBackgroundHttpRequestPtr AppleRequest = StaticCastSharedPtr<FApplePlatformBackgroundHttpRequest>(Request);

#Loc: <Workspace>/Engine/Source/Runtime/Online/BackgroundHTTP/Private/IOS/ApplePlatformBackgroundHttpManager.cpp:912

Scope (from outer to inner):

file
function     void FApplePlatformBackgroundHttpManager::TickRequests

Source code excerpt:

			const int32 OldTotal = FPlatformAtomics::InterlockedIncrement(&NumCurrentlyActiveTasks);
			
			if (OldTotal <= MaxActiveDownloads)
			{
				UE_LOG(LogBackgroundHttpManager, Display, TEXT("Starting Task for Request -- RequestDebugID:%s | CurrentlyActiveRequests:%d"), *(FoundRequestToStart->GetRequestDebugID()), OldTotal);
				FoundRequestToStart->ActivateUnderlyingTask();
			}
			else
			{

#Loc: <Workspace>/Engine/Source/Runtime/Online/BackgroundHTTP/Public/BackgroundHttpManagerImpl.h:38

Scope (from outer to inner):

file
class        class FBackgroundHttpManagerImpl : public IBackgroundHttpManager , public FTSTickerObjectBase

Source code excerpt:

	
	BACKGROUNDHTTP_API virtual int GetMaxActiveDownloads() const override;
	BACKGROUNDHTTP_API virtual void SetMaxActiveDownloads(int MaxActiveDownloads) override;

	
	BACKGROUNDHTTP_API virtual FString GetTempFileLocationForURL(const FString& URL) override;
	
	BACKGROUNDHTTP_API virtual void CleanUpDataAfterCompletingRequest(const FBackgroundHttpRequestPtr Request) override;
	BACKGROUNDHTTP_API virtual void SetCellularPreference(int32 Value) override;

#Loc: <Workspace>/Engine/Source/Runtime/Online/BackgroundHTTP/Public/BackgroundHttpManagerImpl.h:88

Scope (from outer to inner):

file
class        class FBackgroundHttpManagerImpl : public IBackgroundHttpManager , public FTSTickerObjectBase

Source code excerpt:

	/** Count of how many requests we have active **/
	volatile int NumCurrentlyActiveRequests;
	TAtomic<int> MaxActiveDownloads;
	
private:
	BackgroundHttpFileHashHelperRef FileHashHelper;
};

#Loc: <Workspace>/Engine/Source/Runtime/Online/BackgroundHTTP/Public/IOS/ApplePlatformBackgroundHttpManager.h:26

Scope (from outer to inner):

file
class        class FApplePlatformBackgroundHttpManager : public FBackgroundHttpManagerImpl

Source code excerpt:

	virtual void RemoveRequest(const FBackgroundHttpRequestPtr Request) override;
	virtual bool IsGenericImplementation() const override { return false;  }
	virtual void SetMaxActiveDownloads(int MaxActiveDownloads) override;

	/**
	* Constructor
	*/
	FApplePlatformBackgroundHttpManager();

#Loc: <Workspace>/Engine/Source/Runtime/Online/BackgroundHTTP/Public/Interfaces/IBackgroundHttpManager.h:60

Scope (from outer to inner):

file
class        class IBackgroundHttpManager : public TSharedFromThis<IBackgroundHttpManager>

Source code excerpt:

	* Function that sets how many active BackgroundHttpRequests we should have actively downloading at once.
	*
	* @param MaxActiveDownloads the maximum number of downloads that should be active at once
	*/
	virtual void SetMaxActiveDownloads(int MaxActiveDownloads) = 0;

	/**
	 * Function that returns an FString fullpath where we would expect the given URL's temp file to be located
	 *
	 */
	virtual FString GetTempFileLocationForURL(const FString& URL) = 0;
	
	/**