MaxChunkSize

MaxChunkSize

#Overview

name: MaxChunkSize

The value of this variable can be defined or overridden in .ini config files. 1 .ini config file referencing this setting variable.

It is referenced in 49 C++ source files.

#Summary

#Usage in the C++ source code

The purpose of MaxChunkSize is to define the maximum size of a chunk in various subsystems of Unreal Engine 5, particularly in areas related to streaming, memory management, and data processing. Here’s a summary of its usage and implications:

  1. It’s used in multiple Unreal Engine subsystems, including audio streaming, asset packaging, GPU memory allocation, scene culling, and data compression.

  2. In the audio streaming system, MaxChunkSize determines the maximum size of audio chunks that can be loaded and processed at once. This affects memory usage and streaming performance.

  3. For asset packaging and streaming, MaxChunkSize influences how large asset chunks can be, which impacts loading times and memory usage.

  4. In GPU memory management, MaxChunkSize is used to define the maximum size of memory chunks that can be allocated at once.

  5. For scene culling, MaxChunkSize determines the maximum number of instances that can be processed in a single chunk, affecting rendering performance.

  6. The value of MaxChunkSize is often configurable and can be set differently for various platforms or use cases.

  7. It’s typically set in bytes or as a count of elements, depending on the specific use case.

  8. Developers need to be aware that changing MaxChunkSize can have significant impacts on performance, memory usage, and load times.

  9. In some cases, MaxChunkSize is used to ensure that data chunks don’t exceed certain size limits, which might be important for specific hardware or software constraints.

  10. Best practices include carefully tuning MaxChunkSize based on the target hardware and specific requirements of the game or application.

When using this variable, developers should consider the trade-offs between larger chunk sizes (which can improve streaming performance but increase memory usage) and smaller chunk sizes (which can reduce memory usage but might increase overhead). The optimal value often depends on the specific use case and target platform.

#Setting Variables

#References In INI files

Location: <Workspace>/Projects/Lyra/Config/DefaultGame.ini:101, section: [/Script/UnrealEd.ProjectPackagingSettings]

#References in C++ code

#Callsites

This variable is referenced in the following C++ source code:

#Loc: <Workspace>/Engine/Plugins/Animation/ACLPlugin/Source/ACLPlugin/Private/AnimationCompressionLibraryDatabase.cpp:804

Scope (from outer to inner):

file
function     static uint32 CalculateNumChunksToStream

Source code excerpt:


	const uint32 MaxStreamRequestSize = MaxStreamRequestSizeKB * 1024;
	const uint32 MaxChunkSize = Database.get_max_chunk_size();
	return FMath::Max<uint32>(MaxStreamRequestSize / MaxChunkSize, 1);	// Must stream at least one chunk
}

/**
* Visual fidelity is updated through this latent ticker. Depending on the current database fidelity,
* we determine whether we need to stream in/out or do nothing.
* If a fidelity change is ongoing and a new value is requested, it will be queued. Subsequent changes

#Loc: <Workspace>/Engine/Plugins/Experimental/Mutable/Source/CustomizableObject/Internal/MuCO/CustomizableObjectSystemPrivate.h:777

Scope (from outer to inner):

file
class        class UCustomizableObjectSystemPrivate : public UObject, public IStreamingManager

Source code excerpt:

	TWeakPtr<SNotificationItem> UncompiledCustomizableObjectsNotificationPtr;

	/** Map used to cache per platform MaxChunkSize. If MaxChunkSize > 0, streamed data will be split in multiple files */
	TMap<FString, int64> PlatformMaxChunkSize;
#endif
};


namespace impl

#Loc: <Workspace>/Engine/Plugins/Experimental/Mutable/Source/CustomizableObject/Private/MuCO/CustomizableObject.cpp:2113

Scope (from outer to inner):

file
function     void UCustomizableObjectBulk::PrepareBulkData

Source code excerpt:


	uint64 TargetBulkDataFileBytes = CustomizableObject->CompileOptions.PackagedDataBytesLimit;
	const uint64 MaxChunkSize = UCustomizableObjectSystem::GetInstance()->GetMaxChunkSizeForPlatform(TargetPlatform);
	TargetBulkDataFileBytes = FMath::Min(TargetBulkDataFileBytes, MaxChunkSize);


	// TODO:
	// To avoid influence of the order of the streamed data (their index), classify it recursively based on hash values
	// until the tree leaves have either a single block, or a sum of blocks below the desired file size.
	struct FClassifyNode

#Loc: <Workspace>/Engine/Plugins/Experimental/Mutable/Source/CustomizableObject/Private/MuCO/CustomizableObjectSystem.cpp:3985

Scope (from outer to inner):

file
function     uint64 UCustomizableObjectSystem::GetMaxChunkSizeForPlatform

Source code excerpt:

	}

	int64 MaxChunkSize = -1;

	if (!FParse::Value(FCommandLine::Get(), TEXT("ExtraFlavorChunkSize="), MaxChunkSize) || MaxChunkSize < 0)
	{
		FConfigFile PlatformIniFile;
		FConfigCacheIni::LoadLocalIniFile(PlatformIniFile, TEXT("Game"), true, *PlatformName);
		FString ConfigString;
		if (PlatformIniFile.GetString(TEXT("/Script/UnrealEd.ProjectPackagingSettings"), TEXT("MaxChunkSize"), ConfigString))
		{
			MaxChunkSize = FCString::Atoi64(*ConfigString);
		}
	}

	// If no limit is specified default it to MUTABLE_STREAMED_DATA_MAXCHUNKSIZE 
	if (MaxChunkSize <= 0)
	{
		MaxChunkSize = MUTABLE_STREAMED_DATA_MAXCHUNKSIZE;
	}

	GetPrivate()->PlatformMaxChunkSize.Add(PlatformName, MaxChunkSize);

	return MaxChunkSize;
}

#endif // WITH_EDITOR


void UCustomizableObjectSystem::CacheImage(FName ImageId)

#Loc: <Workspace>/Engine/Plugins/Media/AppleProResMedia/Source/AppleProResMedia/Private/AppleProResEncoder/AppleProResEncoder.cpp:403

Scope (from outer to inner):

file
function     bool FAppleProResEncoder::InitializeVideoTrack

Source code excerpt:

	
	// A reccomended chunk size for video codecs is 4MiB for HD codecs.
	const int32 MaxChunkSize = 4 * 1024 * 1024;
	const int32 MaxFrameCount = FMath::Max(MaxChunkSize / TargetCompressedFrameSize, 1); // Figure out how many frames per chunk will fit into 4MiB

	PRStatus Status = ProResFileWriterSetTrackPreferredChunkSize(FileWriter, VideoTrackId, MaxFrameCount * TargetCompressedFrameSize);
	if (Status != 0)
	{
		UE_LOG(LogAppleProResMedia, Error, TEXT("Failed to set preferred chunk size for video track for file writer at path: \"%s\"."), *Options.OutputFilename);
		return false;

#Loc: <Workspace>/Engine/Plugins/Media/AppleProResMedia/Source/AppleProResMedia/Private/AppleProResEncoderProtocol.cpp:149

Scope (from outer to inner):

file
function     bool UAppleProResEncoderProtocol::CreateProResFile

Source code excerpt:

	PRGetCompressedFrameSize(GetSelectedCodecType(), true, Dimensions.width, Dimensions.height, &MaxCompressedFrameSize, &TargetCompressedFrameSize);

	const int32 MaxChunkSize = 4 * 1024 * 1024;
	const int32 MaxFrameCount = FMath::Max(MaxChunkSize / TargetCompressedFrameSize, 1);

	ProResFileWriterSetTrackPreferredChunkSize(FileWriter, VideoTrackID, MaxFrameCount * TargetCompressedFrameSize);

	const ProResFormatDescriptionColorPrimaries ColorPrimaries = GetColorPrimaries();
	const ProResFormatDescriptionTransferFunction TransferFunction = GetTransferFunction();
	const ProResFormatDescriptionYCbCrMatrix Matrix = GetYCbCrMatrix();

#Loc: <Workspace>/Engine/Plugins/VirtualProduction/Rivermax/RivermaxCore/Source/RivermaxCore/Private/RivermaxInputStream.cpp:246

Scope (from outer to inner):

file
namespace    UE::RivermaxCore::Private
function     void FRivermaxInputStream::Process_AnyThread

Source code excerpt:

	{
		const size_t MinChunkSize = 0;
		const size_t MaxChunkSize = 5000;
		const int Timeout = 0;
		const int Flags = 0;
		const rmx_input_completion* Completion = nullptr;
		
		checkSlow(CachedAPI);
		rmx_status Status = CachedAPI->rmx_input_get_next_chunk(&BufferConfiguration.ChunkHandle);

#Loc: <Workspace>/Engine/Plugins/VirtualProduction/Rivermax/RivermaxCore/Source/RivermaxCore/Private/RivermaxInputStream.cpp:1240

Scope (from outer to inner):

file
namespace    UE::RivermaxCore::Private
function     bool FRivermaxInputStream::FinalizeStreamCreation

Source code excerpt:

		// Configure how we wnat to consume chunks
		constexpr size_t MinChunkSize = 0;
		constexpr size_t MaxChunkSize = 5000;
		constexpr int Timeout = 0;
		Status = CachedAPI->rmx_input_set_completion_moderation(StreamId, MinChunkSize, MaxChunkSize, Timeout);
		if (Status != RMX_OK)
		{
			OutErrorMessage = FString::Printf(TEXT("Could not setup expected packet count for stream. Status: %d."), Status);
			return false;
		}

#Loc: <Workspace>/Engine/Source/Developer/AudioFormatADPCM/Private/AudioFormatADPCM.cpp:526

Scope (from outer to inner):

file
class        class FAudioFormatADPCM : public IAudioFormat
function     virtual bool SplitDataForStreaming

Source code excerpt:

	}

	virtual bool SplitDataForStreaming(const TArray<uint8>& SrcBuffer, TArray<TArray<uint8>>& OutBuffers, const int32 InitialMaxChunkSize, const int32 MaxChunkSize) const override
	{
		uint8 const*	SrcData = SrcBuffer.GetData();
		int32			SrcSize = SrcBuffer.Num();
		uint32			BytesProcessed = 0;
		
		FWaveModInfo	WaveInfo;

#Loc: <Workspace>/Engine/Source/Developer/AudioFormatADPCM/Private/AudioFormatADPCM.cpp:569

Scope (from outer to inner):

file
class        class FAudioFormatADPCM : public IAudioFormat
function     virtual bool SplitDataForStreaming

Source code excerpt:

				{
					// Start a new chunk with the reserve memory for the max chunk size
					AddNewChunk(OutBuffers, MaxChunkSize);
					ThisChunkMaxSize = MaxChunkSize;
					CurChunkDataSize = 0;
				}

				// Always add chunks in NumChannels pairs
				for (int32 ChannelIndex = 0; ChannelIndex < NumChannels; ++ChannelIndex)
				{

#Loc: <Workspace>/Engine/Source/Developer/AudioFormatADPCM/Private/AudioFormatADPCM.cpp:618

Scope (from outer to inner):

file
class        class FAudioFormatADPCM : public IAudioFormat
function     virtual bool SplitDataForStreaming

Source code excerpt:

				if (SrcSize > 0)
				{
					DataLeftInCurChunk = MaxChunkSize;
					AddNewChunk(OutBuffers, MaxChunkSize);
				}
			}
		}
		else
		{
			return false;

#Loc: <Workspace>/Engine/Source/Developer/AudioFormatOgg/Private/AudioFormatOgg.cpp:463

Scope (from outer to inner):

file
class        class FAudioFormatOgg : public IAudioFormat
function     virtual bool SplitDataForStreaming

Source code excerpt:

	}

	virtual bool SplitDataForStreaming(const TArray<uint8>& SrcBuffer, TArray<TArray<uint8>>& OutBuffers, const int32 InitialChunkMaxSize, const int32 MaxChunkSize) const override
	{
		// Just chunk purely on MONO_PCM_BUFFER_SIZE
		uint8 const*	SrcData = SrcBuffer.GetData();
		uint32			SrcSize = SrcBuffer.Num();

		// Load the audio quality info to get the number of channels

#Loc: <Workspace>/Engine/Source/Developer/AudioFormatOgg/Private/AudioFormatOgg.cpp:489

Scope (from outer to inner):

file
class        class FAudioFormatOgg : public IAudioFormat
function     virtual bool SplitDataForStreaming

Source code excerpt:

		SrcData += CurChunkDataSize;

		ChunkSize = MaxChunkSize;

		// Set up the rest of the chunks:
		while(SrcSize > 0)
		{
			CurChunkDataSize = FMath::Min<uint32>(SrcSize, ChunkSize);

#Loc: <Workspace>/Engine/Source/Developer/AudioFormatOpus/Private/AudioFormatOpus.cpp:376

Scope (from outer to inner):

file
class        class FAudioFormatOpus : public IAudioFormat
function     bool SplitDataForStreaming

Source code excerpt:

	}

	bool SplitDataForStreaming(const TArray<uint8>& SrcBuffer, TArray<TArray<uint8>>& OutBuffers, const int32 MaxInitialChunkSize, const int32 MaxChunkSize) const override
	{
		// This should not be called if we require a streaming seek-table. 
		if (!ensure(!RequiresStreamingSeekTable()))
		{
			return false;
		}

#Loc: <Workspace>/Engine/Source/Developer/AudioFormatOpus/Private/AudioFormatOpus.cpp:417

Scope (from outer to inner):

file
class        class FAudioFormatOpus : public IAudioFormat
function     bool SplitDataForStreaming

Source code excerpt:

			ProcessedFrames++;

			ChunkSize = MaxChunkSize;
		}
		if (WriteOffset < ReadOffset)
		{
			WriteOffset += AddDataChunk(OutBuffers, LockedSrc + WriteOffset, ReadOffset - WriteOffset);
		}

#Loc: <Workspace>/Engine/Source/Developer/DeveloperToolSettings/Classes/Settings/ProjectPackagingSettings.h:305

Scope (from outer to inner):

file
class        class UProjectPackagingSettings : public UObject

Source code excerpt:

	 */
	UPROPERTY(config, EditAnywhere, Category = Packaging, AdvancedDisplay)
	int64 MaxChunkSize;

	/** 
	 * If enabled, will generate data for HTTP Chunk Installer. This data can be hosted on webserver to be installed at runtime. Requires "Generate Chunks" enabled.
	 */
	UPROPERTY(config, EditAnywhere, Category=Packaging)
	bool bBuildHttpChunkInstallData;

#Loc: <Workspace>/Engine/Source/Developer/Horde/Private/Compute/AgentMessage.cpp:130

Scope (from outer to inner):

file
function     void FAgentMessageChannel::Blob

Source code excerpt:

void FAgentMessageChannel::Blob(const unsigned char* Data, size_t Length)
{
	const size_t MaxChunkSize = ChannelBuffers->Writer.GetChunkMaxLength() - 128 - MessageHeaderLength;
	for (size_t ChunkOffset = 0; ChunkOffset < Length;)
	{
		size_t ChunkLength = std::min(Length - ChunkOffset, MaxChunkSize);
		
		CreateMessage(EAgentMessageType::ReadBlobResponse, ChunkLength + 128);
		WriteInt32((int)ChunkOffset);
		WriteInt32((int)Length);
		WriteFixedLengthBytes(Data + ChunkOffset, ChunkLength);
		FlushMessage();

#Loc: <Workspace>/Engine/Source/Developer/Horde/Private/Storage/Nodes/ChunkNode.cpp:172

Scope (from outer to inner):

file
function     void FChunkNodeWriter::Write

Source code excerpt:

	while (Data.GetSize() > 0)
	{
		uint8* NodeBuffer = (uint8*)Writer.GetOutputBuffer(Options.MaxChunkSize);

		// Add up to the minimum chunk size
		if (NodeLength < Options.MinChunkSize)
		{
			uint64 AppendLength = FMath::Min<uint64>(Options.MinChunkSize - NodeLength, Data.GetSize());
			memcpy(NodeBuffer + NodeLength, Data.GetData(), AppendLength);

#Loc: <Workspace>/Engine/Source/Developer/Horde/Public/Storage/Nodes/ChunkNode.h:20

Scope: file

Source code excerpt:

	int32 MinChunkSize = 4 * 1024;
	int32 TargetChunkSize = 64 * 1024;
	int32 MaxChunkSize = 128 * 1024;
};

/**
 * Node containing a chunk of data
 */
class HORDE_API FChunkNode

#Loc: <Workspace>/Engine/Source/Developer/TargetPlatform/Public/Interfaces/IAudioFormat.h:118

Scope (from outer to inner):

file
class        class IAudioFormat

Source code excerpt:

	 * @param OutBuffers Array of buffers that contain the chunks the original data was split into.
	 * @param FirstChunkMaxSize The maximum size for the chunk that will be loaded inline with it's owning USoundWave asset.
	 * @param MaxChunkSize The maximum chunk size for each chunk. The chunks will be zero-padded to match this chunk size in the bulk data serialization.
	 * @return Whether bulk data could be split for streaming.
	 */
	virtual bool SplitDataForStreaming(const TArray<uint8>& SrcBuffer, TArray<TArray<uint8>>& OutBuffers, const int32 FirstChunkMaxSize, const int32 MaxChunkSize) const {return false;}

	virtual bool RequiresStreamingSeekTable() const { return false; }

	struct FSeekTable
	{
		TArray<uint32> Times;			// Times in AudioFrames.
		TArray<uint32> Offsets;			// Offset in the compressed data.
	};	

	/**
	* Extracts the embedded seek-table, removing it, and outputting it separately.

#Loc: <Workspace>/Engine/Source/Editor/UnrealEd/Private/Commandlets/AssetRegistryGenerator.cpp:346

Scope (from outer to inner):

file
function     bool FAssetRegistryGenerator::GenerateStreamingInstallManifest

Source code excerpt:

		TmpPackagingDir /= InManifestSubDir;
	}
	int64 MaxChunkSize = InOverrideChunkSize > 0 ? InOverrideChunkSize : GetMaxChunkSizePerPlatform(TargetPlatform);

	if (!IFileManager::Get().MakeDirectory(*TmpPackagingDir, true /* Tree */))
	{
		UE_LOG(LogAssetRegistryGenerator, Error, TEXT("Failed to create directory: %s"), *TmpPackagingDir);
		return false;
	}

#Loc: <Workspace>/Engine/Source/Editor/UnrealEd/Private/Commandlets/AssetRegistryGenerator.cpp:582

Scope (from outer to inner):

file
function     bool FAssetRegistryGenerator::GenerateStreamingInstallManifest

Source code excerpt:

		Manifest->GenerateValueArray(ChunkFilenames);

		if (MaxChunkSize > 0)
		{
			PackageFileSizes.Reset();
			PackageFileSizes.Reserve(Manifest->Num());
			const TMap<FName, const FAssetPackageData*>& PackageDataMap = State.GetAssetPackageDataMap();
			for (const TPair<FName, FString>& ManifestEntry : *Manifest)
			{

#Loc: <Workspace>/Engine/Source/Editor/UnrealEd/Private/Commandlets/AssetRegistryGenerator.cpp:759

Scope (from outer to inner):

file
function     bool FAssetRegistryGenerator::GenerateStreamingInstallManifest

Source code excerpt:

				FString Filename = ChunkFilenames[FilenameIndex];
				FString PakListLine = FPaths::ConvertRelativePathToFull(Filename.Replace(TEXT("[Platform]"), *Platform));
				if (MaxChunkSize > 0)
				{
					const int64* ExistingPackageFileSize = PackageFileSizes.Find(PakListLine);
					const int64 PackageFileSize = ExistingPackageFileSize ? *ExistingPackageFileSize : 0;
					CurrentPakSize += PackageFileSize;
					if (MaxChunkSize < CurrentPakSize)
					{
						// early out if we are over memory limit
						bFinishedAllFiles = false;
						NextFileSize = PackageFileSize;
						NextFilename = MoveTemp(PakListLine);
						break;

#Loc: <Workspace>/Engine/Source/Editor/UnrealEd/Private/Commandlets/AssetRegistryGenerator.cpp:786

Scope (from outer to inner):

file
function     bool FAssetRegistryGenerator::GenerateStreamingInstallManifest

Source code excerpt:

				const TCHAR* UnitsText = TEXT("MB");
				int32 Unit = 1000*1000;
				if (MaxChunkSize < Unit * 10)
				{
					Unit = 1;
					UnitsText = TEXT("bytes");
				}
				UE_LOGFMT(LogAssetRegistryGenerator, Error, "Failed to add file {NextFilename} to paklist '{PakListFilename}'. The maximum size for a Pakfile is {MaxChunkFileSize}{UnitsText}, but the file to add is {ActualChunkFileSize}{UnitsText}.", 
					("NextFilename", NextFilename),
					("PakListFilename", PakListFilename),
					("MaxChunkFileSize", MaxChunkSize / Unit),
					("UnitsText", UnitsText),
					("ActualChunkFileSize", (NextFileSize + Unit - 1) / Unit)	// Round the limit down and round the value up, so that the display always shows that the value is greater than the limit
				);
				bSucceeded = false;
				break;
			}

#Loc: <Workspace>/Engine/Source/Programs/Unsync/Private/UnsyncProxy.cpp:661

Scope (from outer to inner):

file
namespace    unsync
function     TResult<> ProxyQuery::DownloadFile

Source code excerpt:

	}

	const uint64 MaxChunkSize = 16_MB;

	FIOWriter& Result = OutputCallback(FileSize);
	if (!Result.IsValid())
	{
		return AppError(L"Failed to create download output stream");
	}

	const uint64		NumChunks = DivUp(FileSize, MaxChunkSize);
	std::vector<FRange> Chunks;
	Chunks.reserve(NumChunks);

	for (uint64 i = 0; i < NumChunks; ++i)
	{
		FRange Range;
		Range.Offset = i * MaxChunkSize;
		Range.Size	 = CalcChunkSize(i, MaxChunkSize, FileSize);
		Chunks.push_back(Range);
	}

	FAtomicError Error;
	FSemaphore	 DownloadSempahore(4);	// up to 4 concurrent connections

#Loc: <Workspace>/Engine/Source/Runtime/Core/Private/ProfilingDebugging/MiscTrace.cpp:129

Scope (from outer to inner):

file
function     void FMiscTrace::OutputScreenshot

Source code excerpt:


	const uint32 DataSize = (uint32) Data.Num();
	const uint32 MaxChunkSize = TNumericLimits<uint16>::Max();
	uint32 ChunkNum = (DataSize + MaxChunkSize - 1) / MaxChunkSize;

	uint32 Id = ScreenshotId.fetch_add(1);
	UE_TRACE_LOG(Misc, ScreenshotHeader, ScreenshotChannel)
		<< ScreenshotHeader.Id(Id)
		<< ScreenshotHeader.Name(Name, uint16(FCString::Strlen(Name)))
		<< ScreenshotHeader.Cycle(Cycle)

#Loc: <Workspace>/Engine/Source/Runtime/Core/Private/ProfilingDebugging/MiscTrace.cpp:145

Scope (from outer to inner):

file
function     void FMiscTrace::OutputScreenshot

Source code excerpt:

	for (uint32 Index = 0; Index < ChunkNum; ++Index)
	{
		uint16 Size = (uint16) FMath::Min(RemainingSize, MaxChunkSize);

		uint8* ChunkData = Data.GetData() + MaxChunkSize * Index;
		UE_TRACE_LOG(Misc, ScreenshotChunk, ScreenshotChannel)
			<< ScreenshotChunk.Id(Id)
			<< ScreenshotChunk.ChunkNum(Index)
			<< ScreenshotChunk.Size(Size)
			<< ScreenshotChunk.Data(ChunkData, Size);

#Loc: <Workspace>/Engine/Source/Runtime/Engine/Private/AudioCompressionSettingsUtils.cpp:339

Scope (from outer to inner):

file
function     FCachedAudioStreamingManagerParams FPlatformCompressionUtilities::BuildCachedStreamingManagerParams

Source code excerpt:

{
	const FAudioStreamCachingSettings& CacheSettings = GetStreamCachingSettingsForCurrentPlatform();
	int32 MaxChunkSize = GetMaxChunkSizeForCookOverrides(GetCookOverrides());

	const int32 MaxChunkSizeOverrideBytes = CacheSettings.MaxChunkSizeOverrideKB * 1024;
	if (MaxChunkSizeOverrideBytes > 0)
	{
		MaxChunkSize = FMath::Min(MaxChunkSizeOverrideBytes, MaxChunkSize);
	}

	// Our number of elements is tweakable based on the minimum cache usage we want to support.
	const float MinimumCacheUsage = FMath::Clamp(MinimumCacheUsageCvar, 0.0f, (1.0f - UE_KINDA_SMALL_NUMBER));
	int32 MinChunkSize = (1.0f - MinimumCacheUsage) * MaxChunkSize;
	
	uint64 TempNumElements = ((CacheSettings.CacheSizeKB * 1024) / MinChunkSize) * FMath::Max(ChunkSlotNumScalarCvar, 1.0f);
	int32 NumElements = FMath::Min(TempNumElements, static_cast<uint64>(TNumericLimits< int32 >::Max()));

	FCachedAudioStreamingManagerParams Params;
	FCachedAudioStreamingManagerParams::FCacheDimensions CacheDimensions;

	// Primary cache defined here:
	CacheDimensions.MaxChunkSize = 256 * 1024; // max possible chunk size (hard coded for legacy streaming path)
	CacheDimensions.MaxMemoryInBytes = CacheSettings.CacheSizeKB > 0 ? CacheSettings.CacheSizeKB * 1024 : FAudioStreamCachingSettings::DefaultCacheSize * 1024;
	CacheDimensions.NumElements = FMath::Max(NumElements, 1); // force at least a single cache element to avoid crashes
	Params.Caches.Add(CacheDimensions);

	// TODO: When settings are added to support multiple sub-caches, add it here.

#Loc: <Workspace>/Engine/Source/Runtime/Engine/Private/AudioDerivedData.cpp:864

Scope (from outer to inner):

file
class        class FStreamedAudioCacheDerivedDataWorker : public FNonAbandonableTask
function     void BuildStreamedAudio

Source code excerpt:

					for (int32 ChunkIndex = 1; ChunkIndex < ChunkBuffers.Num(); ++ChunkIndex)
					{
						// Zero pad the reallocation if the chunk isn't precisely the max chunk size to keep the reads aligned to MaxChunkSize
						int32 AudioDataSize = ChunkBuffers[ChunkIndex].Num();
						check(AudioDataSize != 0);
						ensureMsgf(AudioDataSize <= MaxChunkSizeForCurrentWave, TEXT("Chunk is overbudget by %d bytes"), AudioDataSize - MaxChunkSizeForCurrentWave);

						int32 ZeroPadBytes = 0;

#Loc: <Workspace>/Engine/Source/Runtime/Engine/Private/AudioStreamingCache.cpp:357

Scope (from outer to inner):

file
function     FCachedAudioStreamingManager::FCachedAudioStreamingManager

Source code excerpt:

	for (const FCachedAudioStreamingManagerParams::FCacheDimensions& CacheDimensions : InitParams.Caches)
	{
		CacheArray.Emplace(CacheDimensions.MaxChunkSize, CacheDimensions.NumElements, CacheDimensions.MaxMemoryInBytes);
	}

	// Here we make sure our CacheArray is sorted from smallest MaxChunkSize to biggest, so that GetCacheForWave can scan through these caches to find the appropriate cache for the chunk size.
	CacheArray.Sort();
}

FCachedAudioStreamingManager::~FCachedAudioStreamingManager()
{
}

#Loc: <Workspace>/Engine/Source/Runtime/Engine/Private/AudioStreamingCache.cpp:617

Scope (from outer to inner):

file
function     FAudioChunkCache* FCachedAudioStreamingManager::GetCacheForChunkSize

Source code excerpt:

{
	LLM_SCOPE(ELLMTag::AudioStreamCache);
	// Iterate over our caches until we find the lowest MaxChunkSize cache this sound's chunks will fit into. 
	for (int32 CacheIndex = 0; CacheIndex < CacheArray.Num(); CacheIndex++)
	{
		check(CacheArray[CacheIndex].MaxChunkSize >= 0);
		if (InChunkSize <= ((uint32)CacheArray[CacheIndex].MaxChunkSize))
		{
			return const_cast<FAudioChunkCache*>(&CacheArray[CacheIndex]);
		}
	}

	// If we ever hit this, something may have wrong during cook.
	// Please check to make sure this platform's implementation of IAudioFormat honors the MaxChunkSize parameter passed into SplitDataForStreaming,
	// or that FStreamedAudioCacheDerivedDataWorker::BuildStreamedAudio() is passing the correct MaxChunkSize to IAudioFormat::SplitDataForStreaming.
	ensureMsgf(false, TEXT("Chunks in SoundWave are too large: %d bytes"), InChunkSize);
	return nullptr;
}

int32 FCachedAudioStreamingManager::GetNextChunkIndex(const FSoundWaveProxyPtr&  InSoundWave, uint32 CurrentChunkIndex) const
{

#Loc: <Workspace>/Engine/Source/Runtime/Engine/Private/AudioStreamingCache.cpp:732

Scope (from outer to inner):

file
function     FAudioChunkCache::FAudioChunkCache

Source code excerpt:


FAudioChunkCache::FAudioChunkCache(uint32 InMaxChunkSize, uint32 NumChunks, uint64 InMemoryLimitInBytes)
	: MaxChunkSize(InMaxChunkSize)
	, MostRecentElement(nullptr)
	, LeastRecentElement(nullptr)
	, ChunksInUse(0)
	, MemoryCounterBytes(0)
	, MemoryLimitBytes(InMemoryLimitInBytes)
	, ForceInlineMemoryCounterBytes(0)

#Loc: <Workspace>/Engine/Source/Runtime/Engine/Private/AudioStreamingCache.cpp:1280

Scope (from outer to inner):

file
function     uint64 FAudioChunkCache::ReportCacheSize

Source code excerpt:

	const uint32 NumChunks = CachePool.Num();

	return MaxChunkSize * NumChunks;
}

void FAudioChunkCache::BeginLoggingCacheMisses()
{
	bLogCacheMisses = true;
}

#Loc: <Workspace>/Engine/Source/Runtime/Engine/Public/AudioStreamingCache.h:172

Scope (from outer to inner):

file
class        class FAudioChunkCache

Source code excerpt:

	void SetCacheLookupIDForChunk(const FChunkKey& InChunkKey, uint64 InCacheLookupID);

	const int32 MaxChunkSize;

	// This is for debugging purposes only. Prints the elements in the cache from most recently used to least.
	// Returns the dimensions of this debug log so that multiple caches can be tiled across the screen.
	TPair<int, int> DebugDisplayLegacy(UWorld* World, FViewport* Viewport, FCanvas* Canvas, int32 X, int32 Y, const FVector* ViewLocation, const FRotator* ViewRotation) const;

	// Generate a formatted text file for this cache.

#Loc: <Workspace>/Engine/Source/Runtime/Engine/Public/AudioStreamingCache.h:481

Scope (from outer to inner):

file
function     inline bool operator<

Source code excerpt:

inline bool operator< (const FAudioChunkCache& Element1, const FAudioChunkCache& Element2)
{
	return Element1.MaxChunkSize < Element2.MaxChunkSize;
}

struct FCachedAudioStreamingManagerParams
{
	struct FCacheDimensions
	{
		// The max size, in bytes, of a single chunk of compressed audio.
		// During cook, compressed audio assets will be chunked based on this amount.
		int32 MaxChunkSize;

		// The maximum number of elements stored in a single cache before it is evicted.
		// At runtime, this will be clamped to ensure that it is greater than the amount of
		// sources that can be playing simultaneously.
		int32 NumElements;

#Loc: <Workspace>/Engine/Source/Runtime/Experimental/Chaos/Private/ChaosVisualDebugger/ChaosVisualDebuggerTrace.cpp:575

Scope (from outer to inner):

file
function     void FChaosVisualDebuggerTrace::TraceBinaryData

Source code excerpt:


	const uint32 DataSize = static_cast<uint32>(DataViewToTrace.Num());
	constexpr uint32 MaxChunkSize = TNumericLimits<uint16>::Max();
	const uint32 ChunkNum = (DataSize + MaxChunkSize - 1) / MaxChunkSize;

	UE_TRACE_LOG(ChaosVDLogger, ChaosVDBinaryDataStart, ChaosVDChannel)
		<< ChaosVDBinaryDataStart.Cycle(FPlatformTime::Cycles64())
		<< ChaosVDBinaryDataStart.TypeName(TypeName.GetData(), TypeName.Len())
		<< ChaosVDBinaryDataStart.DataID(DataID)
		<< ChaosVDBinaryDataStart.DataSize(DataSize)

#Loc: <Workspace>/Engine/Source/Runtime/Experimental/Chaos/Private/ChaosVisualDebugger/ChaosVisualDebuggerTrace.cpp:589

Scope (from outer to inner):

file
function     void FChaosVisualDebuggerTrace::TraceBinaryData

Source code excerpt:

	for (uint32 Index = 0; Index < ChunkNum; ++Index)
	{
		const uint16 Size = static_cast<uint16>(FMath::Min(RemainingSize, MaxChunkSize));
		const uint8* ChunkData = DataViewToTrace.GetData() + MaxChunkSize * Index;

		UE_TRACE_LOG(ChaosVDLogger, ChaosVDBinaryDataContent, ChaosVDChannel)
			<< ChaosVDBinaryDataContent.Cycle(FPlatformTime::Cycles64())
			<< ChaosVDBinaryDataContent.DataID(DataID)
			<< ChaosVDBinaryDataContent.RawData(ChunkData, Size);

#Loc: <Workspace>/Engine/Source/Runtime/GeometryFramework/Private/Components/MeshRenderDecomposition.cpp:90

Scope (from outer to inner):

file
function     static void CollectSubDecomposition

Source code excerpt:

	UMaterialInterface* Material, 
	FMeshRenderDecomposition& Decomp, 
	int32 MaxChunkSize,
	FCriticalSection& DecompLock)
{
	int32 MaxTrisPerGroup = MaxChunkSize;
	if (Triangles.Num() < MaxTrisPerGroup)
	{
		DecompLock.Lock();
		int32 i = Decomp.AppendGroup();
		FMeshRenderDecomposition::FGroup& Group = Decomp.GetGroup(i);
		DecompLock.Unlock();

#Loc: <Workspace>/Engine/Source/Runtime/GeometryFramework/Private/Components/MeshRenderDecomposition.cpp:162

Scope (from outer to inner):

file
function     void FMeshRenderDecomposition::BuildChunkedDecomposition

Source code excerpt:



void FMeshRenderDecomposition::BuildChunkedDecomposition(const FDynamicMesh3* Mesh, const FComponentMaterialSet* MaterialSet, FMeshRenderDecomposition& Decomp, int32 MaxChunkSize)
{
	TUniquePtr<FMeshRenderDecomposition> MaterialDecomp = MakeUnique<FMeshRenderDecomposition>();
	BuildMaterialDecomposition(Mesh, MaterialSet, *MaterialDecomp);
	int32 NumMatGroups = MaterialDecomp->Num();

	FCriticalSection Lock;

#Loc: <Workspace>/Engine/Source/Runtime/GeometryFramework/Private/Components/MeshRenderDecomposition.cpp:172

Scope (from outer to inner):

file
function     void FMeshRenderDecomposition::BuildChunkedDecomposition
lambda-function

Source code excerpt:

	{
		const FMeshRenderDecomposition::FGroup& Group = MaterialDecomp->GetGroup(k);
		CollectSubDecomposition(Mesh, Group.Triangles, Group.Material, Decomp, MaxChunkSize, Lock);
	});
}

#Loc: <Workspace>/Engine/Source/Runtime/GeometryFramework/Public/Components/MeshRenderDecomposition.h:93

Scope (from outer to inner):

file
class        class FMeshRenderDecomposition

Source code excerpt:


	/**
	 * Build per-material decomposition, and then split each of those into chunks of at most MaxChunkSize
	 * (actual chunk sizes will be highly variable and some may be very small...)
	 */
	static GEOMETRYFRAMEWORK_API void BuildChunkedDecomposition(const FDynamicMesh3* Mesh, const FComponentMaterialSet* MaterialSet, FMeshRenderDecomposition& Decomp, int32 MaxChunkSize = 1 << 14 /* 16k */ );
};

#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/SignedArchiveReader.cpp:350

Scope (from outer to inner):

file
function     int64 FSignedArchiveReader::CalculateChunkSize

Source code excerpt:

	if (ChunkIndex == (ChunkCount - 1))
	{
		const int64 MaxChunkSize = FPakInfo::MaxChunkDataSize;
		int64 Slack = SizeOnDisk % MaxChunkSize;
		if (!Slack)
		{
			return FPakInfo::MaxChunkDataSize;
		}
		else
		{

#Loc: <Workspace>/Engine/Source/Runtime/RHI/Private/GPUDefragAllocator.cpp:1916

Scope: file

Source code excerpt:

*
* @param MinChunkSize	Minimum number of bytes per random chunk
* @param MaxChunkSize	Maximum number of bytes per random chunk
* @param FreeRatio		Free 0.0-1.0 % of the memory before benchmarking
* @param LockRatio		Lock 0.0-1.0 % of the memory before benchmarking
* @param bFullDefrag	Whether to test full defrag (true) or continuous defrag (false)
* @param bSaveImages	Whether to save before/after images to hard disk (TexturePoolBenchmark-*.bmp)
* @param Filename		[opt] Filename to a previously saved memory layout to use for benchmarking, or nullptr
*/
void FGPUDefragAllocator::Benchmark(int32 MinChunkSize, int32 MaxChunkSize, float FreeRatio, float LockRatio, bool bFullDefrag, bool bSaveImages, const TCHAR* Filename)
{
//untested
#if 0
#if !(UE_BUILD_SHIPPING ||UE_BUILD_TEST)
	int64 OriginalAllocatedMemorySize = AllocatedMemorySize;
	int64 OriginalAvailableMemorySize = AvailableMemorySize;
	FRandomStream Rand(0x1ee7c0de);
	TArray<void*> Chunks;
	TArray<FMemoryChunk*> OriginalChunks;

#Loc: <Workspace>/Engine/Source/Runtime/RHI/Private/GPUDefragAllocator.cpp:2007

Scope (from outer to inner):

file
function     void FGPUDefragAllocator::Benchmark

Source code excerpt:

		do
		{
			int32 ChunkSize = Align((FMath::TruncToInt((MaxChunkSize - MinChunkSize)*Rand.GetFraction()) + MinChunkSize), 4096);
			Ptr = Allocate(ChunkSize, 1, true);
			if (Ptr)
			{
				Chunks.Add(Ptr);
			}
		} while (Ptr);

#Loc: <Workspace>/Engine/Source/Runtime/RHI/Public/GPUDefragAllocator.h:653

Scope (from outer to inner):

file
class        class FGPUDefragAllocator

Source code excerpt:

	*
	* @param MinChunkSize	Minimum number of uint8s per random chunk
	* @param MaxChunkSize	Maximum number of uint8s per random chunk
	* @param FreeRatio		Free 0.0-1.0 of the memory before benchmarking
	* @param LockRatio		Lock 0.0-1.0 % of the memory before benchmarking
	* @param bFullDefrag	Whether to test full defrag (true) or continuous defrag (false)
	* @param bSaveImages	Whether to save before/after images to hard disk (TexturePoolBenchmark-*.bmp)
	* @param Filename		[opt] Filename to a previously saved memory layout to use for benchmarking, or nullptr
	*/
	void	Benchmark(int32 MinChunkSize, int32 MaxChunkSize, float FreeRatio, float LockRatio, bool bFullDefrag, bool bSaveImages, const TCHAR* Filename);
	
	static FORCEINLINE bool IsAligned(const volatile void* Ptr, const uint32 Alignment)
	{
		return !(UPTRINT(Ptr) & (Alignment - 1));
	}

	int32 GetAllocationAlignment() const
	{
		return AllocationAlignment;

#Loc: <Workspace>/Engine/Source/Runtime/Renderer/Private/SceneCulling/SceneCulling.cpp:935

Scope (from outer to inner):

file
class        class FSceneCullingBuilder

Source code excerpt:

	static constexpr int32 CellBlockDimLog2 = FSpatialHash::CellBlockDimLog2;

	static constexpr int32 MaxChunkSize = int32(INSTANCE_HIERARCHY_MAX_CHUNK_SIZE);
	using FPrimitiveState = FSceneCulling::FPrimitiveState;
	using FCellIndexCacheEntry = FSceneCulling::FCellIndexCacheEntry;

	using FBlockLoc = FSceneCulling::FBlockLoc;

	enum class EExplicitBoundsUpdateMode

#Loc: <Workspace>/Engine/Source/Runtime/Renderer/Private/SceneCulling/SceneCulling.cpp:1016

Scope (from outer to inner):

file
class        class FSceneCullingBuilder

Source code excerpt:

	struct FChunkBuilder
	{
		int32 CurrentChunkCount = MaxChunkSize;
		int32 CurrentChunkId = INDEX_NONE;
		

		SC_FORCEINLINE bool IsCurrentChunkEmpty() const
		{
			return CurrentChunkId == INDEX_NONE;

#Loc: <Workspace>/Engine/Source/Runtime/Renderer/Private/SceneCulling/SceneCulling.cpp:1027

Scope (from outer to inner):

file
class        class FSceneCullingBuilder
function     SC_FORCEINLINE void EmitCurrentChunkIfFull

Source code excerpt:

		SC_FORCEINLINE void EmitCurrentChunkIfFull()
		{
			if (CurrentChunkId != INDEX_NONE && CurrentChunkCount >= MaxChunkSize)
			{
				PackedChunkIds.Add(uint32(CurrentChunkId) | (uint32(CurrentChunkCount) << INSTANCE_HIERARCHY_ITEM_CHUNK_COUNT_SHIFT));
				CurrentChunkId = INDEX_NONE;
			}
		}

#Loc: <Workspace>/Engine/Source/Runtime/Renderer/Private/SceneCulling/SceneCulling.cpp:1055

Scope (from outer to inner):

file
class        class FSceneCullingBuilder
function     SC_FORCEINLINE void Add

Source code excerpt:

		{
			TArray<uint32>& PackedCellData = Builder.SceneCulling.PackedCellData;
			if (CurrentChunkCount >= MaxChunkSize)
			{
				EmitCurrentChunk();
				// Allocate a new chunk
				CurrentChunkId = Builder.AllocateChunk();
				CurrentChunkCount = 0;
				Builder.TotalCellChunkDataCount += 1;
			}
			// Emit the instance ID directly to final destination
			int32 ItemOffset = CurrentChunkId * MaxChunkSize + CurrentChunkCount++;
			PackedCellData[ItemOffset] = InstanceId;
		}

		SC_FORCEINLINE void AddRange(FSceneCullingBuilder& Builder, int32 InInstanceIdOffset, int32 InInstanceIdCount)
		{
			// First add all compressible ranges.
			int32 InstanceIdOffset = InInstanceIdOffset;
			int32 InstanceIdCount = InInstanceIdCount;
			while (InstanceIdCount >= MaxChunkSize)
			{
				UPDATE_BUILDER_STAT(Builder, CompRangeCount, 1);

				AddCompressedChunk(Builder, InstanceIdOffset);
				InstanceIdOffset += MaxChunkSize;
				InstanceIdCount -= MaxChunkSize;
			}

			// add remainder individually
			for (int32 InstanceId = InstanceIdOffset; InstanceId < InstanceIdOffset + InstanceIdCount; ++InstanceId)
			{
				Add(Builder, InstanceId);