Oodle

Oodle

#Overview

name: Oodle

This variable is created as a Console Variable (cvar).

It is referenced in 19 C++ source files.

#Summary

#Usage in the C++ source code

The purpose of Oodle is to provide an advanced compression algorithm for Unreal Engine 5, primarily used for network data compression and file compression. Oodle is a high-performance compression library developed by RAD Game Tools (now part of Epic Games) that offers better compression ratios and faster decompression speeds compared to traditional algorithms like Zlib or Gzip.

Oodle is utilized in several Unreal Engine subsystems and modules, including:

  1. Network compression (OodleNetwork plugin)
  2. File compression (PakFile system)
  3. Shader compression
  4. General data compression (CompressedBuffer system)

The value of this variable is typically set through command-line parameters or project settings. For example, in the OodleNetworkHandlerComponent, it can be enabled using the “-Oodle” command-line parameter.

Oodle interacts with other compression-related variables and systems, such as:

  1. Other compression methods (Zlib, Gzip, LZ4)
  2. Compression levels and settings
  3. PakFile compression options

Developers should be aware of the following when using Oodle:

  1. Oodle requires a license, which is included with Unreal Engine 5.
  2. It may not be available on all platforms, so fallback compression methods should be considered.
  3. Oodle offers different compression algorithms (e.g., Kraken, Mermaid, Selkie) with varying trade-offs between compression ratio and speed.

Best practices for using Oodle include:

  1. Use Oodle for network compression when possible, as it can significantly reduce bandwidth usage.
  2. Consider using Oodle for compressing large assets or data files to reduce storage requirements.
  3. Test performance and compression ratios with different Oodle algorithms and settings to find the optimal configuration for your project.
  4. Ensure that all target platforms support Oodle before relying on it exclusively.
  5. Use the appropriate Oodle dictionary files for your project to improve compression efficiency.

By leveraging Oodle compression, developers can improve the performance and efficiency of their Unreal Engine 5 projects, especially in areas like network communication and asset storage.

#References in C++ code

#Callsites

This variable is referenced in the following C++ source code:

#Loc: <Workspace>/Engine/Plugins/Compression/OodleNetwork/Source/Private/OodleNetworkHandlerComponent.cpp:1521

Scope (from outer to inner):

file
function     static bool OodleExec

Source code excerpt:

	bool bReturnVal = false;

	if (FParse::Command(&Cmd, TEXT("Oodle")))
	{
		// Used by unit testing code, to enable/disable Oodle during a unit test
		// NOTE: Do not use while a NetConnection is using Oodle, as this will cause it to break. Debugging/Testing only.
		if (FParse::Command(&Cmd, TEXT("ForceEnable")))
		{
			bool bTurnOn = false;

#Loc: <Workspace>/Engine/Plugins/Compression/OodleNetwork/Source/Private/OodleNetworkHandlerComponent.cpp:1539

Scope (from outer to inner):

file
function     static bool OodleExec

Source code excerpt:

			else if (FParse::Command(&Cmd, TEXT("Default")))
			{
				bTurnOn = FParse::Param(FCommandLine::Get(), TEXT("Oodle"));
			}
			else
			{
				bTurnOn = !bOodleForceEnable;
			}

#Loc: <Workspace>/Engine/Plugins/Compression/OodleNetwork/Source/Private/OodleNetworkHandlerComponent.cpp:1796

Scope (from outer to inner):

file
function     void FOodleComponentModuleInterface::StartupModule

Source code excerpt:


	// If Oodle is force-enabled on the commandline, execute the commandlet-enable command, which also adds to the PacketHandler list
	bOodleForceEnable = FParse::Param(FCommandLine::Get(), TEXT("Oodle"));

#if USE_OODLE_TRAINER_COMMANDLET
  	//if (bOodleForceEnable)
	if (bOodleForceEnable)
	{
		UOodleNetworkTrainerCommandlet::HandleEnable();

#Loc: <Workspace>/Engine/Plugins/Compression/OodleNetwork/Source/Private/OodleNetworkHandlerComponent.cpp:1808

Scope (from outer to inner):

file
function     void FOodleComponentModuleInterface::StartupModule

Source code excerpt:


	// Use an absolute path for this, as we want all relative paths, to be relative to this folder
	GOodleSaveDir = FPaths::ConvertRelativePathToFull(FPaths::Combine(*FPaths::ProjectSavedDir(), TEXT("Oodle")));
	GOodleContentDir = FPaths::ConvertRelativePathToFull(FPaths::Combine(*FPaths::ProjectContentDir(), TEXT("Oodle")));

}

void FOodleComponentModuleInterface::ShutdownModule()
{
}

#Loc: <Workspace>/Engine/Source/Developer/Horde/Private/Storage/Bundles/BundleCompression.cpp:14

Scope (from outer to inner):

file
function     size_t FBundleCompression::GetMaxSize

Source code excerpt:

	case EBundleCompressionFormat::Gzip:
		return FCompression::CompressMemoryBound(NAME_Gzip, Input.GetSize());
	case EBundleCompressionFormat::Oodle:
		return FCompression::CompressMemoryBound(NAME_Oodle, Input.GetSize());
	default:
		FHordePlatform::NotSupported("The specified compression format is not currently supported");
	}
}

#Loc: <Workspace>/Engine/Source/Developer/Horde/Private/Storage/Bundles/BundleCompression.cpp:36

Scope (from outer to inner):

file
function     size_t FBundleCompression::Compress

Source code excerpt:

		verify(FCompression::CompressMemory(NAME_Gzip, Output.GetData(), CompressedSize, Input.GetData(), Input.GetSize()));
		return CompressedSize;
	case EBundleCompressionFormat::Oodle:
		verify(FCompression::CompressMemory(NAME_Oodle, Output.GetData(), CompressedSize, Input.GetData(), Input.GetSize()));
		return CompressedSize;
	default:
		FHordePlatform::NotSupported("The specified compression format is not currently supported");
	}
}

#Loc: <Workspace>/Engine/Source/Developer/Horde/Private/Storage/Bundles/BundleCompression.cpp:57

Scope (from outer to inner):

file
function     void FBundleCompression::Decompress

Source code excerpt:

		verify(FCompression::UncompressMemory(NAME_Gzip, Output.GetData(), Output.GetSize(), Input.GetData(), Input.GetSize()));
		break;
	case EBundleCompressionFormat::Oodle:
		verify(FCompression::UncompressMemory(NAME_Oodle, Output.GetData(), Output.GetSize(), Input.GetData(), Input.GetSize()));
		break;
	default:
		FHordePlatform::NotSupported("The specified compression format is not currently supported");
	}
}

#Loc: <Workspace>/Engine/Source/Developer/Horde/Public/Storage/Bundles/BundleCompression.h:18

Scope: file

Source code excerpt:


	/** Oodle compression (Kraken). */
	Oodle = 3,

	/** Brotli compression. */
	Brotli = 4,
};

/*

#Loc: <Workspace>/Engine/Source/Developer/IoStoreUtilities/Private/ZenFileSystemManifest.cpp:389

Scope (from outer to inner):

file
function     int32 FZenFileSystemManifest::Generate

Source code excerpt:

	FFileFilter OoodleDictionaryFilter = FFileFilter()
		.IncludeExtension(TEXT("udic"));
	AddFilesFromDirectory(TEXT("/{project}/Content/Oodle"), FPaths::Combine(ProjectDir, TEXT("Content"), TEXT("Oodle")), false, &OoodleDictionaryFilter);

	FFileFilter ShaderCacheFilter = FFileFilter()
		.IncludeExtension(TEXT("ushadercache"))
		.IncludeExtension(TEXT("upipelinecache"));
	AddFilesFromDirectory(TEXT("/{project}/Content"), FPaths::Combine(ProjectDir, TEXT("Content")), false, &ShaderCacheFilter);
	AddFilesFromDirectory(FPaths::Combine(TEXT("/{project}"), TEXT("Content"), TEXT("PipelineCaches"), TargetPlatform.IniPlatformName()), FPaths::Combine(ProjectDir, TEXT("Content"), TEXT("PipelineCaches"), TargetPlatform.IniPlatformName()), false, &ShaderCacheFilter);

#Loc: <Workspace>/Engine/Source/Developer/PakFileUtilities/Private/PakFileUtilities.cpp:2298

Scope (from outer to inner):

file
function     bool FPakWriterContext::AddPakFile

Source code excerpt:

		OutputPakFile->Info.GetCompressionMethodIndex(NAME_Gzip);
		OutputPakFile->Info.GetCompressionMethodIndex(TEXT("Bogus"));
		OutputPakFile->Info.GetCompressionMethodIndex(TEXT("Oodle"));
	}

	OutputPakFile->Compressor_Stat_Count.SetNumZeroed(CompressionFormatsAndNone.Num());
	OutputPakFile->Compressor_Stat_RawBytes.SetNumZeroed(CompressionFormatsAndNone.Num());
	OutputPakFile->Compressor_Stat_CompBytes.SetNumZeroed(CompressionFormatsAndNone.Num());

#Loc: <Workspace>/Engine/Source/Runtime/Core/Private/Compression/CompressedBuffer.cpp:39

Scope (from outer to inner):

file
namespace    UE::CompressedBuffer::Private

Source code excerpt:

	None = 0,
	/** Header is followed by an array of compressed block sizes then the compressed blocks. */
	Oodle = 3,
	/** Header is followed by an array of compressed block sizes then the compressed blocks. */
	LZ4 = 4,
};

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

#Loc: <Workspace>/Engine/Source/Runtime/Core/Private/Compression/CompressedBuffer.cpp:139

Scope (from outer to inner):

file
namespace    UE::CompressedBuffer::Private
function     struct FHeader { static constexpr uint32 ExpectedMagic = 0xb7756362; uint32 Magic = ExpectedMagic;
function     static uint32 CalculateCrc32

Source code excerpt:

			OutBlockSize = 0;
			return true;
		case EMethod::Oodle:
			OutCompressor = ECompressedBufferCompressor(Compressor);
			OutCompressionLevel = ECompressedBufferCompressionLevel(CompressionLevel);
			OutBlockSize = uint64(1) << BlockSizeExponent;
			return true;
		default:
			return false;

#Loc: <Workspace>/Engine/Source/Runtime/Core/Private/Compression/CompressedBuffer.cpp:677

Scope (from outer to inner):

file
namespace    UE::CompressedBuffer::Private
function     struct FHeader { static constexpr uint32 ExpectedMagic = 0xb7756362; uint32 Magic = ExpectedMagic;
function     static uint32 CalculateCrc32
class        class FOodleEncoder final : public FBlockEncoder
function     EMethod GetMethod

Source code excerpt:


protected:
	EMethod GetMethod() const final { return EMethod::Oodle; }
	uint8 GetCompressor() const final { return uint8(Compressor); }
	uint8 GetCompressionLevel() const final { return uint8(CompressionLevel); }

	uint64 CompressBlockBound(uint64 RawSize) const final
	{
		return uint64(FOodleDataCompression::CompressedBufferSizeNeeded(int64(RawSize)));

#Loc: <Workspace>/Engine/Source/Runtime/Core/Private/Compression/CompressedBuffer.cpp:767

Scope (from outer to inner):

file
namespace    UE::CompressedBuffer::Private
function     struct FHeader { static constexpr uint32 ExpectedMagic = 0xb7756362; uint32 Magic = ExpectedMagic;
function     static uint32 CalculateCrc32
function     static const FDecoder* GetDecoder

Source code excerpt:

{
	static FNoneDecoder None;
	static FOodleDecoder Oodle;
	static FLZ4Decoder LZ4;
	switch (Method)
	{
	default:
		return nullptr;
	case EMethod::None:
		return &None;
	case EMethod::Oodle:
		return &Oodle;
	case EMethod::LZ4:
		return &LZ4;
	}
}

template <typename BufferType>

#Loc: <Workspace>/Engine/Source/Runtime/Core/Private/Compression/OodleDataCompression.cpp:338

Scope (from outer to inner):

file
namespace    FOodleDataCompression
function     int64 OodleDecode

Source code excerpt:

	int64 OodleDecode(const void * InCompBuf, int64 InCompBufSize64, void * OutRawBuf, int64 InRawLen64) 
	{
		TRACE_CPUPROFILER_EVENT_SCOPE(Oodle.Decode);
		
		UE_LOG(OodleDataCompression, VeryVerbose, TEXT("OodleDecode: %lld -> %lld"),InCompBufSize64,InRawLen64);

		OO_SINTa InCompBufSize = IntCastChecked<OO_SINTa>(InCompBufSize64);
		OO_SINTa InRawLen = IntCastChecked<OO_SINTa>(InRawLen64);

#Loc: <Workspace>/Engine/Source/Runtime/Core/Private/Compression/OodleDataCompression.cpp:432

Scope (from outer to inner):

file
namespace    FOodleDataCompression
function     int64 OodleEncode

Source code excerpt:

								int64 DictionaryBackup)
	{
		TRACE_CPUPROFILER_EVENT_SCOPE(Oodle.Encode);

		OodleLZ_Compressor LZCompressor = CompressorToOodleLZ_Compressor(Compressor);
		OodleLZ_CompressionLevel LZLevel = CompressionLevelToOodleLZ_CompressionLevel(Level);
		
		if ( LZCompressor == OodleLZ_Compressor_Invalid || LZLevel == OodleLZ_CompressionLevel_Invalid )
		{

#Loc: <Workspace>/Engine/Source/Runtime/Core/Public/UObject/UnrealNames.inl:201

Scope: file

Source code excerpt:

REGISTER_NAME(259, LZ4)
REGISTER_NAME(260, Mobile)
REGISTER_NAME(261, Oodle)

// Online
REGISTER_NAME(280,DGram)
REGISTER_NAME(281,Stream)
REGISTER_NAME(282,GameNetDriver)
REGISTER_NAME(283,PendingNetDriver)

#Loc: <Workspace>/Engine/Source/Runtime/Engine/Classes/Engine/RendererSettings.h:252

Scope (from outer to inner):

file
namespace    EShaderCompressionFormat

Source code excerpt:

		None = 0 UMETA(DisplayName = "Do not compress", ToolTip = "Fastest, but disk and memory footprint will be large"),
		LZ4 = 1 UMETA(DisplayName = "LZ4", ToolTip = "Compressing using LZ4"),
		Oodle = 2 UMETA(DisplayName = "Oodle", ToolTip = "Compressing using Oodle (default)"),
		Zlib = 3 UMETA(DisplayName = "ZLib", ToolTip = "Compressing using Zlib")
	};
}

UENUM()
namespace ELumenSoftwareTracingMode

#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Public/IPlatformFilePak.h:266

Scope (from outer to inner):

file
function     void Serialize

Source code excerpt:

			CompressionMethods.Add(NAME_Zlib);
			CompressionMethods.Add(NAME_Gzip);
			CompressionMethods.Add(TEXT("Oodle"));
		}
		else
		{
			// we need to serialize a known size, so make a buffer of "strings"
			const int32 BufferSize = CompressionMethodNameLen * MaxNumCompressionMethods;
			ANSICHAR Methods[BufferSize];