Pak
Pak
#Overview
name: Pak
The value of this variable can be defined or overridden in .ini config files. 7
.ini config files referencing this setting variable.
It is referenced in 62
C++ source files.
#Summary
#Usage in the C++ source code
The purpose of the “Pak” variable is to handle Unreal Engine’s Pak (Package) file system. Pak files are a way to package and compress game assets for efficient loading and distribution. Here’s a breakdown of its usage and importance:
-
The “Pak” variable is used across various subsystems and modules in Unreal Engine, primarily in the PakFile and AssetRegistry systems.
-
It is used to manage the mounting, unmounting, and access of Pak files, which contain game assets and data.
-
The Pak system is crucial for optimizing game asset loading, reducing file I/O operations, and managing compressed game content.
-
The value of this variable is typically set when mounting a Pak file using the FPakPlatformFile::Mount function.
-
Other variables that interact with it include PakFile, PakOrder, and various file path and asset management variables.
-
Developers should be aware that the Pak system is central to how Unreal Engine manages game assets, especially in shipped games.
-
Best practices when using this variable include:
- Ensuring proper mounting and unmounting of Pak files
- Managing Pak file order for proper asset overriding
- Using the Pak system for optimizing asset loading in shipping builds
-
The Pak system is particularly important for platforms with limited storage or slow I/O, as it allows for efficient asset compression and loading.
-
Developers should be cautious when modifying the Pak system, as it can have significant impacts on game performance and asset management.
-
The Pak system is also used in conjunction with the AssetRegistry for efficient asset discovery and loading in packaged games.
Understanding and properly utilizing the Pak system is crucial for optimizing game performance and managing game assets effectively in Unreal Engine projects.
#Setting Variables
#References In INI files
Location: <Workspace>/Engine/Config/BaseEngine.ini:2527, section: [DerivedDataBackendGraph]
- INI Section:
DerivedDataBackendGraph
- Raw value:
(Type=ReadPak, Filename="%GAMEDIR%DerivedDataCache/DDC.ddp")
- Is Array:
False
Location: <Workspace>/Engine/Config/BaseEngine.ini:2540, section: [InstalledDerivedDataBackendGraph]
- INI Section:
InstalledDerivedDataBackendGraph
- Raw value:
(Type=ReadPak, Filename="%GAMEDIR%DerivedDataCache/DDC.ddp")
- Is Array:
False
Location: <Workspace>/Engine/Config/BaseEngine.ini:2552, section: [NoZenLocalFallback]
- INI Section:
NoZenLocalFallback
- Raw value:
(Type=ReadPak, Filename="%GAMEDIR%DerivedDataCache/DDC.ddp")
- Is Array:
False
Location: <Workspace>/Engine/Config/BaseEngine.ini:2561, section: [InstalledNoZenLocalFallback]
- INI Section:
InstalledNoZenLocalFallback
- Raw value:
(Type=ReadPak, Filename="%GAMEDIR%DerivedDataCache/DDC.ddp")
- Is Array:
False
Location: <Workspace>/Engine/Config/BaseEngine.ini:2579, section: [ZenDDC]
- INI Section:
ZenDDC
- Raw value:
(Type=ReadPak, Filename="%GAMEDIR%DerivedDataCache/DDC.ddp")
- Is Array:
False
Location: <Workspace>/Engine/Config/BaseEngine.ini:2600, section: [DerivedDataBackendGraph_Fill_Seattle]
- INI Section:
DerivedDataBackendGraph_Fill_Seattle
- Raw value:
(Type=ReadPak, Filename="%GAMEDIR%DerivedDataCache/DDC.ddp")
- Is Array:
False
Location: <Workspace>/Engine/Config/BaseEngine.ini:2609, section: [NoShared]
- INI Section:
NoShared
- Raw value:
(Type=ReadPak, Filename="%GAMEDIR%DerivedDataCache/DDC.ddp")
- Is Array:
False
#References in C++ code
#Callsites
This variable is referenced in the following C++ source code:
#Loc: <Workspace>/Engine/Plugins/Runtime/GameFeatures/Source/GameFeatures/Private/GameFeaturePluginStateMachine.cpp:1834
Scope (from outer to inner):
file
function void OnPakFileMounted
Source code excerpt:
void OnPakFileMounted(const IPakFile& PakFile)
{
if (FPakFile* Pak = (FPakFile*)(&PakFile))
{
const FStringView ShortUrl = StateProperties.PluginIdentifier.GetIdentifyingString();
UE_LOG(LogGameFeatures, Display, TEXT("Mounted Pak File for (%.*s) with following files:"), ShortUrl.Len(), ShortUrl.GetData());
TArray<FString> OutFileList;
Pak->GetPrunedFilenames(OutFileList);
for (const FString& FileName : OutFileList)
{
UE_LOG(LogGameFeatures, Display, TEXT("(%s)"), *FileName);
}
}
}
#Loc: <Workspace>/Engine/Source/Developer/PakFileUtilities/Private/PakFileUtilities.cpp:2209
Scope (from outer to inner):
file
function bool FPakWriterContext::Initialize
Source code excerpt:
// set bDoUseOodleDespiteNoPluginCompression = true to let Oodle compress those files
bool bDoUseOodleDespiteNoPluginCompression = false;
GConfig->GetBool(TEXT("Pak"), TEXT("bDoUseOodleDespiteNoPluginCompression"), bDoUseOodleDespiteNoPluginCompression, GEngineIni);
if ( bDoUseOodleDespiteNoPluginCompression && CompressionFormatsAndNone[0] == NAME_Oodle )
{
// NoPluginCompression sets are empty
UE_LOG(LogPakFile, Display, TEXT("Oodle enabled on 'NoPluginCompression' files"));
#Loc: <Workspace>/Engine/Source/Developer/PakFileUtilities/Private/PakFileUtilities.cpp:2220
Scope (from outer to inner):
file
function bool FPakWriterContext::Initialize
Source code excerpt:
{
TArray<FString> ExtensionsToNotUsePluginCompression;
GConfig->GetArray(TEXT("Pak"), TEXT("ExtensionsToNotUsePluginCompression"), ExtensionsToNotUsePluginCompression, GEngineIni);
for (const FString& Ext : ExtensionsToNotUsePluginCompression)
{
NoPluginCompressionExtensions.Add(Ext);
}
TArray<FString> FileNamesToNotUsePluginCompression;
GConfig->GetArray(TEXT("Pak"), TEXT("FileNamesToNotUsePluginCompression"), FileNamesToNotUsePluginCompression, GEngineIni);
for (const FString& FileName : FileNamesToNotUsePluginCompression)
{
NoPluginCompressionFileNames.Add(FileName);
}
}
#Loc: <Workspace>/Engine/Source/Runtime/AssetRegistry/Private/AssetRegistry.cpp:344
Scope (from outer to inner):
file
namespace UE::AssetRegistry::Premade
class class FPreloader
function bool TrySetPath
Source code excerpt:
}
bool TrySetPath(const IPakFile& Pak)
{
for (FString& LocalPath : GetPriorityPaths())
{
if (Pak.PakContains(LocalPath))
{
ARPath = MoveTemp(LocalPath);
return true;
}
}
return false;
#Loc: <Workspace>/Engine/Source/Runtime/AssetRegistry/Private/AssetRegistry.cpp:399
Scope (from outer to inner):
file
namespace UE::AssetRegistry::Premade
class class FPreloader
function void DelayedInitialize
lambda-function
Source code excerpt:
// The PAK with the main registry isn't mounted yet
PakMountedDelegate = FCoreDelegates::GetOnPakFileMounted2().AddLambda([this](const IPakFile& Pak)
{
FScopeLock Lock(&StateLock);
if (LoadState == EState::NotFound && TrySetPath(Pak))
{
KickPreload();
// Remove the callback from OnPakFileMounted2 to avoid wasting time in all future PakFile mounts
// Do not access any of the lambda captures after the call to Remove, because deallocating the
// DelegateHandle also deallocates our lambda captures
FDelegateHandle LocalPakMountedDelegate = PakMountedDelegate;
#Loc: <Workspace>/Engine/Source/Runtime/Engine/Private/AssetManager.cpp:2277
Scope (from outer to inner):
file
function bool UAssetManager::FindMissingChunkList
Source code excerpt:
// Grab pak platform file
FPakPlatformFile* Pak = (FPakPlatformFile*)FPlatformFileManager::Get().FindPlatformFile(TEXT("PakFile"));
check(Pak);
for (const FSoftObjectPath& Asset : AssetList)
{
FAssetData FoundData;
GetAssetDataForPath(Asset, FoundData);
TSet<int32> FoundChunks, MissingChunks, ErrorChunks;
#Loc: <Workspace>/Engine/Source/Runtime/Engine/Private/AssetManager.cpp:2298
Scope (from outer to inner):
file
function bool UAssetManager::FindMissingChunkList
Source code excerpt:
// If chunk install thinks the chunk is available, we need to double check with the pak system that it isn't
// pending decryption
if (Location >= EChunkLocation::LocalSlow && Pak->AnyChunksAvailable())
{
Location = Pak->GetPakChunkLocation(PakchunkId);
}
ChunkLocationCache.Add(PakchunkId, Location);
}
EChunkLocation::Type ChunkLocation = ChunkLocationCache[PakchunkId];
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:187
Scope (from outer to inner):
file
function void FPakPlatformFile::GetPrunedFilenamesInChunk
Source code excerpt:
GetMountedPaks(Paks);
for (const FPakListEntry& Pak : Paks)
{
if (Pak.PakFile && Pak.PakFile->GetFilename() == InPakFilename)
{
Pak.PakFile->GetPrunedFilenamesInChunk(InChunkIDs, OutFileList);
break;
}
}
}
void FPakPlatformFile::GetFilenamesFromIostoreByBlockIndex(const FString& InContainerName, const TArray<int32>& InBlockIndex, TArray<FString>& OutFileList)
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:778
Scope (from outer to inner):
file
function void FPakPlatformFile::GetPrunedFilenamesInPakFile
Source code excerpt:
GetMountedPaks(Paks);
for (const FPakListEntry& Pak : Paks)
{
if (Pak.PakFile && Pak.PakFile->GetFilename() == InPakFilename)
{
Pak.PakFile->GetPrunedFilenames(OutFileList);
break;
}
}
}
void FPakPlatformFile::GetFilenamesFromIostoreContainer(const FString& InContainerName, TArray<FString>& OutFileList)
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:2069
Scope (from outer to inner):
file
class class FPakPrecacher
function uint16* RegisterPakFile
Source code excerpt:
CachedPakData.Add(FPakData(InActualPakFile, Handle, File, PakFileSize));
PakIndexPtr = &CachedPaks.Add(InActualPakFile, static_cast<uint16>(CachedPakData.Num() - 1));
FPakData& Pak = CachedPakData[*PakIndexPtr];
if (OffsetAndPakIndexOfSavedBlocked.Num() == 0)
{
// the 1st cache must exist and is used by all sharing pak files
OffsetAndPakIndexOfSavedBlocked.AddDefaulted(1);
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:2094
Scope (from outer to inner):
file
class class FPakPrecacher
function uint16* RegisterPakFile
Source code excerpt:
}
if (Pak.ActualPakFile->GetCacheType() == FPakFile::ECacheType::Individual || GPakCache_CachePerPakFile != 0 )
{
Pak.ActualPakFile->SetCacheIndex(OffsetAndPakIndexOfSavedBlocked.Num());
OffsetAndPakIndexOfSavedBlocked.AddDefaulted(1);
}
else
{
Pak.ActualPakFile->SetCacheIndex(0);
}
UE_LOG(LogPakFile, Log, TEXT("New pak file %s added to pak precacher."), *PakFilename);
// Load signature data
Pak.Signatures = FPakPlatformFile::GetPakSignatureFile(*PakFilename);
if (Pak.Signatures.IsValid())
{
// We should never get here unless the signature file exists and is validated. The original FPakFile creation
// on the main thread would have failed and the pak would never have been mounted otherwise, and then we would
// never have issued read requests to the pak precacher.
check(Pak.Signatures);
// Check that we have the correct match between signature and pre-cache granularity
int64 NumPakChunks = Align(PakFileSize, FPakInfo::MaxChunkDataSize) / FPakInfo::MaxChunkDataSize;
ensure(NumPakChunks == Pak.Signatures->ChunkHashes.Num());
}
}
return PakIndexPtr;
}
#if !UE_BUILD_SHIPPING
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:2155
Scope (from outer to inner):
file
class class FPakPrecacher
function FJoinedOffsetAndPakIndex FirstUnfilledBlockForRequest
Source code excerpt:
int64 Offset = GetRequestOffset(Request.OffsetAndPakIndex);
int64 Size = Request.Size;
FPakData& Pak = CachedPakData[PakIndex];
check(Offset + Request.Size <= Pak.TotalSize && Size > 0 && Request.GetPriority() >= AIOP_MIN && Request.GetPriority() <= AIOP_MAX && Request.Status != EInRequestStatus::Complete && Request.Owner);
if (PakIndex != GetRequestPakIndex(ReadHead))
{
// this is in a different pak, so we ignore the read head position
ReadHead = 0;
}
if (ReadHead)
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:2184
Scope (from outer to inner):
file
class class FPakPrecacher
function FJoinedOffsetAndPakIndex FirstUnfilledBlockForRequest
Source code excerpt:
}
if (Pak.CacheBlocks[(int32)EBlockStatus::Complete] != IntervalTreeInvalidIndex)
{
OverlappingNodesInIntervalTreeMask<FCacheBlock>(
Pak.CacheBlocks[(int32)EBlockStatus::Complete],
CacheBlockAllocator,
FirstByte,
LastByte,
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
Pak.BytesToBitsShift,
&InFlightOrDone[0]
);
}
if (Request.Status == EInRequestStatus::Waiting && Pak.CacheBlocks[(int32)EBlockStatus::InFlight] != IntervalTreeInvalidIndex)
{
OverlappingNodesInIntervalTreeMask<FCacheBlock>(
Pak.CacheBlocks[(int32)EBlockStatus::InFlight],
CacheBlockAllocator,
FirstByte,
LastByte,
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
Pak.BytesToBitsShift,
&InFlightOrDone[0]
);
}
for (uint32 Index = 0; Index < NumQWords; Index++)
{
if (InFlightOrDone[Index] != MAX_uint64)
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:2236
Scope (from outer to inner):
file
class class FPakPrecacher
function bool AddRequest
Source code excerpt:
uint16 PakIndex = GetRequestPakIndex(Request.OffsetAndPakIndex);
int64 Offset = GetRequestOffset(Request.OffsetAndPakIndex);
FPakData& Pak = CachedPakData[PakIndex];
check(Offset + Request.Size <= Pak.TotalSize && Request.Size > 0 && Request.GetPriority() >= AIOP_MIN && Request.GetPriority() <= AIOP_MAX && Request.Status == EInRequestStatus::Waiting && Request.Owner);
static TArray<uint64> InFlightOrDone;
int64 FirstByte = AlignDown(Offset, PAK_CACHE_GRANULARITY);
int64 LastByte = Align(Offset + Request.Size, PAK_CACHE_GRANULARITY) - 1;
uint32 NumBits = IntCastChecked<uint32>((PAK_CACHE_GRANULARITY + LastByte - FirstByte) / PAK_CACHE_GRANULARITY);
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:2253
Scope (from outer to inner):
file
class class FPakPrecacher
function bool AddRequest
Source code excerpt:
}
if (Pak.CacheBlocks[(int32)EBlockStatus::Complete] != IntervalTreeInvalidIndex)
{
Request.Status = EInRequestStatus::Complete;
OverlappingNodesInIntervalTree<FCacheBlock>(
Pak.CacheBlocks[(int32)EBlockStatus::Complete],
CacheBlockAllocator,
FirstByte,
LastByte,
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
[this, &Pak, FirstByte, LastByte](TIntervalTreeIndex Index) -> bool
{
CacheBlockAllocator.Get(Index).InRequestRefCount++;
MaskInterval(Index, CacheBlockAllocator, FirstByte, LastByte, Pak.BytesToBitsShift, &InFlightOrDone[0]);
return true;
}
);
for (uint32 Index = 0; Index < NumQWords; Index++)
{
if (InFlightOrDone[Index] != MAX_uint64)
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:2284
Scope (from outer to inner):
file
class class FPakPrecacher
function bool AddRequest
Source code excerpt:
if (Request.Status == EInRequestStatus::Waiting)
{
if (Pak.CacheBlocks[(int32)EBlockStatus::InFlight] != IntervalTreeInvalidIndex)
{
Request.Status = EInRequestStatus::InFlight;
OverlappingNodesInIntervalTree<FCacheBlock>(
Pak.CacheBlocks[(int32)EBlockStatus::InFlight],
CacheBlockAllocator,
FirstByte,
LastByte,
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
[this, &Pak, FirstByte, LastByte](TIntervalTreeIndex Index) -> bool
{
CacheBlockAllocator.Get(Index).InRequestRefCount++;
MaskInterval(Index, CacheBlockAllocator, FirstByte, LastByte, Pak.BytesToBitsShift, &InFlightOrDone[0]);
return true;
}
);
for (uint32 Index = 0; Index < NumQWords; Index++)
{
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:2318
Scope (from outer to inner):
file
class class FPakPrecacher
function bool AddRequest
Source code excerpt:
#if PAK_EXTRA_CHECKS
OverlappingNodesInIntervalTree<FCacheBlock>(
Pak.CacheBlocks[(int32)EBlockStatus::InFlight],
CacheBlockAllocator,
FirstByte,
LastByte,
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
[this, &Pak, FirstByte, LastByte](TIntervalTreeIndex Index) -> bool
{
check(0); // if we are complete, then how come there are overlapping in flight blocks?
return true;
}
);
#endif
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:2336
Scope (from outer to inner):
file
class class FPakPrecacher
function bool AddRequest
Source code excerpt:
{
AddToIntervalTree<FPakInRequest>(
&Pak.InRequests[Request.GetPriority()][(int32)Request.Status],
InRequestAllocator,
NewIndex,
Pak.StartShift,
Pak.MaxShift
);
}
check(&Request == &InRequestAllocator.Get(NewIndex));
if (Request.Status == EInRequestStatus::Complete)
{
NotifyComplete(NewIndex);
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:2450
Scope (from outer to inner):
file
class class FPakPrecacher
function void TrimCache
Source code excerpt:
uint16 PakIndex = GetRequestPakIndex(OffsetAndPakIndex);
int64 Offset = GetRequestOffset(OffsetAndPakIndex);
FPakData& Pak = CachedPakData[PakIndex];
MaybeRemoveOverlappingNodesInIntervalTree<FCacheBlock>(
&Pak.CacheBlocks[(int32)EBlockStatus::Complete],
CacheBlockAllocator,
Offset,
Offset,
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
[this](TIntervalTreeIndex BlockIndex) -> bool
{
FCacheBlock &Block = CacheBlockAllocator.Get(BlockIndex);
if (!Block.InRequestRefCount)
{
UE_LOG(LogPakFile, VeryVerbose, TEXT("FPakReadRequest[%016llX, %016llX) Discard Cached"), Block.OffsetAndPakIndex, Block.OffsetAndPakIndex + Block.Size);
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:2513
Scope (from outer to inner):
file
class class FPakPrecacher
function void TrimCache
Source code excerpt:
uint16 PakIndex = GetRequestPakIndex(OffsetAndPakIndex);
int64 Offset = GetRequestOffset(OffsetAndPakIndex);
FPakData& Pak = CachedPakData[PakIndex];
bool bRemovedAll = true;
MaybeRemoveOverlappingNodesInIntervalTree<FCacheBlock>(
&Pak.CacheBlocks[(int32)EBlockStatus::Complete],
CacheBlockAllocator,
Offset,
Offset,
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
[this, CurrentTime, &bRemovedAll](TIntervalTreeIndex BlockIndex) -> bool
{
FCacheBlock &Block = CacheBlockAllocator.Get(BlockIndex);
if (!Block.InRequestRefCount && (CurrentTime - Block.TimeNoLongerReferenced >= GPakCache_TimeToTrim))
{
UE_LOG(LogPakFile, VeryVerbose, TEXT("FPakReadRequest[%016llX, %016llX) Discard Cached Based on Time"), Block.OffsetAndPakIndex, Block.OffsetAndPakIndex + Block.Size);
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:2588
Scope (from outer to inner):
file
class class FPakPrecacher
function void TrimCache
Source code excerpt:
uint16 PakIndex = GetRequestPakIndex(OffsetAndPakIndex);
int64 Offset = GetRequestOffset(OffsetAndPakIndex);
FPakData& Pak = CachedPakData[PakIndex];
MaybeRemoveOverlappingNodesInIntervalTree<FCacheBlock>(
&Pak.CacheBlocks[(int32)EBlockStatus::Complete],
CacheBlockAllocator,
Offset,
Offset,
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
[this](TIntervalTreeIndex BlockIndex) -> bool
{
FCacheBlock &Block = CacheBlockAllocator.Get(BlockIndex);
if (!Block.InRequestRefCount)
{
UE_LOG(LogPakFile, VeryVerbose, TEXT("FPakReadRequest[%016llX, %016llX) Discard Cached"), Block.OffsetAndPakIndex, Block.OffsetAndPakIndex + Block.Size);
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:2624
Scope (from outer to inner):
file
class class FPakPrecacher
function void RemoveRequest
Source code excerpt:
int64 Offset = GetRequestOffset(Request.OffsetAndPakIndex);
int64 Size = Request.Size;
FPakData& Pak = CachedPakData[PakIndex];
check(Offset + Request.Size <= Pak.TotalSize && Request.Size > 0 && Request.GetPriority() >= AIOP_MIN && Request.GetPriority() <= AIOP_MAX && int32(Request.Status) >= 0 && int32(Request.Status) < int32(EInRequestStatus::Num));
bool RequestDontCache = (Request.PriorityAndFlags & AIOP_FLAG_DONTCACHE) != 0;
if (RemoveFromIntervalTree<FPakInRequest>(&Pak.InRequests[Request.GetPriority()][(int32)Request.Status], InRequestAllocator, Index, Pak.StartShift, Pak.MaxShift))
{
int64 OffsetOfLastByte = Offset + Size - 1;
MaybeRemoveOverlappingNodesInIntervalTree<FCacheBlock>(
&Pak.CacheBlocks[(int32)EBlockStatus::Complete],
CacheBlockAllocator,
Offset,
OffsetOfLastByte,
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
[this, OffsetOfLastByte, RequestDontCache](TIntervalTreeIndex BlockIndex) -> bool
{
FCacheBlock &Block = CacheBlockAllocator.Get(BlockIndex);
check(Block.InRequestRefCount);
if (!--Block.InRequestRefCount)
{
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:2676
Scope (from outer to inner):
file
class class FPakPrecacher
function void RemoveRequest
Source code excerpt:
}
);
if (!Pak.ActualPakFile->GetUnderlyingCacheTrimDisabled())
{
TrimCache(false, PakIndex);
}
OverlappingNodesInIntervalTree<FCacheBlock>(
Pak.CacheBlocks[(int32)EBlockStatus::InFlight],
CacheBlockAllocator,
Offset,
Offset + Size - 1,
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
[this](TIntervalTreeIndex BlockIndex) -> bool
{
FCacheBlock &Block = CacheBlockAllocator.Get(BlockIndex);
check(Block.InRequestRefCount);
Block.InRequestRefCount--;
return true;
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:2712
Scope (from outer to inner):
file
class class FPakPrecacher
function void NotifyComplete
Source code excerpt:
uint16 PakIndex = GetRequestPakIndex(Request.OffsetAndPakIndex);
int64 Offset = GetRequestOffset(Request.OffsetAndPakIndex);
FPakData& Pak = CachedPakData[PakIndex];
check(Offset + Request.Size <= Pak.TotalSize && Request.Size > 0 && Request.GetPriority() >= AIOP_MIN && Request.GetPriority() <= AIOP_MAX && Request.Status == EInRequestStatus::Complete);
check(Request.Owner && Request.UniqueID);
if (Request.Status == EInRequestStatus::Complete && Request.UniqueID == Request.Owner->UniqueID && RequestIndex == Request.Owner->InRequestIndex && Request.OffsetAndPakIndex == Request.Owner->OffsetAndPakIndex)
{
UE_LOG(LogPakFile, VeryVerbose, TEXT("FPakReadRequest[%016llX, %016llX) Notify complete"), Request.OffsetAndPakIndex, Request.OffsetAndPakIndex + Request.Size);
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:2756
Scope (from outer to inner):
file
class class FPakPrecacher
function FJoinedOffsetAndPakIndex GetNextBlock
Source code excerpt:
for (; BestNext == MAX_uint64 && PakIndex < CachedPakData.Num(); PakIndex++)
{
FPakData& Pak = CachedPakData[PakIndex];
if (Pak.InRequests[Priority][(int32)EInRequestStatus::Complete] != IntervalTreeInvalidIndex)
{
bAnyOutstanding = true;
}
if (Pak.InRequests[Priority][(int32)EInRequestStatus::Waiting] != IntervalTreeInvalidIndex)
{
uint64 Limit = uint64(Pak.TotalSize - 1);
if (BestNext != MAX_uint64 && GetRequestPakIndex(BestNext) == PakIndex)
{
Limit = GetRequestOffset(BestNext) - 1;
}
OverlappingNodesInIntervalTreeWithShrinkingInterval<FPakInRequest>(
Pak.InRequests[Priority][(int32)EInRequestStatus::Waiting],
InRequestAllocator,
uint64(Offset),
Limit,
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
[this, &Pak, &BestNext, &BestPakIndex, PakIndex, &Limit, LocalLastReadRequest](TIntervalTreeIndex Index) -> bool
{
FJoinedOffsetAndPakIndex First = FirstUnfilledBlockForRequest(Index, LocalLastReadRequest);
check(LocalLastReadRequest != 0 || First != MAX_uint64); // if there was not trimming, and this thing is in the waiting list, then why was no start block found?
if (First < BestNext)
{
BestNext = First;
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:2820
Scope (from outer to inner):
file
class class FPakPrecacher
function bool AddNewBlock
Source code excerpt:
uint16 PakIndex = GetRequestPakIndex(BestNext);
int64 Offset = GetRequestOffset(BestNext);
FPakData& Pak = CachedPakData[PakIndex];
check(Offset < Pak.TotalSize);
int64 FirstByte = AlignDown(Offset, PAK_CACHE_GRANULARITY);
int64 LastByte = FMath::Min(Align(FirstByte + (GPakCache_MaxRequestSizeToLowerLevelKB * 1024), PAK_CACHE_GRANULARITY) - 1, Pak.TotalSize - 1);
check(FirstByte >= 0 && LastByte < Pak.TotalSize && LastByte >= 0 && LastByte >= FirstByte);
uint32 NumBits = IntCastChecked<uint32>((PAK_CACHE_GRANULARITY + LastByte - FirstByte) / PAK_CACHE_GRANULARITY);
uint32 NumQWords = (NumBits + 63) >> 6;
static TArray<uint64> InFlightOrDone;
InFlightOrDone.Reset();
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:2838
Scope (from outer to inner):
file
class class FPakPrecacher
function bool AddNewBlock
Source code excerpt:
}
if (Pak.CacheBlocks[(int32)EBlockStatus::Complete] != IntervalTreeInvalidIndex)
{
OverlappingNodesInIntervalTreeMask<FCacheBlock>(
Pak.CacheBlocks[(int32)EBlockStatus::Complete],
CacheBlockAllocator,
FirstByte,
LastByte,
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
Pak.BytesToBitsShift,
&InFlightOrDone[0]
);
}
if (Pak.CacheBlocks[(int32)EBlockStatus::InFlight] != IntervalTreeInvalidIndex)
{
OverlappingNodesInIntervalTreeMask<FCacheBlock>(
Pak.CacheBlocks[(int32)EBlockStatus::InFlight],
CacheBlockAllocator,
FirstByte,
LastByte,
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
Pak.BytesToBitsShift,
&InFlightOrDone[0]
);
}
static TArray<uint64> Requested;
Requested.Reset();
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:2878
Scope (from outer to inner):
file
class class FPakPrecacher
function bool AddNewBlock
Source code excerpt:
break;
}
if (Pak.InRequests[Priority][(int32)EInRequestStatus::Waiting] != IntervalTreeInvalidIndex)
{
OverlappingNodesInIntervalTreeMask<FPakInRequest>(
Pak.InRequests[Priority][(int32)EInRequestStatus::Waiting],
InRequestAllocator,
FirstByte,
LastByte,
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
Pak.BytesToBitsShift,
&Requested[0]
);
}
if (Priority == AIOP_MIN)
{
break;
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:2929
Scope (from outer to inner):
file
class class FPakPrecacher
function bool AddNewBlock
Source code excerpt:
AddToIntervalTree<FCacheBlock>(
&Pak.CacheBlocks[(int32)EBlockStatus::InFlight],
CacheBlockAllocator,
NewIndex,
Pak.StartShift,
Pak.MaxShift
);
TArray<TIntervalTreeIndex> Inflights;
for (int32 Priority = AIOP_MAX;; Priority--)
{
if (Pak.InRequests[Priority][(int32)EInRequestStatus::Waiting] != IntervalTreeInvalidIndex)
{
MaybeRemoveOverlappingNodesInIntervalTree<FPakInRequest>(
&Pak.InRequests[Priority][(int32)EInRequestStatus::Waiting],
InRequestAllocator,
uint64(FirstByte),
uint64(FirstByte + Size - 1),
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
[this, &Block, &Inflights](TIntervalTreeIndex RequestIndex) -> bool
{
Block.InRequestRefCount++;
if (FirstUnfilledBlockForRequest(RequestIndex) == MAX_uint64)
{
InRequestAllocator.Get(RequestIndex).Next = IntervalTreeInvalidIndex;
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:2966
Scope (from outer to inner):
file
class class FPakPrecacher
function bool AddNewBlock
Source code excerpt:
#if PAK_EXTRA_CHECKS
OverlappingNodesInIntervalTree<FPakInRequest>(
Pak.InRequests[Priority][(int32)EInRequestStatus::InFlight],
InRequestAllocator,
uint64(FirstByte),
uint64(FirstByte + Size - 1),
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
[](TIntervalTreeIndex) -> bool
{
check(0); // if this is in flight, then why does it overlap my new block
return false;
}
);
OverlappingNodesInIntervalTree<FPakInRequest>(
Pak.InRequests[Priority][(int32)EInRequestStatus::Complete],
InRequestAllocator,
uint64(FirstByte),
uint64(FirstByte + Size - 1),
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
[](TIntervalTreeIndex) -> bool
{
check(0); // if this is complete, then why does it overlap my new block
return false;
}
);
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:3005
Scope (from outer to inner):
file
class class FPakPrecacher
function bool AddNewBlock
Source code excerpt:
FPakInRequest& CompReq = InRequestAllocator.Get(Fli);
CompReq.Status = EInRequestStatus::InFlight;
AddToIntervalTree(&Pak.InRequests[CompReq.GetPriority()][(int32)EInRequestStatus::InFlight], InRequestAllocator, Fli, Pak.StartShift, Pak.MaxShift);
}
StartBlockTask(Block);
return true;
}
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:3032
Scope (from outer to inner):
file
class class FPakPrecacher
function bool HasRequestsAtStatus
Source code excerpt:
for (uint16 PakIndex = 0; PakIndex < CachedPakData.Num(); PakIndex++)
{
FPakData& Pak = CachedPakData[PakIndex];
for (int32 Priority = AIOP_MAX;; Priority--)
{
if (Pak.InRequests[Priority][(int32)Status] != IntervalTreeInvalidIndex)
{
return true;
}
if (Priority == AIOP_MIN)
{
break;
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:3129
Scope (from outer to inner):
file
class class FPakPrecacher
function void StartBlockTask
Source code excerpt:
UE_LOG(LogPakFile, VeryVerbose, TEXT("FPakReadRequest[%016llX, %016llX) StartBlockTask"), Block.OffsetAndPakIndex, Block.OffsetAndPakIndex + Block.Size);
uint16 PakIndex = GetRequestPakIndex(Block.OffsetAndPakIndex);
FPakData& Pak = CachedPakData[PakIndex];
RequestsToLower[IndexToFill].BlockIndex = Block.Index;
RequestsToLower[IndexToFill].RequestSize = Block.Size;
RequestsToLower[IndexToFill].Memory = nullptr;
check(&CacheBlockAllocator.Get(RequestsToLower[IndexToFill].BlockIndex) == &Block);
#if USE_PAK_PRECACHE && CSV_PROFILER
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:3167
Scope (from outer to inner):
file
class class FPakPrecacher
function void StartBlockTask
Source code excerpt:
};
RequestsToLower[IndexToFill].RequestHandle = Pak.Handle->ReadRequest(GetRequestOffset(Block.OffsetAndPakIndex), Block.Size, Priority, &CallbackFromLower);
RedundantReadTracker.CheckBlock(GetRequestOffset(Block.OffsetAndPakIndex), Block.Size);
#if CSV_PROFILER
FJoinedOffsetAndPakIndex OldLastReadRequest = LastReadRequest;
LastReadRequest = Block.OffsetAndPakIndex + Block.Size;
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:3199
Scope (from outer to inner):
file
class class FPakPrecacher
function void CompleteRequest
Source code excerpt:
uint16 PakIndex = GetRequestPakIndex(Block.OffsetAndPakIndex);
int64 Offset = GetRequestOffset(Block.OffsetAndPakIndex);
FPakData& Pak = CachedPakData[PakIndex];
check(!Block.Memory && Block.Size);
check(!bWasCanceled); // this is doable, but we need to transition requests back to waiting, inflight etc.
if (!RemoveFromIntervalTree<FCacheBlock>(&Pak.CacheBlocks[(int32)EBlockStatus::InFlight], CacheBlockAllocator, Block.Index, Pak.StartShift, Pak.MaxShift))
{
check(0);
}
if (Block.InRequestRefCount == 0 || bWasCanceled)
{
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:3241
Scope (from outer to inner):
file
class class FPakPrecacher
function void CompleteRequest
Source code excerpt:
Block.Status = EBlockStatus::Complete;
AddToIntervalTree<FCacheBlock>(
&Pak.CacheBlocks[(int32)EBlockStatus::Complete],
CacheBlockAllocator,
Block.Index,
Pak.StartShift,
Pak.MaxShift
);
TArray<TIntervalTreeIndex> Completeds;
for (int32 Priority = AIOP_MAX;; Priority--)
{
if (Pak.InRequests[Priority][(int32)EInRequestStatus::InFlight] != IntervalTreeInvalidIndex)
{
MaybeRemoveOverlappingNodesInIntervalTree<FPakInRequest>(
&Pak.InRequests[Priority][(int32)EInRequestStatus::InFlight],
InRequestAllocator,
uint64(Offset),
uint64(Offset + Block.Size - 1),
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
[this, &Completeds](TIntervalTreeIndex RequestIndex) -> bool
{
if (FirstUnfilledBlockForRequest(RequestIndex) == MAX_uint64)
{
InRequestAllocator.Get(RequestIndex).Next = IntervalTreeInvalidIndex;
Completeds.Add(RequestIndex);
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:3282
Scope (from outer to inner):
file
class class FPakPrecacher
function void CompleteRequest
Source code excerpt:
FPakInRequest& CompReq = InRequestAllocator.Get(Comp);
CompReq.Status = EInRequestStatus::Complete;
AddToIntervalTree(&Pak.InRequests[CompReq.GetPriority()][(int32)EInRequestStatus::Complete], InRequestAllocator, Comp, Pak.StartShift, Pak.MaxShift);
NotifyComplete(Comp); // potentially scary recursion here
}
}
TrimCache();
}
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:3307
Scope (from outer to inner):
file
class class FPakPrecacher
function bool GetCompletedRequestData
Source code excerpt:
int64 Size = DoneRequest.Size;
FPakData& Pak = CachedPakData[PakIndex];
check(Offset + DoneRequest.Size <= Pak.TotalSize && DoneRequest.Size > 0 && DoneRequest.GetPriority() >= AIOP_MIN && DoneRequest.GetPriority() <= AIOP_MAX && DoneRequest.Status == EInRequestStatus::Complete);
int64 BytesCopied = 0;
#if 0 // this path removes the block in one pass, however, this is not what we want because it wrecks precaching, if we change back GetCompletedRequest needs to maybe start a new request and the logic of the IAsyncFile read needs to change
MaybeRemoveOverlappingNodesInIntervalTree<FCacheBlock>(
&Pak.CacheBlocks[(int32)EBlockStatus::Complete],
CacheBlockAllocator,
Offset,
Offset + Size - 1,
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
[this, Offset, Size, &BytesCopied, Result, &Pak](TIntervalTreeIndex BlockIndex) -> bool
{
FCacheBlock &Block = CacheBlockAllocator.Get(BlockIndex);
int64 BlockOffset = GetRequestOffset(Block.OffsetAndPakIndex);
check(Block.Memory && Block.Size && BlockOffset >= 0 && BlockOffset + Block.Size <= Pak.TotalSize);
int64 OverlapStart = FMath::Max(Offset, BlockOffset);
int64 OverlapEnd = FMath::Min(Offset + Size, BlockOffset + Block.Size);
check(OverlapEnd > OverlapStart);
BytesCopied += OverlapEnd - OverlapStart;
FMemory::Memcpy(Result + OverlapStart - Offset, Block.Memory + OverlapStart - BlockOffset, OverlapEnd - OverlapStart);
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:3343
Scope (from outer to inner):
file
class class FPakPrecacher
function bool GetCompletedRequestData
Source code excerpt:
);
if (!RemoveFromIntervalTree<FPakInRequest>(&Pak.InRequests[DoneRequest.GetPriority()][(int32)EInRequestStatus::Complete], InRequestAllocator, DoneRequest.Index, Pak.StartShift, Pak.MaxShift))
{
check(0); // not found
}
ClearRequest(DoneRequest);
#else
OverlappingNodesInIntervalTree<FCacheBlock>(
Pak.CacheBlocks[(int32)EBlockStatus::Complete],
CacheBlockAllocator,
Offset,
Offset + Size - 1,
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
[this, Offset, Size, &BytesCopied, Result, &Pak](TIntervalTreeIndex BlockIndex) -> bool
{
FCacheBlock &Block = CacheBlockAllocator.Get(BlockIndex);
int64 BlockOffset = GetRequestOffset(Block.OffsetAndPakIndex);
check(Block.Memory && Block.Size && BlockOffset >= 0 && BlockOffset + Block.Size <= Pak.TotalSize);
int64 OverlapStart = FMath::Max(Offset, BlockOffset);
int64 OverlapEnd = FMath::Min(Offset + Size, BlockOffset + Block.Size);
check(OverlapEnd > OverlapStart);
BytesCopied += OverlapEnd - OverlapStart;
FMemory::Memcpy(Result + OverlapStart - Offset, Block.Memory + OverlapStart - BlockOffset, OverlapEnd - OverlapStart);
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:3419
Scope (from outer to inner):
file
class class FPakPrecacher
function bool QueueRequest
Source code excerpt:
uint16 PakIndex = *PakIndexPtr;
FPakData& Pak = CachedPakData[PakIndex];
check(Pak.Name == File && Pak.TotalSize == PakFileSize && Pak.Handle);
TIntervalTreeIndex RequestIndex = InRequestAllocator.Alloc();
FPakInRequest& Request = InRequestAllocator.Get(RequestIndex);
FJoinedOffsetAndPakIndex RequestOffsetAndPakIndex = MakeJoinedRequest(PakIndex, Offset);
Request.OffsetAndPakIndex = RequestOffsetAndPakIndex;
Request.Size = Size;
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:3531
Scope (from outer to inner):
file
class class FPakPrecacher
function void Unmount
Source code excerpt:
uint16 PakIndex = It->Value;
TrimCache(true);
FPakData& Pak = CachedPakData[PakIndex];
int64 Offset = MakeJoinedRequest(PakIndex, 0);
bool bHasOutstandingRequests = false;
OverlappingNodesInIntervalTree<FCacheBlock>(
Pak.CacheBlocks[(int32)EBlockStatus::Complete],
CacheBlockAllocator,
0,
Offset + Pak.TotalSize - 1,
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
[&bHasOutstandingRequests](TIntervalTreeIndex BlockIndex) -> bool
{
check(!"Pak cannot be unmounted with outstanding requests");
bHasOutstandingRequests = true;
return false;
}
);
OverlappingNodesInIntervalTree<FCacheBlock>(
Pak.CacheBlocks[(int32)EBlockStatus::InFlight],
CacheBlockAllocator,
0,
Offset + Pak.TotalSize - 1,
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
[&bHasOutstandingRequests](TIntervalTreeIndex BlockIndex) -> bool
{
check(!"Pak cannot be unmounted with outstanding requests");
bHasOutstandingRequests = true;
return false;
}
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:3571
Scope (from outer to inner):
file
class class FPakPrecacher
function void Unmount
Source code excerpt:
{
OverlappingNodesInIntervalTree<FPakInRequest>(
Pak.InRequests[Priority][(int32)EInRequestStatus::InFlight],
InRequestAllocator,
0,
Offset + Pak.TotalSize - 1,
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
[&bHasOutstandingRequests](TIntervalTreeIndex BlockIndex) -> bool
{
check(!"Pak cannot be unmounted with outstanding requests");
bHasOutstandingRequests = true;
return false;
}
);
OverlappingNodesInIntervalTree<FPakInRequest>(
Pak.InRequests[Priority][(int32)EInRequestStatus::Complete],
InRequestAllocator,
0,
Offset + Pak.TotalSize - 1,
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
[&bHasOutstandingRequests](TIntervalTreeIndex BlockIndex) -> bool
{
check(!"Pak cannot be unmounted with outstanding requests");
bHasOutstandingRequests = true;
return false;
}
);
OverlappingNodesInIntervalTree<FPakInRequest>(
Pak.InRequests[Priority][(int32)EInRequestStatus::Waiting],
InRequestAllocator,
0,
Offset + Pak.TotalSize - 1,
0,
Pak.MaxNode,
Pak.StartShift,
Pak.MaxShift,
[&bHasOutstandingRequests](TIntervalTreeIndex BlockIndex) -> bool
{
check(!"Pak cannot be unmounted with outstanding requests");
bHasOutstandingRequests = true;
return false;
}
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:3626
Scope (from outer to inner):
file
class class FPakPrecacher
function void Unmount
Source code excerpt:
{
UE_LOG(LogPakFile, Log, TEXT("Pak file %s removed from pak precacher."), *PakFile.ToString());
if (Pak.ActualPakFile != UnmountedPak)
{
if (UnmountedPak)
{
UE_LOG(LogPakFile, Warning, TEXT("FPakPrecacher::Unmount found multiple PakFiles with the name %s. Unmounting all of them."), *PakFile.ToString());
}
Pak.ActualPakFile->SetIsMounted(false);
}
It.RemoveCurrent();
check(Pak.Handle);
delete Pak.Handle;
Pak.Handle = nullptr;
Pak.ActualPakFile = nullptr;
int32 NumToTrim = 0;
for (int32 Index = CachedPakData.Num() - 1; Index >= 0; Index--)
{
if (!CachedPakData[Index].Handle)
{
NumToTrim++;
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:6591
Scope (from outer to inner):
file
function void FPakFile::PruneDirectoryIndex
Source code excerpt:
// Caller holds WriteLock on DirectoryIndexLock
TArray<FString> FileWildCards, DirectoryWildCards, OldWildCards;
GConfig->GetArray(TEXT("Pak"), TEXT("DirectoryIndexKeepFiles"), FileWildCards, GEngineIni);
GConfig->GetArray(TEXT("Pak"), TEXT("DirectoryIndexKeepEmptyDirectories"), DirectoryWildCards, GEngineIni);
GConfig->GetArray(TEXT("Pak"), TEXT("DirectoryRootsToKeepInMemoryWhenUnloadingPakEntryFilenames"), OldWildCards, GEngineIni); // Legacy name, treated as both KeepFiles and KeepEmptyDirectories
DirectoryWildCards.Append(OldWildCards);
FileWildCards.Append(OldWildCards);
int32 NumKeptEntries = 0;
if (PrunedDirectoryIndex)
{
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:7681
Scope (from outer to inner):
file
function void FPakFile::ValidateDirectorySearch
Source code excerpt:
TArray<FString> WildCards, OldWildCards;
GConfig->GetArray(TEXT("Pak"), TEXT("IndexValidationIgnore"), WildCards, GEngineIni);
auto IsIgnore = [&WildCards](const FString& FilePath)
{
for (const FString& WildCard : WildCards)
{
if (FilePath.MatchesWildcard(WildCard))
{
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:8011
Scope (from outer to inner):
file
function void FPakPlatformFile::HandlePakListCommand
Source code excerpt:
TArray<FPakListEntry> Paks;
GetMountedPaks(Paks);
for (auto Pak : Paks)
{
Ar.Logf(TEXT("%s Mounted to %s"), *Pak.PakFile->GetFilename(), *Pak.PakFile->GetMountPoint());
}
}
void FPakPlatformFile::HandlePakCorruptCommand(const TCHAR* Cmd, FOutputDevice& Ar)
{
#if USE_PAK_PRECACHE
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:8028
Scope (from outer to inner):
file
function void FPakPlatformFile::HandleReloadPakReadersCommand
Source code excerpt:
TArray<FPakListEntry> Paks;
GetMountedPaks(Paks);
for (FPakListEntry& Pak : Paks)
{
Pak.PakFile->RecreatePakReaders(LowerLevel);
}
}
#endif // !UE_BUILD_SHIPPING
FPakPlatformFile::FPakPlatformFile()
: LowerLevel(NULL)
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:8388
Scope: file
Source code excerpt:
// UE_DEPRECATED(4.26, "UnloadPakEntryFilenamesIfPossible is deprecated.")
bool bUnloadPakEntryFilenamesIfPossible = false;
GConfig->GetBool(TEXT("Pak"), TEXT("UnloadPakEntryFilenamesIfPossible"), bUnloadPakEntryFilenamesIfPossible, GEngineIni);
if (bUnloadPakEntryFilenamesIfPossible)
{
UE_LOG(LogPakFile, Warning, TEXT("The UnloadPakEntryFilenamesIfPossible has been deprecated and is no longer sufficient to specify the unloading of pak files.\n")
TEXT("The choice to not load pak files is now made earlier than Ini settings are available.\n")
TEXT("To specify that filenames should be removed from the runtime PakFileIndex, use the new runtime delegate FPakPlatformFile::GetPakSetIndexSettingsDelegate().\n")
TEXT("In a global variable constructor that executes before the process main function, bind this delegate to a function that sets the output bool bKeepFullDirectory to false.\n")
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:8418
Scope: file
Source code excerpt:
if (FPakFile::bSomePakNeedsPruning)
{
for (auto& Pak : Paks)
{
FPakFile* PakFile = Pak.PakFile;
if (PakFile->bWillPruneDirectoryIndex)
{
check(PakFile->bHasPathHashIndex);
FWriteScopeLock DirectoryLock(PakFile->DirectoryIndexLock);
if (PakFile->bNeedsLegacyPruning)
{
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:8445
Scope: file
Source code excerpt:
uint64 EntriesSize = 0;
for (auto& Pak : Paks)
{
FPakFile* PakFile = Pak.PakFile;
{
FPakFile::FScopedPakDirectoryIndexAccess ScopeAccess(*PakFile);
DirectoryHashSize += GetRecursiveAllocatedSize(PakFile->DirectoryIndex);
#if ENABLE_PAKFILE_RUNTIME_PRUNING
{
DirectoryHashSize += GetRecursiveAllocatedSize(PakFile->PrunedDirectoryIndex);
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:8473
Scope (from outer to inner):
file
function bool FPakPlatformFile::Mount
Source code excerpt:
if (LowerLevel->FileExists(InPakFilename))
{
TRefCountPtr<FPakFile> Pak = new FPakFile(LowerLevel, InPakFilename, bSigned, bLoadIndex);
if (Pak.GetReference()->IsValid())
{
if (!Pak->GetInfo().EncryptionKeyGuid.IsValid() || UE::FEncryptionKeyManager::Get().ContainsKey(Pak->GetInfo().EncryptionKeyGuid))
{
if (InPath != nullptr)
{
Pak->SetMountPoint(InPath);
}
FString PakFilename = InPakFilename;
if (PakFilename.EndsWith(TEXT("_P.pak")))
{
// Prioritize based on the chunk version number
// Default to version 1 for single patch system
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:8515
Scope (from outer to inner):
file
function bool FPakPlatformFile::Mount
Source code excerpt:
FPakListEntry Entry;
Entry.ReadOrder = PakOrder;
Entry.PakFile = Pak;
Pak->SetIsMounted(true);
PakFiles.Add(Entry);
PakFiles.StableSort();
if (OutPakListEntry)
{
*OutPakListEntry = MoveTemp(Entry);
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:8529
Scope (from outer to inner):
file
function bool FPakPlatformFile::Mount
Source code excerpt:
else
{
UE_LOG(LogPakFile, Display, TEXT("Deferring mount of pak \"%s\" until encryption key '%s' becomes available"), InPakFilename, *Pak->GetInfo().EncryptionKeyGuid.ToString());
check(!UE::FEncryptionKeyManager::Get().ContainsKey(Pak->GetInfo().EncryptionKeyGuid));
FPakListDeferredEntry& Entry = PendingEncryptedPakFiles[PendingEncryptedPakFiles.Add(FPakListDeferredEntry())];
Entry.Filename = InPakFilename;
Entry.Path = InPath;
Entry.ReadOrder = PakOrder;
Entry.EncryptionKeyGuid = Pak->GetInfo().EncryptionKeyGuid;
Entry.PakchunkIndex = Pak->PakchunkIndex;
Pak.SafeRelease();
return false;
}
}
else
{
UE_LOG(LogPakFile, Warning, TEXT("Failed to mount pak \"%s\", pak is invalid."), InPakFilename);
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:8550
Scope (from outer to inner):
file
function bool FPakPlatformFile::Mount
Source code excerpt:
if (bPakSuccess && IoDispatcherFileBackend.IsValid())
{
FGuid EncryptionKeyGuid = Pak->GetInfo().EncryptionKeyGuid;
FAES::FAESKey EncryptionKey;
if (!UE::FEncryptionKeyManager::Get().TryGetKey(EncryptionKeyGuid, EncryptionKey))
{
if (!EncryptionKeyGuid.IsValid() && FCoreDelegates::GetPakEncryptionKeyDelegate().IsBound())
{
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:8572
Scope (from outer to inner):
file
function bool FPakPlatformFile::Mount
Source code excerpt:
{
UE_LOG(LogPakFile, Display, TEXT("Mounted IoStore container \"%s\""), *UtocPath);
Pak->IoContainerHeader = MakeUnique<FIoContainerHeader>(MountResult.ConsumeValueOrDie());
PackageStoreBackend->Mount(Pak->IoContainerHeader.Get(), PakOrder);
#if WITH_EDITOR
FString OptionalSegmentUtocPath = FPaths::ChangeExtension(InPakFilename, FString::Printf(TEXT("%s.utoc"), FPackagePath::GetOptionalSegmentExtensionModifier()));
if (FPlatformFileManager::Get().GetPlatformFile().FileExists(*OptionalSegmentUtocPath))
{
MountResult = IoDispatcherFileBackend->Mount(*OptionalSegmentUtocPath, PakOrder, EncryptionKeyGuid, EncryptionKey);
if (MountResult.IsOk())
{
Pak->OptionalSegmentIoContainerHeader = MakeUnique<FIoContainerHeader>(MountResult.ConsumeValueOrDie());
PackageStoreBackend->Mount(Pak->OptionalSegmentIoContainerHeader.Get(), PakOrder);
UE_LOG(LogPakFile, Display, TEXT("Mounted optional segment extension IoStore container \"%s\""), *OptionalSegmentUtocPath);
}
else
{
UE_LOG(LogPakFile, Warning, TEXT("Failed to mount optional segment extension IoStore container \"%s\" [%s]"), *OptionalSegmentUtocPath, *MountResult.Status().ToString());
}
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:8621
Scope (from outer to inner):
file
function bool FPakPlatformFile::Mount
Source code excerpt:
// Avoid calling Broadcast if not in use; Broadcast even on an unsubscribed
// non-threadsafe delegate is not threadsafe.
FCoreDelegates::OnPakFileMounted2.Broadcast(*Pak);
}
PRAGMA_ENABLE_DEPRECATION_WARNINGS
FCoreDelegates::GetOnPakFileMounted2().Broadcast(*Pak);
}
UE_LOG(LogPakFile, Display, TEXT("Mounted Pak file '%s', mount point: '%s'"), InPakFilename, *Pak->GetMountPoint());
UE_LOG(LogPakFile, Verbose, TEXT("OnPakFileMounted2Time == %lf"), OnPakFileMounted2Time);
// skip this check for the default mountpoint, it is a frequently used known-good mount point
FString NormalizedPakMountPoint = FPaths::CreateStandardFilename(Pak->GetMountPoint());
bool bIsMountingToRoot = NormalizedPakMountPoint == FPaths::CreateStandardFilename(FPaths::RootDir());
#if WITH_EDITOR
bIsMountingToRoot |= NormalizedPakMountPoint == FPaths::CreateStandardFilename(FPaths::GameFeatureRootPrefix());
#endif
if (!bIsMountingToRoot)
{
FString OutPackageName;
const FString& MountPoint = Pak->GetMountPoint();
if (!FPackageName::TryConvertFilenameToLongPackageName(MountPoint, OutPackageName))
{
// Possibly the mount point is a parent path of mount points, e.g. <ProjectRoot>/Plugins,
// parent path of <ProjectRoot>/Plugins/PluginA and <ProjectRoot>/Plugins/PluginB.
// Do not warn in that case.
FString MountPointAbsPath = FPaths::ConvertRelativePathToFull(MountPoint);
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:8668
Scope (from outer to inner):
file
function bool FPakPlatformFile::Mount
Source code excerpt:
else
{
Pak.SafeRelease();
}
}
else
{
UE_LOG(LogPakFile, Warning, TEXT("Failed to open pak \"%s\""), InPakFilename);
}
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:8746
Scope (from outer to inner):
file
function bool FPakPlatformFile::ReloadPakReaders
Source code excerpt:
TArray<FPakListEntry> Paks;
GetMountedPaks(Paks);
for (FPakListEntry& Pak : Paks)
{
if (!Pak.PakFile->RecreatePakReaders(LowerLevel))
{
return false;
}
}
if (IoDispatcherFileBackend)
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:8834
Scope (from outer to inner):
file
function int32 FPakPlatformFile::MountAllPakFiles
Source code excerpt:
TSet<FString> ExistingPaksFileName;
// Find the single pak we just mounted
for (const FPakListEntry& Pak : ExistingPaks)
{
ExistingPaksFileName.Add(Pak.PakFile->GetFilename());
}
for (int32 PakFileIndex = 0; PakFileIndex < FoundPakFiles.Num(); PakFileIndex++)
{
const FString& PakFilename = FoundPakFiles[PakFileIndex];
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp:8902
Scope (from outer to inner):
file
function IPakFile* FPakPlatformFile::HandleMountPakDelegate
Source code excerpt:
}
FPakListEntry Pak;
if (Mount(*PakFilePath, PakOrder, nullptr, true, &Pak))
{
return Pak.PakFile;
}
return nullptr;
}
bool FPakPlatformFile::HandleUnmountPakDelegate(const FString& PakFilePath)
{
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Public/IPlatformFilePak.h:2493
Scope (from outer to inner):
file
class class FPakPlatformFile : public IPlatformFile
function FString ConvertToPakRelativePath
Source code excerpt:
* @param Relative filename.
*/
FString ConvertToPakRelativePath(const TCHAR* Filename, const FPakFile* Pak)
{
FString RelativeFilename(Filename);
return RelativeFilename.Mid(Pak->GetMountPoint().Len());
}
FString ConvertToAbsolutePathForExternalAppForRead(const TCHAR* Filename) override
{
// Check in Pak file first
TRefCountPtr<FPakFile> Pak;
if (FindFileInPakFiles(Filename, &Pak))
{
return FString::Printf(TEXT("Pak: %s/%s"), *Pak->GetFilename(), *ConvertToPakRelativePath(Filename, Pak));
}
else
{
return LowerLevel->ConvertToAbsolutePathForExternalAppForRead(Filename);
}
}
#Loc: <Workspace>/Engine/Source/Runtime/PakFile/Public/IPlatformFilePak.h:2516
Scope (from outer to inner):
file
class class FPakPlatformFile : public IPlatformFile
function FString ConvertToAbsolutePathForExternalAppForWrite
Source code excerpt:
{
// Check in Pak file first
TRefCountPtr<FPakFile> Pak;
if (FindFileInPakFiles(Filename, &Pak))
{
return FString::Printf(TEXT("Pak: %s/%s"), *Pak->GetFilename(), *ConvertToPakRelativePath(Filename, Pak));
}
else
{
return LowerLevel->ConvertToAbsolutePathForExternalAppForWrite(Filename);
}
}