CustomStencil
CustomStencil
#Overview
name: CustomStencil
The value of this variable can be defined or overridden in .ini config files. 1
.ini config file referencing this setting variable.
It is referenced in 11
C++ source files.
#Summary
#Usage in the C++ source code
The purpose of CustomStencil is to provide a way for developers to assign custom stencil values to objects in the scene, which can be used for various rendering and masking effects. This variable is part of Unreal Engine’s rendering system, specifically related to custom depth and stencil functionality.
CustomStencil is primarily used in the following Unreal Engine subsystems and modules:
- Movie Render Pipeline: Used in the MoviePipelineDeferredPasses for configuring custom stencil values during movie rendering.
- Material System: Referenced in material translation and expression evaluation.
- Rendering System: Used in custom depth rendering, mobile shading, and Nanite composition.
The value of CustomStencil is typically set on primitive components using the SetCustomDepthStencilValue function. It can also be modified through material expressions and shader code.
CustomStencil interacts with other variables such as:
- bRenderCustomDepth: Determines whether custom depth rendering is enabled for a component.
- CustomDepthStencilWriteMask: Controls which bits of the stencil buffer are affected by the custom stencil value.
Developers should be aware of the following when using CustomStencil:
- It’s part of the custom depth/stencil rendering system, which may have performance implications if overused.
- The availability and behavior of CustomStencil can vary depending on the rendering path (e.g., forward vs. deferred) and platform.
- For mobile platforms, there may be limitations on how CustomStencil can be sampled in shaders.
Best practices for using CustomStencil include:
- Use it sparingly and only when necessary for specific rendering effects or gameplay mechanics.
- Be aware of the performance impact, especially on mobile platforms.
- Coordinate its usage with other systems that may rely on stencil buffer functionality.
- When using it in materials, consider the material domain and any platform-specific limitations.
- In movie rendering scenarios, carefully manage the stencil values to achieve desired layering effects.
By following these guidelines, developers can effectively utilize CustomStencil for advanced rendering techniques while maintaining performance and cross-platform compatibility.
#Setting Variables
#References In INI files
Location: <Workspace>/Engine/Config/BaseEngine.ini:2904, section: [Engine.BufferVisualizationMaterials]
- INI Section:
Engine.BufferVisualizationMaterials
- Raw value:
(Material="/Engine/BufferVisualization/CustomStencil.CustomStencil", Name=LOCTEXT("BaseCustomStencilMat", "Custom Stencil"))
- Is Array:
False
#References in C++ code
#Callsites
This variable is referenced in the following C++ source code:
#Loc: <Workspace>/Engine/Plugins/MovieScene/MovieRenderPipeline/Source/MovieRenderPipelineRenderPasses/Private/MoviePipelineDeferredPasses.cpp:588
Scope (from outer to inner):
file
function void UMoviePipelineDeferredPassBase::RenderSample_GameThreadImpl
function FStencilValues
Source code excerpt:
: bRenderCustomDepth(false)
, StencilMask(ERendererStencilMask::ERSM_Default)
, CustomStencil(0)
{
}
bool bRenderCustomDepth;
ERendererStencilMask StencilMask;
int32 CustomStencil;
};
// Now for each stencil layer we reconfigure all the actors custom depth/stencil
TArray<FString> AllStencilLayerNames = GetStencilLayerNames();
if (bAddDefaultLayer)
{
#Loc: <Workspace>/Engine/Plugins/MovieScene/MovieRenderPipeline/Source/MovieRenderPipelineRenderPasses/Private/MoviePipelineDeferredPasses.cpp:621
Scope (from outer to inner):
file
function void UMoviePipelineDeferredPassBase::RenderSample_GameThreadImpl
Source code excerpt:
FStencilValues& Values = PreviousValues.Add(PrimitiveComponent);
Values.StencilMask = PrimitiveComponent->CustomDepthStencilWriteMask;
Values.CustomStencil = PrimitiveComponent->CustomDepthStencilValue;
Values.bRenderCustomDepth = PrimitiveComponent->bRenderCustomDepth;
}
}
}
}
}
#Loc: <Workspace>/Engine/Plugins/MovieScene/MovieRenderPipeline/Source/MovieRenderPipelineRenderPasses/Private/MoviePipelineDeferredPasses.cpp:708
Scope (from outer to inner):
file
function void UMoviePipelineDeferredPassBase::RenderSample_GameThreadImpl
Source code excerpt:
for (TPair<UPrimitiveComponent*, FStencilValues>& KVP : PreviousValues)
{
KVP.Key->SetCustomDepthStencilValue(KVP.Value.CustomStencil);
KVP.Key->SetCustomDepthStencilWriteMask(KVP.Value.StencilMask);
KVP.Key->SetRenderCustomDepth(KVP.Value.bRenderCustomDepth);
}
}
}
}
#Loc: <Workspace>/Engine/Source/Runtime/Engine/Private/Materials/HLSLMaterialTranslator.cpp:7478
Scope (from outer to inner):
file
function void FHLSLMaterialTranslator::UseSceneTextureId
Source code excerpt:
if (!bSceneTextureSupportsDecal)
{
// Note: For DBuffer decals CustomDepth and CustomStencil are not available if r.CustomDepth.Order = 1
Errorf(TEXT("Decals can only access SceneDepth, CustomDepth, CustomStencil, and WorldNormal."));
}
const bool bSceneTextureRequiresSM5 = SceneTextureId == PPI_WorldNormal;
if (bSceneTextureRequiresSM5)
{
#Loc: <Workspace>/Engine/Source/Runtime/Engine/Private/Materials/MaterialExpressionHLSL.cpp:2119
Scope (from outer to inner):
file
function bool UMaterialExpressionSceneTexture::GenerateHLSLExpression
Source code excerpt:
if (!bSceneTextureSupportsDecal)
{
// Note: For DBuffer decals CustomDepth and CustomStencil are not available if r.CustomDepth.Order = 1
return Generator.Error(TEXT("Decals can only access SceneDepth, CustomDepth, CustomStencil, and WorldNormal."));
}
}
if (SceneTextureId == PPI_SceneColor && MaterialDomain != MD_Surface)
{
#Loc: <Workspace>/Engine/Source/Runtime/Engine/Public/MaterialSceneTextureId.h:65
Scope: file
Source code excerpt:
/** Ambient Occlusion, single channel */
PPI_AmbientOcclusion UMETA(DisplayName="Ambient Occlusion"),
/** Scene stencil, contains CustomStencil mesh property of the opaque objects rendered with CustomDepth */
PPI_CustomStencil UMETA(DisplayName="CustomStencil"),
/** Material base, RGB color (GBuffer) */
PPI_StoredBaseColor UMETA(DisplayName="BaseColor (as stored in GBuffer)"),
/** Material specular, single channel (GBuffer) */
PPI_StoredSpecular UMETA(DisplayName="Specular (as stored in GBuffer)"),
/** Scene Velocity */
#Loc: <Workspace>/Engine/Source/Runtime/Engine/Public/MaterialShared.h:2181
Scope: file
Source code excerpt:
ENGINE_API bool MaterialUsesSceneDepthLookup_GameThread() const;
/** The material usage mask of CustomDepth and CustomStencil*/
ENGINE_API uint8 GetCustomDepthStencilUsageMask_GameThread() const;
/** Note: This function is only intended for use in deciding whether or not shader permutations are required before material translation occurs. */
ENGINE_API bool MaterialMayModifyMeshPosition() const;
/** Get the runtime virtual texture output attribute mask for the material. */
#Loc: <Workspace>/Engine/Source/Runtime/Engine/Public/SceneTexturesConfig.h:199
Scope: file
Source code excerpt:
uint32 bPreciseDepthAux : 1;
// (Mobile) True if CustomStencil are sampled in a shader
uint32 bSamplesCustomStencil : 1;
// (Mobile) True if MSAA targets can be memoryless
uint32 bMemorylessMSAA : 1;
// (XR) True if we can request an XR depth swapchain
#Loc: <Workspace>/Engine/Source/Runtime/Renderer/Private/CustomDepthRendering.cpp:394
Scope (from outer to inner):
file
function bool FSceneRenderer::RenderCustomDepthPass
Source code excerpt:
{
const FSceneTexturesConfig& Config = FSceneTexturesConfig::Get();
// TextureView is not supported in GLES, so we can't lookup CustomDepth and CustomStencil from a single texture
// Do a copy of the CustomDepthStencil texture if CustomStencil is sampled in a shader.
if (IsOpenGLPlatform(ShaderPlatform))
{
if (Config.bSamplesCustomStencil)
{
FRDGTextureRef CustomStencil = GraphBuilder.CreateTexture(CustomDepthTextures.Depth->Desc, TEXT("CustomStencil"));
AddCopyTexturePass(GraphBuilder, CustomDepthTextures.Depth, CustomStencil);
CustomDepthTextures.Stencil = GraphBuilder.CreateSRV(FRDGTextureSRVDesc::CreateWithPixelFormat(CustomStencil, PF_X24_G8));
}
}
else
{
CustomDepthTextures.Stencil = GraphBuilder.CreateSRV(FRDGTextureSRVDesc::CreateWithPixelFormat(CustomDepthTextures.Depth, PF_X24_G8));
}
#Loc: <Workspace>/Engine/Source/Runtime/Renderer/Private/MobileShadingRenderer.cpp:134
Scope: file
Source code excerpt:
{
bool bUsesCustomDepthStencil = false;
// whether CustomStencil is sampled as a textures
bool bSamplesCustomStencil = false;
};
static FMobileCustomDepthStencilUsage GetCustomDepthStencilUsage(const FViewInfo& View)
{
FMobileCustomDepthStencilUsage CustomDepthStencilUsage;
#Loc: <Workspace>/Engine/Source/Runtime/Renderer/Private/Nanite/NaniteComposition.cpp:677
Scope (from outer to inner):
file
function void EmitCustomDepthStencilTargets
Source code excerpt:
PassParameters->ClusterPageData = Nanite::GStreamingManager.GetClusterPageDataSRV(GraphBuilder);
PassParameters->CustomDepth = CustomDepth;
PassParameters->CustomStencil = CustomStencilSRV;
PassParameters->RenderTargets[0] = OutCustomStencil ? FRenderTargetBinding(OutCustomStencil, ERenderTargetLoadAction::ENoAction) : FRenderTargetBinding();
PassParameters->RenderTargets.DepthStencil = FDepthStencilBinding(OutCustomDepth, ERenderTargetLoadAction::ENoAction, StencilLoadAction, FExclusiveDepthStencil::DepthWrite_StencilNop);
FPixelShaderUtils::AddFullscreenPass(
GraphBuilder,
View.ShaderMap,