Video summary

Physically Based Cel Shading

Main summary

Key takeaways

Technology

Tech summary: “Physically Based Cel Shading” (UE-style postprocess/material workflow)

Core thesis / why most “cell shading in real engines” tutorials fail

  • Most tutorials fail because you can’t get a true non-photorealistic cel look by doing ad-hoc edits inside a physically based engine without ending up with messy “workarounds.”
  • In a deferred/far rendering pipeline, lighting is already computed before you can reliably intervene per-light/per-surface.
  • Therefore, without changing engine source code, the only sensible place to manipulate the final lighting result is the post-process stage—but post-processing only sees the final composed result, not individual light contributions.

Initial common approach (postprocess) and its problems

A typical tutorial workflow the speaker critiques:

  • Create an Unbound PostProcess Volume.
  • Use a Post Process Material running before the tone mapper (so it operates in HDR space).
  • Start from PostProcessInput0 (HDR render output).
  • Build a cel look by:
    • Sampling diffuse color (or a similar pass),
    • Desaturating to compute brightness/bands,
    • Using IF nodes / thresholds to quantize brightness into a small number of bands.

Key issues highlighted:

  • Sky becomes black
    • Sky data isn’t present in the buffers used for the diffuse pass (J-buffers mention).
  • Metals become black
    • Diffuse doesn’t contain their color; color effectively shifts into specular through the pipeline.
  • Loss of light color
    • Desaturation removes color info; later attempts to reintroduce color from base color can lose the feeling of “lit by light.”
  • A cited attempted fix: “prismatica devs”
    • Normalize the postprocess input to reintroduce hue.
    • The speaker argues the normalization is mathematically incorrect for full color recovery (sphere vs cube problem in RGB space).
    • Also, desaturation-derived “brightness” isn’t mathematically correct for perceived/accurate exposure.

Net result:

  • Even with patches, the approach has “fallacies,” and complexity increases.

A more correct solution: quantize exposure (EV) in HDR while respecting PBR

Instead of banding “diffuse brightness,” the speaker builds cel shading by working with exposure values (EV) directly, preserving physically based behavior.

Steps / tech details emphasized

  • Ensure postprocess runs before tone mapping so inputs are in HDR and correspond to physically meaningful luminance.
  • Brightness extraction:
    • Compute brightness from RGB via a max component approach (described as correct for “pure-color” brightness/exposure).
  • EV derivation:
    • Luminance varies across a huge nonlinear range.
    • Use log logic (base-2 log) to map scene luminance into an EV space where banding works cleanly.
  • Banding workflow:
    • Convert color representation (speaker uses HSV/HSL/HSB) to separate hue from value/brightness.
    • Quantize the V (brightness/EV) into discrete steps.
    • Reconstruct the final image by converting back to RGB and reapplying base color.
  • Smooth quantization:
    • A “smooth step” style material function reduces harsh artifacts by smoothing between steps.
    • Quantization compression controls let you adjust band density/intensity.

Additional stylization passes (hue quantization, sky mask, metals control)

Beyond EV banding, additional artistic constraints preserve pipeline consistency:

  • Hue quantization

    • Restrict lighting hues to a limited palette (speaker mentions something like primary/secondary/tertiary-ish groups—12 hues).
    • Keep some blur between bands instead of maximum contrast to avoid an unpleasant look.

  • Sky fix

    • Create a mask from base color to detect sky regions with near-zero base values.
    • Restore the original sky region without cel quantization artifacts.

  • Metals refinement

    • Metals often produce too many EV steps.
    • Use the metallic pass as a mask to apply fewer EV bands on metallic surfaces only.
    • Add a custom “fake rim/lighting” effect on metals (see next section).

Fake rim reflection / physically-inspired enhancement for metallic surfaces

To make metal look more alive without “breaking PBR,” the speaker uses a controlled rim mask:

  • Compute a rim-like mask using a dot(camera vector, world normals) (Fresnel-like behavior).
  • Optional improvement:
    • Convert cosine → angle behavior for more linear thresholding.
  • Shape the rim:
    • Scale/remap and square the gradient to control the rim shape.
  • Modulate EV using metallic mask + rim mask:
    • Raise EV on the upper silhouette of metal surfaces by a fixed amount,
    • Lower EV on the lower silhouette by the same amount.
  • Framed as self-balanced and stable across lighting conditions.

Camera exposure utilities (EV-based masking for chromatic stylization)

  • Use pre-exposure and compute camera EV via log2.
  • Compare camera EV offset vs scene EV to create a mask for shadows/highlights.
  • Use the mask to interpolate between two colors and multiply by scene brightness for chromatic stylization.
  • This feature is described as “most dangerous” and is disabled in the final example.

Outline generation: EV-safe outlines via depth/normal edge filter

Outline rendering is added as an extra postprocess/filter step:

  • Use an existing outline filter (speaker references a downloadable sample project).
  • Core approach:
    • Detect edges via a filter (depth-based and/or normal-based),
    • Threshold to decide outline presence,
    • Multiply outlines over the shaded result.

Improved outline behavior:

  • Distance fading
    • Thins outlines with distance rather than fading them via transparency.

Implementation tweak:

  • Expose kernel size as an input in a custom node.
  • Clamp/floor kernel size to integers.
  • Avoid degenerate behavior when kernel size < 2 (return no outline).

More advanced “EV-consistent” outline blending:

  • Goal: add “black lines” without unintentionally darkening exposure.
  • Treat the outline filter output as an EV offset (positive/negative for outside/inside edges).
  • Combine depth-outline with another component (camera-vector related) for better behavior.
  • Because filter outputs can be overly sensitive (picking up triangle edges):
    • Separate sign and absolute value,
    • Apply thresholds/clamps to suppress near-zero edge responses,
    • Use different clamp thresholds for inner vs outer outlines,
    • Apply the result before converting EV back to brightness.

Production/quality guidance: scene setup constraints

Before shader work, match physically based constraints:

  • Proper lighting setup is mandatory.
    • Example values given for a sun: lux and source angle aimed at sharp shadows.
  • Postprocess tuned to create a low-contrast “flat” base before cel operations:
    • Tone mapper parameters (slope/toe/white clip referenced),
    • Remove bloom/lens flares/vignette and reduce color corrections temporarily.
  • GI choice (Lumen)
    • Keep Lumen on, but reduce artifacts by lowering update speed and using higher quality where needed.
    • Disabling Lumen is possible; then SSAO could be enabled if desired.

Material pipeline rules for consistent results:

  • Use a single test material with parameters for base color/specular/roughness/metallic driven via Custom Primitive Data.
  • “Default setup” for cel-friendly materials:
    • Specular = 0
    • Roughness = 1
    • Metallic = 0
    • This yields pure diffuse shading (direction-based) without specular highlights moving with the camera.
  • Base color saturation cap
    • Base color saturation capped around 0.5.
    • High saturation reportedly introduces issues (via base color buffer zeros), causing inconsistent results.

Download/support references

  • Mentions the final result/project can be downloaded via the video description.
  • Notes the sample project includes material functions (e.g., smooth-step quantization) and the output outline filter.
  • Thanks patrons; offers a discount code for the download.

Main speakers / sources

  • Primary speaker/creator: The author of the video explaining the physically based cel shading method.
  • Referenced external sources:
    • “prismatica devs” (normalization-based fix attempt for sky/metal/color issues).
    • Epic’s Substrate / new material pipeline (context for why diffuse vs metal color data behaves differently).
  • Referenced community/series:
    • Prior “lighting in video games” series by “Aros” (lighting knowledge informing the method).
  • Referenced tool/documentation context:
    • Mentions that documentation explaining nodes better would have helped (no specific author named).

Original video