VRChat stencil shader techniques give creators powerful control over silhouette and outline effects in virtual worlds. These approaches help avatars and objects stand out with crisp edges, especially in stylized or crowded social scenes.
By leveraging depth and normals, stencil buffers enable reliable rendering of outlines without expensive overdraw. The following sections detail setup, optimization, and troubleshooting for VRChat projects using the Udon or Unity shader graph.
| Topic | Detail | Benefit | Tool Support |
|---|---|---|---|
| Silhouette Rendering | Draws an expanded mesh with inverted normals to capture outlines | Clear edge emphasis without modifying base textures | Shader Graph, Udon + Transforms |
| Depth-Based Detection | Uses depth differences to decide where outlines appear | Consistent lines even with complex camera angles | Depth Normals node, Camera Space |
| Normals-Based Outlines | Extrudes vertices where surface normals diverge strongly | Highlights sharp edges and stylized contours | Normal strength slider, Outline color |
| Performance Safeguards | Culling, LOD, and draw call control | Stable FPS in busy avatar hubs | Shader variants, material reuse |
Understanding Stencil Buffer Basics
The stencil buffer stores per-pixel integer tags that shaders can test and modify. In VRChat projects, these tags decide whether certain pixels should draw an outline or skip the effect entirely.
Designers set stencil operations in the shader, including Keep, Replace, Add, and Compare. By combining these with depth and normals tests, creators can isolate edges that remain visible only when conditions match.
This approach avoids the heavy overhead of post-processing edge detection. Controlled stencil logic keeps performance predictable on the varied hardware used by VRChat users.
Unity Shader Graph nodes expose most required settings, while Udon behavior scripts allow runtime control for dynamic silhouette toggling.
Stencil Setup in Unity Shader Graph
Building the Base Pass
Create a Standard Shader Graph with albedo, normal, and smoothness inputs. Use Sub Shader and Pass settings to ensure the base material renders without unwanted transparency ordering issues.
Adding Stencil Tags
Insert Stencil Op blocks that Write or Compare mask values. Common patterns include marking the object ID, then using Reference and ReadMask to control subsequent passes.
Outlining Pass Configuration
Add a second Pass set to ZWrite On and Cull Front. Scale vertices along normals using the Normal node and Strength slider, then apply Stencil Compare to allow drawing only where reference tags match.
Optimizing for VRChat Avatars
Draw Call Management
Combine materials where possible and minimize shader variants. Shared stencil logic across similar meshes cuts down on state changes and keeps frame times stable.
Level of Detail Strategies
Use LOD groups to disable expensive outline passes at distance. Lower strength and thinner silhouettes for far avatars reduce overdraw without breaking visual recognition.
Instancing and Material Properties
Enable GPU Instancing for repeated props and accessories. Dynamic keywords let users switch effects on or off from Udon without editing the shader each time.
Performance and Debugging Tips
Profile with Unity Frame Debugger to verify that stencil operations do not cause redundant draws. Watch for overlapping masks that could prevent expected outlines from appearing.
Test on integrated and mobile GPUs, since depth precision varies. Adjust camera near and far planes to improve depth buffer accuracy, which stabilizes outline consistency.
Keep fallback shaders simple to support older devices while still conveying the core silhouette intent to all players.
Best Practices and Next Steps
- Define a clear reference value for stencil tags to avoid accidental masking
- Use consistent cull mode across silhouette passes to prevent missing edges
- Limit outline intensity and thickness to maintain readability on small avatars
- Share materials across similar assets to reduce draw call overhead
- Test edge cases such as fast motion, tight camera angles, and complex joints
FAQ
Reader questions
Why are my outlines flickering when the camera moves quickly?
This usually stems from depth precision issues or culling artifacts. Stabilize the near and far plane distances, ensure consistent backface culling, and consider a slightly thicker extrusion in the silhouette pass to reduce flicker.
Can I use stencil shaders with Udon and dynamic objects?
Yes, you can drive stencil reference values via Udon scripts at runtime. Sync object IDs each frame, and use shared materials so that changing tags immediately affects silhouette visibility.
How do I keep outlines from appearing through walls or other avatars?
Add depth or occlusion checks before applying the stencil mask. Compare the depth buffer or use a ray-based fallback in Udon to verify that the silhouette fragment faces an unobstructed view direction.
Will outline shaders break VRChat emissive materials or transparency?
Not if the passes are ordered correctly. Set the outline pass to render after opaque geometry but before transparent layers, and disable depth writing for transparent parts to preserve expected blending behavior.