Introduction
If you've played a recent build of TeamSpec, you might have noticed that the guns and the characters look a lot better than they used to. The glowing parts of a few weapons finally glow in first person, the normal maps on the guns actually show up, and every player has a rim light around their silhouette, similar to the one TF2 is known for. All of that came out of one engine change. I taught Godot how to stack small shaders on top of whatever material a mesh is already drawing with, and how to turn each of them on and off per object. I call them stacked shaders. In this article I'll go over the problem they solve, how they work, and what TeamSpec uses them for. I think anyone who has tried to make a first-person viewmodel in Godot will find this useful, and probably a lot of people who haven't.
Here's some quick vocabulary to better understand this article if you don't work with shaders much:
- Shader: a small program that runs on the GPU. It decides where each vertex of a mesh ends up on screen and what color each pixel gets.
- Material: a shader plus the values it gets fed (textures, colors, numbers). In Godot, every surface of a mesh is drawn with a material.
- Pass: one time the GPU draws a mesh. A mesh usually gets drawn several times per frame: once into the depth buffer, once for its color, and once into each shadow map that can see it.
- Pipeline: the compiled, ready-to-run version of a shader together with all the settings it gets drawn with. Compiling one can take long enough to cause a visible stutter if it happens in the middle of a game.
- Viewmodel: the arms and gun you see in first person.
The problem: Godot thinks in materials
In Godot, shader code belongs to a material. If you want something to look different, you change its material. That's fine most of the time, but some effects don't really belong to a material. They belong to an object.
The best example is the viewmodel. The arms and the gun in first person are drawn with their own field of view, so the gun looks the same no matter what FOV you play at, and they always have to draw in front of the world, so your gun doesn't clip into a wall when you walk up to it. Both of those are shader tricks: you swap the projection matrix in the vertex shader, and you mess with the depth. The problem is that the arms and the gun use dozens of materials between them. Every weapon has its own, and the arms are a second copy of your character model, wearing whatever cosmetics you have equipped. Each of those materials has its own shader that knows nothing about viewmodels, and some of them are imported straight from a model file, so there isn't even any shader code for me to edit.
The rim light has the same problem. It goes on everything drawn on a player: the body, every cosmetic, the gun in their hands and the guns in their holsters. That's a lot of materials, made by different people at different times, and I didn't want to touch any of them.
How I used to do it
For a long time TeamSpec handled this the brute force way. A converter took every material on the viewmodel, grabbed its shader code (or built one from a template for Godot's standard materials), spliced the viewmodel code into it as text, and compiled the result as a brand new shader.
It worked, but it was the kind of code you never stop fixing, because shader code isn't meant to be glued together as text. A local variable in the injected code happened to share its name with one in the afterburn shader (the fire on a burning player), so the combined shader didn't compile, and what you saw in game was an arm drawn completely black. Render modes conflicted, uniforms got declared twice, two lighting functions can't be merged, and materials rebuilt from the template quietly lost things like their emission, so the glowing parts of some guns just didn't glow in first person. And since every converted material was a brand new shader, each one compiled the first time it got drawn, so the game also had to bake all of them ahead of time to avoid stutters.
Then I added the rim light, which came with a problem of its own: a regular Godot shader can't read the ambient light. Godot multiplies the ambient light by your albedo at the very end of its lighting code, and you never get to see the value. The most important part of a TF2-style rim (the top edges of a character glowing even when they're standing in shade) is built on the ambient light. The workaround was to draw the rim as a second pass over every character mesh, which means every character mesh gets drawn twice per frame, and that cost grows with every player on the server.
What I wanted instead
What I wanted was easy to describe. I wanted to write an effect once as its own small shader, turn it on for any mesh by name, in any combination with other effects, and have it run inside whatever material that mesh already draws with, in the same pass. If you've used Blender, it's a lot like its modifier stack, but for shaders: each one runs on top of whatever the ones before it produced, which is why I call them stacked shaders. The idea isn't new, either. Apple's SceneKit has had something similar for years (it calls them shader modifiers), which injects small pieces of shader code into its built-in materials at fixed points.
There was one more requirement, and it ended up shaping the whole design: turning a stacked shader on or off must never compile anything. I had been burned by compile stutters enough already.
Compile once per material, switch per object
The obvious way to stack shaders is to generate a new shader for every combination: material A with the lens, material A with the lens and the rim, material B with just the rim, and so on. That's just the converter again with extra steps. Every new combination is a new shader that compiles on its first draw, and the number of combinations explodes as you add more effects.
So I went the other way. Every material that gets drawn by at least one object with a stacked shader on gets exactly one extra version of its shader, which I call its variant. The variant contains the material's own code plus every stacked shader in the project, and each one's code sits behind a check that basically asks "does this object have my bit set?" Each object carries a small bitmask that says which stacked shaders it has turned on. Turning one on or off flips a bit, and since the variant was already compiled with every stacked shader inside it, nothing new ever has to compile.
You'd think all those branches would be expensive, but they really aren't. Every pixel of an object takes the same branch, so the GPU never has to split its work, and a branch that's off costs close to nothing. The one thing to watch out for is size, since every variant carries every stacked shader whether it's on or not, so I keep them small. Objects with nothing stacked on them keep drawing with the plain shader, exactly like before, so they don't pay anything.
The bitmask lives in the block of per-object data the renderer already uploads to the GPU, in a slot that turned out to be padding nothing in the engine used, so it costs no extra memory at all. It's 32 bits wide, so a project can have up to 32 stacked shaders.
How one shader gets inside another
The part that replaced all of the converter's text surgery is that the stacking now happens inside Godot's shader compiler. When a variant gets built, the material's shader compiles exactly the way it always has, and each stacked shader gets compiled on its own next to it. Everything a stacked shader declares gets renamed with a prefix that's unique to it, so its names can never collide with the material's or with another stacked shader's. That alone gets rid of the black arm bug for good. Then each stacked shader's code gets inserted at fixed hook points in the engine's shader template, inside its bit check.
There are three hooks, one for each function a Godot spatial shader can have:
- The vertex hook runs before the material's vertex function. A stacked shader can replace the projection matrix there, and the material's own vertex code still runs normally on top of it. That's how the viewmodel lens works.
- The fragment hook runs after the material's fragment function, so a stacked shader gets to see the material's final albedo, emission, roughness and so on, and can add to them.
- The light hook runs once per light, right after the material's own lighting for that light. The stacked shader sees just that one light's diffuse and specular contribution, and it can add to it, scale it, or replace it.
The stacked shaders run in the order they're listed in the project settings, and that's the stacking: each one sees whatever the material and every shader before it in the stack produced.
Since one variant serves every object that draws with that material, a stacked shader can't do anything that would change how the material gets drawn for an object that has it turned off. So the engine refuses to load a stacked shader that uses regular uniforms (it uses per-object or global ones instead), render modes, transparency, depth writes, or the screen texture. The one exception is turning shadows off, which I made into a runtime switch that every shadow lookup in a variant checks. A few of those are the same things the converter used to trip over, and it's a lot nicer to get an error that explains the problem than an arm that's drawn black.
Not paying for it in the shadow pass
One detail made a big difference for performance. Godot is clever about shadows: most opaque materials don't do anything special in their vertex shader, so they all share one cheap default material for their shadows. If the variant were used for every pass, then compiling the viewmodel lens into, say, a body material would cost every character wearing that material its cheap shadow path, even with the lens off. So an object only uses the variant in its depth and shadow passes when one of the stacked shaders it has on actually changes where its vertices end up. A character that only has the rim on casts its shadows exactly the way it did before the rim existed.
What's in TeamSpec's stack
Here's what TeamSpec stacks on the viewmodel and the characters, in the order it runs. There's also a lens that draws your legs when you look down in first person, and two shaders for the HUD portrait, which I'll get to after.
The first is the viewmodel lens. It rebuilds the projection matrix from your viewmodel FOV setting, and it squeezes the viewmodel's depth toward the camera so it always draws in front of the world. It leaves shadow passes alone, because a shadow map is drawn through the light's projection, not yours.
The second is the viewmodel light. The viewmodel is the one object you look at the entire time you're playing, and it swings through every lighting condition in the scene whenever you turn around. On one of my maps, the same forearm measured about 66 out of 255 with the sun in front of the player and 235 with it behind them. So the viewmodel gets its own lighting: a softer, wrapped version of each light that moves some light from the lit side over to the dark side without adding any, and no specular highlight sliding across the gun every time you turn.
The third is the rim, the TF2-style rim light from Valve's 2007 paper "Illustrative Rendering in Team Fortress 2". It has two parts. The ambient rim makes the top edges of a character glow, even in shade, and it's the part that really sells the look. The per-light rim adds a bright edge when a light is behind the character. The per-light half was easy with the light hook; it folds into the material's own highlight by keeping whichever of the two is brighter, the way Valve's shader does, so it never stacks a second highlight on top of the first. The ambient half needed the one thing a Godot shader couldn't do, which is see the ambient light, so I added a new output to every spatial shader: a color that gets lit only by the ambient light at that pixel and gets added to the final result. The rim writes its edge mask there, and the engine does the lighting.
That turned out to be only half of the problem, and I didn't notice the other half until I changed my skin color to black on one of the darker maps and the outline mostly disappeared. Godot only ever reads the ambient light for the direction the surface is facing, so the top edges of a character were lit by the sky above them, or by the ceiling indoors. That isn't where a TF2 rim comes from. Valve's shader reads its ambient cube along the line from the surface to your eye, so the edge gets whatever light is between you and the character, which is why an enemy standing in a dark doorway still has a bright outline when you're the one standing in the light. A shader can't choose that direction by itself, because the engine works out the ambient light after the shader has already finished, so the new output now comes with a second value: the direction to read the ambient in. The rim points it at the camera, and the engine does a second ambient lookup in that direction, through the sky, the lightmap probes and the reflection probes, the same way it does the first one. The rim also keeps a small floor under that ambient, since about one lightmap probe in six on that map is completely black, and a dark character standing in one of those spots had no outline at all. Because the rim runs inside the material instead of on top of it, it respects everything the material does. Ambient occlusion darkens it, fog fades it, and when a character is on fire, the fire covers the rim instead of the rim glowing on top of the fire. It also means one less draw for every character mesh, every frame.
The fourth is the same rim at a quarter of the strength, for first person. The camera looks straight down the length of the gun, so its whole top is nearly edge-on to your view, and a full-strength rim coated every gun in a pale sheen that hid its textures and normal maps.
The fifth is the sun. The viewmodel can't use real shadows: its geometry still sits about half a metre in front of your eyes, so when you walk up to a wall, your arms are physically inside of it, and the shadow map puts a dark smear on them. So this one turns shadows off for the viewmodel (the runtime switch from earlier) and instead scales the sun by how visible it is from your eye, which the game checks with a ray once per tick.
This is also where the order of the stack matters. A stacked shader can't change how much of a light reaches a point, because Godot treats that value as read-only, so there's no way for the sun to tell the rim "the sun is blocked right now." It doesn't need to. Because the sun comes after the rim in the stack, it scales everything the sun contributed, the rim's share included. The rim doesn't know anything about the viewmodel, and it doesn't have to.
Drawing the same body twice
The newest use of all this is the little portrait of your character in the bottom-left corner of the HUD. The usual way to do something like that is a second camera rendering into a texture, but that renders the whole scene again. Instead, your own third-person body, which the game keeps posing even while you're in first person, has two more shaders stacked on it. One is another lens, which maps the body into the corner of the screen. The other replaces the map's lighting with a little studio setup, so the portrait looks the same in a sunny courtyard as it does in a dark basement. Since the lens leaves shadow passes alone, the same mesh still casts its real shadow on the ground. There's no second camera and no copy of the character to keep dressed.
In third person your body already has to stand in the world, so one draw can't do both. For that I added a stacked copy to the engine: the body gets drawn a second time with those shaders on top, using the same pose and materials, and the copy stays out of the shadow maps so it doesn't cast a second shadow.
Watch out for batching
The first time I played with all of this in a match, my first-person shotgun kept flickering into a black silhouette, but only once bots were carrying shotguns too. When Godot builds its list of things to draw, objects with identical sort keys get merged into a single instanced draw call, and a merged draw uses the first object's shaders for all of them. My shotgun needed the lens in its depth pass and a bot's didn't, but they had the same key, so depending on which came first, mine got its depth drawn without the lens. The fix was a new bit in the sort key, so two objects that need different shaders can never get merged. If you're ever doing anything per object inside a renderer that batches draws, that's the lesson I'd pass on: anything you decide per object has to be part of whatever the batcher compares, or the batcher will end up making that decision for you.
Why everything looks better
Most of the visual improvement came from the system getting out of the materials' way. Since stacked shaders run inside the real material instead of a converted copy of it, every material keeps everything it had: the glowing parts of guns finally glow in first person, and nothing gets drawn double-sided unless its material asks for it. Add the rim respecting ambient occlusion and fog, and the difference is pretty big.
Performance got better too. Every character mesh lost a whole extra draw per frame, the first-person and third-person versions of a gun now share their color pipelines, changing your viewmodel FOV is one global value instead of a walk through every converted material, and the converter and its folder of pre-baked shaders are gone completely. A scene that doesn't stack anything renders exactly the same, pixel for pixel, as it did before the change.
Limits
Not everything should be a stacked shader. Anything with its own geometry or its own render settings still gets its own pass, like the afterburn (a transparent shell pushed out from the skin) and the enemy outline (drawn with a stencil test). The stacked shaders still run inside those passes, though, so the fire on your arms in first person goes through the viewmodel lens like everything else. Stacked shaders also only work in Forward+, Godot's desktop renderer. And a variant still has to compile the first time it's drawn, so the game's warmup system records which stacked shaders each mesh has on and compiles the right variants while the map loads.
Conclusion
This is one of the engine changes I'd most like to see in upstream Godot. Per-object shader effects come up all the time (viewmodels, rim lights, hit flashes, team colors, dissolve effects), and right now the only answers are to edit every material by hand, draw the object again in another pass, or render a second camera. The idea itself isn't that complicated: compile one variant per material with every effect behind a bit, keep the bits in per-object data, merge the code in the compiler instead of on the source text, and order the effects instead of making them talk to each other. If you're building something similar, the two things I'd watch out for are draw batching (anything decided per object has to be part of the sort key) and the read-only values in the light function, which you can usually work around by picking a good order.
I thank you for taking the time to read this, and I hope it saves somebody out there from writing their own shader converter!