Improving shadow map utilization for cascaded shadow maps

Jonathan Blow is working on stable cascaded shadow maps, and trying to reduce wasted texels due to the way the shadow frustum is set up. Well, as it happens I have my own minor tweak to Valient’s technique that also reduces wastage, so I’ figured I’d describe it here with pretty pictures. I’m going to assume that you’re roughly familiar with the general idea, if not, read e.g. Blow’s first blog post on the topic, or ShaderX 6.

The root of the problem is that with realistic camera FOVs, you often get a situation like below.

The first slice of the view frustum (the trapezoidal shape in the middle) is inscribed in a sphere (for rotational invariance – don’t want those shadows flickering!), and then that sphere inscribed in a box (the shadow map’s orthogonal view frustum). Now, you could take the position that all that red area is wasted, since only the green area is inside the frustum and the first splitting plane. Blow performs a few very clever tricks to overlap the shadow map slices to avoid all those red parts taking up too much, for a total of 25% memory savings.

However, who says that only shadow texels within the first slice are useful? Anything to the right of the splitting plane is also within the view frustum, and can easily be used for shadowing. Why reject these high-res shadow texels and skip to the next slice just because we’re on the other side of the splitting plane?

So rather than trying so hard to not store these “wasted” shadow map texels, we should instead make sure we’re making use of those texels whenever we can so that they’ aren’t wasted anymore! As we sample the shadow map we just test the current point against the first shadow _frustum – _not the bounding sphere, and not the splitting plane, the frustum. That changes the picture to something like this:

This is all pretty standard so far. My own little tweak comes from the realization that if we’re not going to bother using the splitting plane as our boundary for sampling then why bother with it at all? The implication is that when we compute our bounding sphere we shouldn’t worry about how it fits with some arbitrarily positioned plane, we should only worry about how it fits with the frustum itself (and previous slices). In other words, we position our bounding sphere (of the same size) to minimize waste by just shifting it to the right. We have to stop before we pass the near plane of the view frustum, of course. Like so:

As you see we’ve reduced texel waste for the first slice by quite a lot here. We’re using approximately 5/6 of that texture space, instead of 1/6 like the first scenario. However, notice what happened at the corners where the splitting plane intersects the frustum, they’re outside the sphere! That means that we’ve effectively moved the splitting plane back compared to the previous image. If you were in the business of worrying about the location of the splitting plane this would be a problem  – luckily we aren’t, so it’s not. Freeing ourselves from the constraint of the splitting plane means we can find a bounding sphere with a better overall utilization of the shadow map.

If the FOV increases, or the sphere gets smaller, the “corners” (determined by the original splitting plane) will eventually drop outside not just the sphere, but the shadow map too, and for those pixels we do need to drop to the second slice when sampling, which of course means that we have to be careful that the second slice is far enough back that it doesn’t leave any gaps. This also means that there are minor areas near the edges of the screen where we may drop to a lower resolution shadow map sooner than we would with the original scheme, but in return for that we get a _big _area in the centre of the screen that can now use a higher res shadow map due to the better utilization.

Now, most of the first slice will actually overlap the second slice – we will still always use the first one when we can to maximize the amount of pixels that get to use highest shadow resolution, but there is some overlap where parts of the second slice will be “overruled” by the previous slice. In practice, however, this isn’t such a big deal because the second bounding sphere is usually several times larger than the first, so as a proportion of the overall area the overlap is actually quite minor.

For example, take a look at this illustration of the second slice:

We’ve zoomed out a bit. You can see the first slice as the rectangular red area to the left and centre (and note how the bounding sphere has been carefully positioned to just barely cover the intersection of the previous slice with the frustum, leaving no gaps).

So while there is still some waste for things that are outside the frustum, and there’s also wastage for texels that are redundant because they’re already covered by the previous slice, by and large we’re getting very good texel utilization.