Shader that renders Border Outlines

Merry Almost-Christmas! I normally do a blog post every month on the 25th, but I wanted to do this one a few days before then because of the holidays.

Last month I had described my plan to render border lines on the ground (think territory in a strategy game, or movement ranges for tactics games). Here’s an image showing the result of my experimentation:

8 inner-fade

Hey, looks pretty good if I do say so myself! That’s a smoothly glowing outline drawn on the ground, surrounding a discrete region of a square grid, with the outline nicely rounded at the corners.

Alright, so now I’ll run down my process of experimentation. I started by mocking up a black and white image of a region on a square grid. I blurred the image similar to an SDF texture (indeed, it’s possible what I did literally is signed distance fields, but frankly I’m not an expert in the technical semantics) to assign to my shader:


I then used the image (along with a main grass texture) in a shader that responds to the shades of grey in the image, drawing a yellow line around the white region:

2 shader-blur

That looks pretty good, right? But hold on a minute, there’s a problem that’s apparent when we zoom in:

3 close-blur

Ugh, there’s some pretty ugly zig-zagging going on where the line curves. I guess the greyscale texture doesn’t have enough bit depth to depict a smooth curve.

Next I decided to try an image without blurring. Although there won’t be any blurring done in the image itself, the texels will be interpolated when sampling the texture, so perhaps that interpolation will be more detailed than blurring in the image itself.

Here’s what it looked like at first:

1 shader-tiny

The corners definitely look a lot better, without that ugly wobbling. However the outline in general is a lot more square than I want. So I tried reducing the resolution of the greyscale image, to cause the texture sampling to interpolate more. I mean, really tiny:


For reference, here’s what it looks like with that image as the main texture:

5 albedo-pixels

That’s pretty blurry, so I’m hoping that’ll work well with the border rendering:

4 shader-pixels

Nice. That’s pretty much exactly how I want the border lines to look. Here’s an image of the border line on the blurry texture instead of grass to help explain what is going on:

6 explain-pixels

Next I knocked up a quick script to try generating this texture on the fly, instead of only using premade images:

Double nice. The script generated a blank texture at startup, and then drew on the texture in response to mouse clicks. As you can see in the video, this performed nice and snappy since I was drawing individual pixels (as opposed to large blurry shapes) on a small texture.

At this point the experiment was basically a success, and I considered stopping here. However I read on the forum that conditional branching in a shader is bad for performance, and the shader I wrote had an ‘if’ statement in it.

Thus I rewrote the shader using step() instead. I should profile them to see if there’s any performance difference, but after making that change I noticed a functional improvement that change allowed. While the line rendered using both ‘if’ and step() had hard-edges, I realized I could use smoothstep() in the same way as step() and get a smoothly fading edge on the line. Here is what it looked like when I replaced step() with smoothstep():

7 smoothstep

Cool! And a little tweaking of the step values resulted in the image from the beginning of this post, with a lot of fade on the inside and a relatively sharp edge on the outside. This effect is all set for future games on a tactical grid.

At this point there are a few more things I may experiment with, so perhaps I’ll have done that in my next post. First off, I’m considering rewriting the line rendering with the routines explained in this SDF tutorial. While the glowy outline looks quite smooth, the hard-edged outline had some aliasing in it. What I wrote was a pretty simplistic approach to rendering the line, and that tutorial seems to explain a more sophisticated approach that looks better.

The other improvement I’m considering is using vertex colors instead of a texture in order to support arbitrarily shaped grids. One major limitation from the tiny texture is that the bordered regions need to be a square grid, just like the pixels in the image. However it’s pretty common for strategy games to use a hex grid, making the technique of bordering pixels useless.

While thinking about that problem, I had a realization that I could generate a mesh of hexagons and then sample the vertex colors in the shader. The colors will interpolate between black and white just like values in the texture, but now with any shape from the mesh, not just the square grid of a texture.

This would be such a massive improvement, I’m likely to try this second experiment even if I don’t bother with the line rendering change. I already know of a great tutorial series on generating hex maps. We’ll see next month!

2 thoughts on “Shader that renders Border Outlines

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s