Download PDF

Normal Map Workflow Page 3

Once you've rendered out your normal map successfully from max, which might take a few tries till you get your cage set up well, you will have to clean it up. I didn't remember to save one of the early renders of this normal map, but this one still has some errors in it.

See the collar of the shirt? How it wobbles back and forth? That's something that needs cleaning up. Also the headphone strap at the bottom right is a bit wobbly. This normal map started out with several holes in it also, where the cage wasn't quite large enough, but it wasn't worth rerendering it, so I cleaned them up in photoshop.

Rather than painting in RGB, which will not only create weird smoothing errors, and hurts your brain, break it down into channels. If you are used to keeping an imaginary light source in your head for traditional texturing, this should be fairly easy to work with. When you want to use the smudge brush per channel, be sure to tick the option box at the top that says "use all layers" or it will not work.

Here is an example of an area I cleaned up. See how easy it is to understand the changes I made when you approach it per channel? The result is much cleaner than had I tried to fix it in the RGB all at once. Fix all the chunky areas like these, before moving on to adding detail with the NVfilter

 

After you have cleaned up your Normal map by painting out the errors and holes in each channel, it's time to use the NVfilter to add smaller details. The general rule I use, is if anything is less than an 1/8th inch thick, it should be left to this stage to create. Small details like these don't really benefit from being modeled, and would take much too long to model in high poly, if it's even possible to model them at all. Cloth seams, surface texture, grit, pocks, dings, rust, skin pores, and scratches can all be made very easily using the NVfilter. Which can be had here.

When you prepare a layer to use the filter on, you make it like a traditional bump map. Meaning that the lighter a pixel is, the higher off the surface it will be in the normal map. The blacker it is, the more indented it will seem. Middle grey will appear to lie directly on the surface of the normal map.

I normally prepare my details in layers according to groups. I will do the threads on a shoe in one, and the grit for a tshirt on another. That way I can run the filter on a different scale per layer as is appropriate for the detail in question.

After running the filter, it's good to remove the neutral 128.128.255 purple by using the magic wand set to 0 tollerance and the contiguous box unticked. Click on the large expanse of purple to select it all, and hit the delete key to remove it.

After you have done the above steps, you are ready to combine the filtered layer, with the baked layer. The first step is to select the blue channel only by going into the channels tabs, and then hit ctrl+l to bring up the levels control. Then reduce the blue channels value from 255 to 127. Then put the entire layer on overlay mode. I have an action you can load that does this all in one step called "normal fix".

The blue channel from your baked layer, has depth information in it. It's already fairly bleached out, but it's accurate. The blue channel from your nvfiltered version contains almost no value in it. If you didn't half the blue channel from it, it would blow the blue channel out from your baked layer. By halfing the value, when you put it on overlay, it only modulates the blue channel, rather than blowing it out.

After putting in all the fine details using the Normal Map filter on hand painted layers, you can add even more detail by giving certain materials a different grit than others. The baked versions tend to all be very smooth, which reads well in game, but having some with a rougher "tooth" or a small pattern to diffuse the light can help add realism and varying material types. I like to use photoshops pattern tools for this effect. I have a small library of tileable images that I can use at any point in time. The best part about the pattern function, is that because it works on small tileable images, you can use it on any texture reguardless of size. Sometimes I do have to have two versions of a pattern, one for very small textures, and another for larger textures, so that the detail is still the appropriate size.

Below is an example of a small tileable black and white image. I use the pencil tool to make sure it's very crisp, so that it will read well in the final texture. Define it as a pattern first, then use the fill tool on a blank layer above your baked normal map layer.

When you are ready to apply a surface pattern to your materials, first create a mask for them, so it will be easy to constrain the pattern to only the area you want it. After you've filled only those areas, follow the steps used above for the shoe detail to combine the pattern and the baked layer.

Overlaying details in photoshop can greatly aid your normal map creation process. It's faster than modeling all of it into the high poly in many cases, and sometimes let's you add details you would have no way of adding otherwise. Add as many layers as you want, but be sure to make a flattened version at the end and run the normal map filter with the "normalize only" setting so that it's all mathematically correct for your engine.