Ok, so here’s a real quicky… you want nice thick edges around your characters? You want soft edges? You want edges that have texture and line variation? You want edges that have a gradient on them? Here’s how you too can do this. The limits of this technique are that if your polys aren’t connected to other polys (as around the edge of the models neck in the examples) it wont work there, but it will work around shapes with hard edges (better than fresnel) in fact softer edges do give it more of a hard time, so in combination with fresnel you should get some pretty nice effects. This method will never replace a dedicated toon render engine as there are flaws, but for the meanwhile it can get some excellent results.
This is based on an old technique and workaround that those of you who’ve used Lightwave might remember. I’ve also seen tutorials on doing this in Max too, so I’m just showing all you C4D users the light.
To begin with you have to duplicate the object you wish to outline (so if your object is ultra complex this might not be the way to go for you).
What we’re going to do is use Decal Shading (which is where one side of a polygon has a different texture to the other side. The easiest way to think of this is like a playing card. On one side it has an image of the card value, maybe that’s the Queen of Hearts. On the other side it usually has a decorative pattern, mostly there to make sure that you can’t see the value of the card through it if it’s a cheap set. Ok, got that?). This will allow us to basically show the backside of the texture, but not the front. What we’re going to do is scale up our object based on it’s normal’s, and then texture it so that you see the larger object’s back faces. Here’s how:
The Basic Shader
|Make a new texture, call it outline, or edge, or whatever you feel comfortable with. First off switch off everything apart from Transparency and Displacement.
Go to Transparency and put in a BhodiNUT Buffy channel. This will cause only the back faces of the polygons to be visible.
Go to the Displacement channel and put in a BhodiNUT Gradient, take out the middle colour tag on the gradient, and change both colours to white (so you’ve just got a solid white colour). We’re using a displacement to make this model larger than the original (this is how we create the outline). Change the Maximum Height value to something that suits your models size (this is the width of the outline). Then make sure the Strength is set to 100% (this just makes sure that the outline will be set to the Maximum height value).
Next to make the texture of the original object flat shaded (like a cartoon) go to the Render Settings dialog box and in the Effects section add a Cel Renderer Post Effect, then in the Cel Renderer options make sure that Color is checked, and then check Quantize, and bring the Steps down to 2. Keep Edges unchecked but Outline and Illumination checked.
A More Advanced Shader
|This is just taking the Basic Shader a step further, and adding our own custom cel shader to the original object to give us a little bit more control.
Start out with the Edge shader from the last tutorial. First off we might want the colour to change within out lines, but we don’t want it to be affected by illumination (although you can get a great embossed effect if you use the Color channel with no shadows using a Render/Compositing Tag). So what we do is switch off the Color channel of the material, and switch on the illumination channel.
In the Illumination channel we can put in whatever texture we want to appear through the lines, because Luminance isn’t affected by lighting it will appear nice and flat (just how we want), we could just colour it or if you wanted you could put in noise to make the line grainy, or a fresnel gradient (although it would have to be quite small and close to the 90 degree angle for it to be visible) to make the line tube like. What we’re going to do is simply put in a Gradient so that the line at the bottom of the object is Black, but at the top is Red.
Now because we’re taking this a stage further we’re going to play around with a tint across our object too. So to start off add a BhodiNUT Fusion, Edit this and enable the Alpha, put a BhodiNUT Buffy in the alpha channel, and add a BhodiNUT Gradient to the bottom channel (this is the back of the polys), and edit it. In the gradient options change the settings to match the following:
Next put in another BhodiNUT Gradient into the top channel of the Fusion. Edit the gradient so that it’s just got two black swatches in it (this gives us a plain black colour which is all we want, the tint will be created by changing the transparency of this black… we could have put anything into this channel, this is the tint layer for us.
Now click OK to exit the Fusion, and from the drop down next to the text “BhodiNUT Fusion” in the Texture block, select “Copy Channel”.
Go to the Transparency channel and from the same drop down on that page select “Paste Channel” to paste our fusion into the transparency channel. What we will want to do here is make the transparency gradated while keeping the solid back faces solid. So edit the BhodiNUT Fusion and start by editing the top channels gradient. With transparency remember that white is transparent while black is opaque. So change the gradient type to a 2D-V and change the two colours in it so that the leftmost one is a medium to light gray, and the right most colour is white. Press OK to go back to the Fusion and now change the bottom gradient so that it’s all black (we want the back faces solid remember, although you could again put anything here… if you wanted the line to fade out as it left the object then a fresnel might do the trick, or a good noise or texture might make the line look hatched, experiment to see what you can do). The alpha should just remain a BhodiNUT Buffy. Press OK to finish editing the Fusion.
The line is still very even so now lets change that to make a more organic line. Go to the Displacement channel in the material and add a BhodiNUT Noise to it. Edit the BhodiNUT Noise and change the settings to make the noise nice and large, to give a good uneven line shape across the surface (without being too frenetic, or causing sharp edges to appear where there shouldn’t be), you can even animate the noise if you want that hand-drawn line flickering, changing shape effect.
Now the Edge Shader is complete, time to move on and start editing the original objects shader to make a nice basic cel shader.
|Create a new Material for the original objects surface. in this material we only want the Luminance channel switched on (although it will still be convincing if you use Bump, Displacement, and Alpha, I’d suggest leaving it at that if at all possible).|
|Now the simplest way possible to make a cel shader using Smells Like Almonds is colorize a BhodiNUT Lumas shader (which gives you the luminance of the surface) and change the gradients interpolation in the BhodiNUT Colorizer to “None”, that way the colours of the gradient will be nice and stepped, and will give you the cel shades that you want. However I want to give my cell shader more of a graffiti spray paint style, so I’ll guide you through how to do this.|
|Start by adding a BhodiNUT Fusion to the Luminance channel. Edit it, and in the top channel put in another BhodiNUT Fusion, this we will use to get our spray can texture using overlay, in the bottom channel put in a BhodiNUT Colorizer, this is going to give a gentle cel shade and colour to the surface.|
|Edit the BhodiNUT Colorizer and add a BhodiNUT Lumas channel to it. Change the ramp to give you three clearly demarcated ramps, going left to right, dark to light. These would normally be our cell colours, but I want a slight softness to their edges.|
|Next edit the BhodiNUT Lumas channel, and switch off all the specular layers, then go to the Diffuse layer of the Lumas, and change Illumination to 100% and the Color to white. Click OK and then copy the BhodiNUT Lumas channel (as we’re going to use this elsewhere). Then click OK to get out of the BhodiNUT Colorizer and go back to the Fusion.|
|Edit the BhodiNUT Fusion in the top channel and paste the BhodiNUT Lumas that we copied into the top channel of the fusion.|
|In the bottom channel make a new BhodiNUT 3D Noise channel, edit it, and change the scale of the noise right down so that it’s pretty small.|
|Click OK to get back to the second BhodiNUT Fusion channel, and change the Mode (blending mode) to “Levr”. What this does is it takes the top channels luminance and depending on the luminance of the bottom channel outputs either white or black, so it’s like using a hard light blend and then upping the contrast al the way, although it does work in reverse!|
Some samples of different tiles patterns and noise patterns using this technique
Take down the Opacity slider a little bit to around 85%, as we don’t want completely harsh edges and click Invert Output. as Levr on it’s own works in reverse! Click OK to go back to the original Fusion channel, and change the Mode to “Overlay”, if you want you can lower the Opacity a bit to get more of your Colorizer cell shading showing through. Click OK again to go back to the Luminance channel of the material. Make the Mix “Multiply” and change the colour of the channel to match your objects colour,
Now change the Cel Renderer settings in the Render Settings dialog box, so that Quantize is switched off and you’re ready to render.
To carry on…
Further experiments with this can give nice planetary atmospheric halos, and other effects. You can also blend in a BhodiNUT Fresnel in the luminance channel of your cel shader and ramp it up with a black edge to help with the edge shading.
Another thing you can do to give you real time feedback of the edge shading is to either use a Smooth shift operator on your object to scale it up as much as you want (for the edge) or to use DiTools to give you real time feedback for the displacement map in the Edge Shader (remember to turn off the actual edge shader’s displacement otherwise it will be twice as large). Then rather than using Buffy, you can make a front face shader (usually totally transparent) and a back face shader (whatever colour you want your lines to be) and just apply these directly to your object, and in the settings for how the materials are applied make sure that they’re set to “Front” and “Back” Side respectively. That way you should get real time toon shading directly in C4D’s view ports!