Rich Toon Shader R&D

Light Shade

The shading is not coming from a flattened toon shader or a flattened specular or stepped lighting.

It is coming from a partial blending which create a soft boundary between primary lighted area and self shadow color (for shadowed and self-occluded) and adding color of smooth indirect light.

You can take a look at the below picture to help what I meant by that.

Shadow Color

It can be retrieve in 2 ways for different cases.

Case A - inherited from its primary color (automatic)

We want the color to be inherited from its primary color. We can encode the primary color of the character into HSV (Hue, Saturation, Value) and shift the V (Value) of it for self shadowed areas in the shader.

hsv rgb2hsv(rgb in)

{

    hsv         out;

    double      min, max, delta;

    min = in.r < in.g ? in.r : in.g;

    min = min  < in.b ? min  : in.b;

    max = in.r > in.g ? in.r : in.g;

    max = max  > in.b ? max  : in.b;

    out.v = max;                                // v

    delta = max - min;

    if (delta < 0.00001)

    {

        out.s = 0;

        out.h = 0; // undefined, maybe nan?

        return out;

    }

    if( max > 0.0 ) { // NOTE: if Max is == 0, this divide would cause a crash

        out.s = (delta / max);                  // s

    } else {

        // if max is 0, then r = g = b = 0              

        // s = 0, h is undefined

        out.s = 0.0;

        out.h = NAN;                            // its now undefined

        return out;

    }

    if( in.r >= max )                           // > is bogus, just keeps compilor happy

        out.h = ( in.g - in.b ) / delta;        // between yellow & magenta

    else

    if( in.g >= max )

        out.h = 2.0 + ( in.b - in.r ) / delta;  // between cyan & yellow

    else

        out.h = 4.0 + ( in.r - in.g ) / delta;  // between magenta & cyan

    out.h *= 60.0;                              // degrees

    if( out.h < 0.0 )

        out.h += 360.0;

    return out;

}

After V (Value) of HSV is shifted we can multiply the scene indirect light color to its shadow color using NdotL

For Efficiency / Optimization

Single Render Call / Color baked into Vertex Color : If this was a single render call, I’d consider three textures with the information encoded into each of the specular color (for bright spots), the self shadow color ( for shadowed and self-occluded) and the primary color for all other cases. Other case for flat shaded, you could bake it into vertex colors.

LWRP : Unity has LWRP (light weight render pipeline) this year ( still beta but if the project is longer than 6 month from now to release then its good ) which can control these render passes more deeper level based on what the look is needed.

Optimize lighting : If indirect light is something that makes the the scene heavier when there are lots of characters (at least more than 20 for example) then single pass rendering might have best chance to try which is baking the each color into vertex color. Of course, LOD would do lots of job based on level design on the environment too.

Imposter for hair or faraway NPC : For static hair efficiency, Imposter could be one of the option based on how much hair will move based on characters. (maybe NPC can have these only for optimization)

LOD for shader types : Based on distance, not only modeling can use LOD but toon shader can be divided into 3 different shader types. Most far-away characters to use cheap toon flat shader. Mid far-away characters to use toon stepping-lighting shader. Most close-up characters to have toon partial blending with indirect light color for best look.