Chapter 9

Postscreen Processing and Image Effects

Detecting scene edges with the Sobel operator

Edge detection is an image-processing technique used to detect boundaries in an image. It is widely used in the field of computer vision, data visualization, and surface topology. For example, the pencil sketch effect of an image is nothing, but an application of edge detection algorithm. This recipe will demonstrate the edge detection technique using the Sobel operator or filter. 

A Sobel filter measures the change in the gradient of an image in which it recognizes the regions of an image where the frequency of the color transition is higher. These higher transition regions shows sharp changes in the gradient that eventually correspond to the edges. The Sobel operator uses convolution kernels to detect the edge portions in the image. A convolution kernel is a matrix that contains predefined weights that formulate the calculation of the current pixel based on the neighboring pixels intensity and weights contained in the convolution matrix itself. 

Making the scene blur with the Gaussian blur equation

The blur effect is an image processing technique that softens an image or makes it hazy. As a result, the image appears smoother like viewing it through a translucent mirror. It reduces the overall sharpness of the image by decreasing the image noise. It's used in many applications, such as blooming effect, depth-of-field, fuzzy glass, and heat haze effect.

The blurring effect is implemented using the Gaussian blur equation. The Gaussian blur equation makes use of the convolution filter to process image pixels. Bigger the size of the convolution filter, better and dense is the blur effect. Each pixel's color is mixed with the neighboring pixel's color. This mixing is performed on the basis of a weight system. Closer pixels are given more weight as compared to farther ones.

Making a scene real-time glow with the bloom effect

Blooming is a very useful post screen processing technique that makes a real-time scene glow. With this effect, certain parts of the scene appear highly brighter and give an illusion of emitting scattered light in the atmosphere. This technique is widely used in gaming and cinematic effects.

The working principle of the bloom effect is very simple. First, the scene is rendered to an offline framebuffer or texture, where its texture is used as an input in the next stage that detects the bright portions in the scene and writes in a new texture. This texture is then passed on to the horizontal and vertical blur, which applies the Gaussian blurring effect to make it blurred and scattered a bit.

Painting the scene like a cartoon shading

Among various different kinds of shaders, the toon shader is well known for producing cartoon-shaded scenes. The cartoon shading technique is implemented in the fragment shader. The fundamental basis of this shader is the quantization of colors.

In this, a range of colors are represented by a single type of color. Mathematically, color values are constrained from a continuous set of values (in floating numbers) to a relatively small discrete color set (represented by integer values). In addition to the quantization of color, the edges of the geometry are also highlighted using the Sobel operator.

Generating an embossed scene

Embossing is a technique in which the scene appears raised or highlighted with some 3D depth. The working logic of the emboss shader is similar to the edge detection technique. Here, the detected edges are used to highlight the image based on edge angles.

In this recipe, edges are detected by taking the difference between two consecutive texels in any arbitrary direction. The difference of these two results in a new color intensity, where each component (RGB) are compared among themselves to find the greater magnitude component (max). This component is then used to clamp between low (0.0) and high (1.0). This operation results in three color intensities: white (derived from low), black (derived from high) 1.0, and emboss (derived from the max component).

Implementing grayscale and CMYK conversions

The grayscale or luminance is an important topic that digital image processing is incomplete without discussing its practical implementation. Luminance is widely used in various applications of image processing. Edge detection, cartoon shading, and emboss effect are examples that we implemented in this chapter, which make use of luminance. In this recipe, you will learn how to covert an RGB color space to luminance and CMYK.

Numerically, a grayscale is a linear interpolation between black and white, depending on the color depth. A depth of 8 bits represent 256 varying shades from white to black. However, with four, only 16 shades can be represented. The black color is the darkest possible shade, which is the total absence of transmitted or reflected light. The lightest possible shade is white, which is the total transmission or reflection of light at all visible. Intermediate shades of gray are represented by equal levels of three primary colors (red, green, and blue) to transmit light or equal amounts of three primary pigments (cyan, magenta and yellow) for reflected light.

Implementing fish eye effect

A fisheye lens effect create a strong visual distortion to produce wide panoramic or hemispherical image. Fisheye lenses effect is an other prodedural shader implemented in the GL SL 3.0. As a result, edges in the scene look curved and bowed around the center of this virtual sphere. This effect makes the scene looks like wrapped around a curved surface.

The barrel distortion is used to achieve fish eye effect where it's applied on the fragments. This recipe implements the fragment shader which makes the texture coordinates distorted resulting in a magnifying lens effect or a fish eye lens effect.

Geometry distortion with barrel distortion

This sample shows the geometry distortion of the given mesh using vertex shader. This is a variant type of previous recipe and it is not a post processing technique. Here, the barrel distortion is implemented using the vertex shader. 

The difference between the previous and this recipe is- in the former, the physical geometry was not distorted it's only the appearance that was changed using fragment manipulation, here the vertex position is changed as result the geometry appears distorded.

Implementing the binocular view with procedural texturing

This recipe implements a binocular view effect, where a scene is rendered as if it's visualized from the binocular itself. We will implement this effect by programing a procedural shader. Alternatively, in another technique, the alpha-mapped texture is used instead. In this approach, an alpha-masked texture containing a binocular view image is superimposed on top of the scene. This way, only those parts of the scene are visible that belong to the non-masked texture region.

The procedural textured approach is also relatively simpler. Here, the scene is programmed in the fragment shader where the binocular view effect is created using texture coordinates of the vertices. Texture coordinates are used to create a logical circular region on the rendered image. The fragment that belongs outside the circumference of this circular region are rendered with an opaque color (say black). This opacity reduces as the distance shrinks toward the center point of this circular region.

Twirling the image

Twirling is a very common effect used in animations. When applied to a rendered scene or image, it distorts the appearance within the circular region and produces a radial circular motion of the texels where these are moved around the center of the circular region, producing a whirlpool-like effect.

Programmatically, for a given image, an arbitrary texel is chosen as a center. A fixed distance from the center of the circle defines a locus of the circumference. All the texels falling under this circumference are being applied to the rotation. The rotation of the texels within the circle decreases with the distance from the center and diminishes at the circumference edge.

Sphere illusion with textured quadrilateral

This recipe will demonstrate a performance efficient technique, which makes use of the procedural texture to produce the illusion of a real 3D object. In the Gouraud shading, fragments are painted with light shadings based on the direction of the light source and the geometry shape. For instance, in Chapter 5, Light and Materials, we implemented the diffuse light on a spherical model, which contains a very high number of vertices. This recipe technique renders the same diffused sphere, but using only four vertices. It fakes the light shading in such a way that the difference between the two becomes indistinguishable.

The performance is directly proportional to the number of fragments it renders to the screen. For example, the surface area covered by a single fullscreen rendering sphere is equivalent to several tiny spheres covering up the same surface area on the screen.