Computer Graphics Notes

Computer graphics falls broadly into two fields. Image processing uses a computer to manipulate a digitised image. The image starts out in the "real world", and the processing is concerned with enhancing the image (eg, to detect faint objects) or transforming the picture in some way.

In contrast, image rendering works from a computer model of a scene, and produces a picture without the "real world" being involved at all.

Before we look at these two fields in more detail, lets look at what we mean by a computer image.

Pixels and Pictures

If you look closely at a black-and-white picture in a newspaper, you will see that it is made up of a lot of black dots. The same is true of computer pictures. If you enlarge the scale of a picture in ClarisWorks, you get to see the individual dots. A dot is called a pixel (a contraction of "picture element"). Eg,

Each dot can be arranged on a 2-d grid:

Using the grid, each dot has a particular row and column coordinate. In Pascal, we could represent a small 40 x 40 picture as the array:

  bw_pict: array[ 1..40, 1..40 ] of boolean;

A boolean is a value that is either true or false. Since each dot in the picture is either black or white, we can use 'true' to mean 'Black' and 'false' to mean 'White' (or vice-versa; all that matters is that we are consistent). This is sometimes called a bit of information.

If we wanted a colour picture, we would need to more information in each dot. How much more information? Well, two booleans would give us 4 colours (we could code the colours as true-true, true-false, false-true and false-false). If we used three booleans, we could have 8 colours, etc. Eight booleans is a popular number (so popular is is universally known as a byte). This allows 256 colours. Another popular choice is 16 bits (=2 bytes), giving a choice of 65,536 colours. And if that's not enough, you can go for 3 bytes (=24 bits) with over 16 million colours. We could represent these colourful arrays in Pascal in a number of ways; here are some examples:

  col16:    array[ 1..40, 1..40 ] of (0..3);
  col256a:  array[ 1..40, 1..40 ] of (0..255);
  col256b:  array[ 1..40, 1..40 ] of char;    {-- equivalent to col256a }
  col64k:   array[ 1..40, 1..40 ] of integer;
  millions: array[ 1..40, 1..40 ] of (0..16777215);

Now, pause for a minute and ask yourself: "But what are the colours"? This is a good question to ask. It shows you are thinking. You see, there are no colours inside the computer. Colours only come into it when we display the picture on a monitor, or print it on a colour printer. These output devices have their own interpretation of what values correspond to what colour. Thus, a number '205' might correspond to the colour 'red', '409' for 'blue', etc.

Image Processing

Start with an image (from digital camera/scanned photograph)
Load image into program as 2-dimensional array (see the example Pascal program on the 07.111 server).
Manipulate the array to: remove noise, identify parts, change colours, etc.

Note that image processing data always represents a flat (2-dimensional) world. Also, the resolution (amount of detail present) is fixed.

Image Rendering

(This is what you will see on the video)

Start with 3-dimensional computer representation of a scene. This representation will contain data describing, say, "a sphere of radius 10, centered at 120,200", or "a cylinder, length 50, radius 3, ...".
The scene data describes sizes and coordinates independent of screen/printer resolution
Scenes are built from a small number of regular geometric shapes, arranged together to form larger objects (eg, a table may have 4 cylinder legs, and a rectangular prism top)
The shapes have surfaces, which can be given different reflective properties (colour, shine, texture, etc.)
The scene is completed by defining light sources and ambient light levels, and by locating an imaginary "camera"
A "photograph" of the scene can then be generated by a technique such as ray tracing

Ray Tracing

[ In the video, the animated detective is named Ray Tracy. This is an allusion to the following rendering technique. ]

Ray tracing is a technique used to produce high-quality pictures from a computer model of a scene. The basic idea is an extension of ray casting. Ray casting is illustrated in the following diagram:

The idea is to cast a ray from the "camera" through the view plane (ie, following in reverse a ray of light from the final picture to the eye) and testing the scene objects to see what object the ray strikes. A simple test is then made to check that the object is not in a shadow at that point.

The ray casting is repeated for each ray that passes from the camera through a pixel in the viewing screen. For a 400 x 400 picture, this will require 160,000 individual rays.

Ray casting is a very simple technique, but it cannot cope with reflections from other shiny objects. The only parts of an object deemed to be lit are those directly illuminated.

Ray tracing produces much more realistic images by allowing secondary rays to be spawned when a ray intersects a reflective or transparent object.

Since each ray can spawn one or more new rays, ray tracing involves much more work than ray casting. It is not unusual for ray tracing to require several million rays.

Issues

Modelling complex object shapes (eg, face); build from simpler parts (cf. Lego)
"Soft" objects (clouds) -- the parts of a cloud are still cloud, not simpler objects!
Hidden surfaces; shading
Surfaces that scatter light, rather than simply reflecting it
In the video, look for:
- mirror surfaces
- cartoons constructed from simple shapes
- sharp edges; distinct shadows
- repetitions of the same shape
- other tricks to reduce the work in making a scene