The University of Auckland, Auckland, New Zealand
KLUWER ACADEMIC PUBLISHERS : Dordrecht e.a., 1999
Image analysis is one of the most challenging areas in today's computer science, and image technologies are used in a host of applications. This book concentrates on image textures and presents novel techniques for their simulation, retrieval, and segmentation using specific Gibbs random fields with multiple pairwise interaction between signals as probabilistic image models. These models and techniques were developed mainly during the previous five years (in relation to April 1999 when these words were written).
While scanning these pages you may notice that, in spite of long equations, the mathematical background is extremely simple. I have tried to avoid complex abstract constructions and give explicit physical (to be specific, ``image-based'') explanations to all the mathematical notions involved. Therefore it is hoped that the book can be easily read both by professionals and graduate students in computer science and electrical engineering who take an interest in image analysis and synthesis. Perhaps, mathematicians studying applications of random fields may find here some less traditional, and thus controversial, views and techniques.
If you do not like such dreadful things as probabilities, distributions, functions, derivatives, and so on, then you may go directly to the last three chapters containing a large body of experimental results. Experiments have been conducted with very different image textures so that you may readily see when the proposed models do adequately describe particular textures or are completely helpless. If you are still interested in the proposed modelling techniques, then there exist, at least, two options: to change your own attitude to this straightforward mathematics or to simply use its final ``easy--to--program'' results after skipping the boring derivations. The latter option seems to be a bit more realistic.
Probabilistic image models for image processing and analysis have been the subject of numerous papers, conferences, and workshops over many years, and today you can find many different books on image modelling. So it is desirable to underline in which respects this book differs from the existing literature.
To summarize briefly, it tailors models and modelling techniques to images, rather than the reverse. Most of the known probabilistic image models have been developed and successfully used in other scientific domains such as statistical physics, signal processing, or theory of measurements. An extension of a physical model to images calls for analogies between the basic notions that describe a physical object (a system of particles, a particle, interactions of particles, an energy of the interaction, a temperature, and so on) and the notions used to represent an image. Although distant parallels between them do exist, images differ much from every physical system of interacting particles, and the models borrowed from physics do not reflect salient features of image textures under consideration. Moreover, some physical notions, for example, a temperature or an energy, may even be misleading when used in the image modelling context.
Thus, unlike more traditional approaches, this book makes an attempt to show that modern image analysis is worthy of specific probabilistic image models and modelling techniques which can be, in some cases, both simpler and more efficient than their well-known counterparts borrowed from other scientific domains.
This book covers the main theoretical and practical aspects of image modelling by Gibbs random fields with multiple pairwise pixel interactions and demonstrates the ability of these models in texture simulation, retrieval, and segmentation. For ease of reading, it reproduces basic mathematical definitions and theorems concerning Markov and Gibbs random fields, exponential families of probability distributions, stochastic relaxation and stochastic approximation techniques and relates these definitions and theorems directly to texture models.
This book briefly overviews traditional Markov/Gibbs image models but concentrates most attention on novel Markov and non-Markov Gibbs models with the arbitrary, rather than pre-defined, structures and the arbitrary strengths of multiple pairwise pixel interactions. The most attractive feature of our models is that both an interaction structure of a texture and quantitative interaction strengths specified by Gibbs potentials can be learnt from a given training sample.
As shown in this book, it is an easy matter to obtain rather close analytic first approximations of Gibbs potentials and use them to recover the most characteristic interaction structure. The potentials are then refined by stochastic approximation, and we indicate ways of converting such a refinement into a controllable simulated annealing technique of generating images, similar, in a probabilistic sense, to a training sample.
This book discusses a wide range of experiments with various textures from the well-known album of Brodatz and the digital ``VisTex'' collection of the MIT Media Laboratory as well as with aerial and space images of the Earth's surface. Our experiments show that many natural grayscale textures can be efficiently simulated in the proposed way. Our models allow the reliable query-by-texture retrieval from large image data bases which is to some extent scale-- and orientation--invariant. Also, considerable attention is given to supervised texture segmentation. It is considered as a simulation of a desired region map using a conditional Gibbs model of region maps corresponding to a grayscale piecewise-homogeneous texture, and the same controllable simulated annealing technique can be used to segment piecewise-homogeneous textures.
Of course, a large number of theoretical and practical problems still remain to be solved, but the current results alone seem to justify our efforts in studying and exploiting these models and techniques.
We treat image textures as samples of specific Gibbs random fields with multiple pairwise interactions between gray levels or other signals in the pixels. In later chapters you will encounter both positive and negative results of simulating, retrieving, and segmenting natural image textures using these Gibbs models.
Our eyes solve image retrieval and segmentation problems so easily that you may be puzzled why even the most successful computational models rank far below, both in quality and processing rate. For me, human vision is a miracle of Nature, and more than three dozen years spent in computer vision and image processing domains have only strengthened this belief. It is difficult to imagine that our vision (and, generally, all plant and animal life) could be created in line with the conventional theory of evolution. Thus I do not expect that the foreseeable future will bridge wide gap between human and computer visual skills. But, it is still a very attractive challenge to fit our visual abilities to a Procrustean bed of mathematical models and algorithms and to emulate, somehow, one or another side of human vision with these fundamentally ``inhuman'' tools. Fortunately, there are many natural image textures that can be closely approximated by our models and modelling techniques. So these models and techniques are worthy of investigation.
You will find in this book novel types of Gibbs random field image models that differ from or generalize the known ones. In many cases the proposed models are more convenient for describing spatially homogeneous and piecewise-homogeneous textures than traditional models.
Image homogeneity is quantitatively defined in this book in terms of conditional probability distributions of spatial signal combinations. Mostly the distributions are assumed to be translation invariant, that is, in different parts of a homogeneous image that can be superimposed by translation, the sample relative frequencies of signal combinations are expected to be almost the same. The translation-invariant textures constitute only a tiny part of all possible image textures but the described techniques of texture simulation and retrieval can use more general transformations of the homogeneous parts including not only translation but also limited rotation and scale invariance of the corresponding distributions.
Gibbs image models are specified by a finite set of quantitative parameters describing the geometric structure and quantitative strengths of pixel interactions. The interaction structure is formed by characteristic subsets of interacting pixels which are included in a model. The interaction strengths are given by so-called Gibbs potentials, or potential functions, that control the probability distributions of all the possible gray level combinations in these subsets.
It should be particularly emphasized that an interaction between the pixels, or equivalently, between the gray levels or other signals in the pixels, has no physical meaning. It reflects only the fact that some spatial signal combinations in a particular texture are more frequent than others: the less uniform the probability distribution of signal combinations in the pixels that have a particular spatial arrangement, the stronger the interaction between these pixels.
As in the majority of more traditional image models, our models are restricted to no more than pairwise pixel interactions. However, unlike the traditional models, we presume that every model involves an arbitrary structure of multiple pairwise pixel interactions, and the structure may vary for different textures. Also, we presume that every model may have arbitrary interaction strengths.
Strange as it may seem, our models with multiple pairwise pixel interactions turn out to be both more general and computationally simpler than the well-known particular cases such as auto-binomial or Gauss-Markov models, which are widely used in statistical physics and theory of measurements. These traditional models have mostly pre-defined structures and potential functions, contrary to our models which place almost no restriction on the interaction structure and potential values.
As distinct from traditional models, our models permit us to learn, or estimate, from a given training sample not only the potentials but also the structure of multiple pairwise pixel interactions.
You will find that such a generalization of the Gibbs models simplifies notably the computational techniques for learning the model parameters. The learning techniques introduced in this book are based firstly on analytic and subsequently on stochastic approximations of the unconditional or conditional maximum likelihood estimates (MLE) of the potentials. After the learning stage, every spatially homogeneous or piecewise-homogeneous image texture is described quantitatively by a particular set of the most characteristic families of translation invariant pixel pairs that specifies the interaction structure and by particular potentials for each family that specify the interaction strengths.
You will find that the proposed models give a more penetrating insight into the physical meaning of the parameters to be learnt with respect to image textures. The model parameters have a natural interpretation in terms of spatial self-similarity between the image patches, namely, in terms of statistical repeatability of signal combinations. In particular, a grayscale image texture is described as a whole by a set of particular gray level co-occurrence or difference histograms collected over the characteristic families of pixel pairs. The histograms are sufficient statistics for our models, and the book shows how the potentials depend explicitly on them.
The signal histograms allow the formation of a model-based interaction map for recovering the interaction structure. The interaction map shows the contributions of various families of pixel pairs to the overall probability of a given training sample, and even a simple thresholding can recover the characteristic families.
Also, you will find in the book a new modelling scenario of simulating samples in the vicinity of a given training sample as regards their probabilities in the context of a particular Gibbs model. The proposed scenario, called Controllable Simulated Annealing (CSA), is simpler, but more efficient in texture simulation or segmentation than the traditional modelling scenarios. These traditional scenarios are based on a prior estimation of the model parameters and subsequent image generation by stochastic relaxation or on a conventional ``blind'' stochastic gradient search by simulated annealing.
You will find in Chapters 5 - 7 many experimental results of simulation (generation), retrieval, and segmentation of different image textures. These results outline pros and cons of the proposed Gibbs models. We restrict our consideration to only supervised texture simulation and segmentation when each texture is represented by an adequate training sample (e.g., in the case of a piecewise-homogeneous texture this sample contains both the grayscale textured image and the corresponding map of homogeneous regions in this image). Under the alternative modelling scenario, both the simulation of spatially homogeneous or piecewise-homogeneous textures and the segmentation of piecewise-homogeneous textures are achieved in almost the same way. Also, the proposed models allow for computationally simple scale and orientation adaptation for texture simulation or retrieval.
You will not find in this book a thorough review of other known texture models although some limitations of more traditional Gibbs models, being overcome by the proposed approach, are outlined. Such comprehensive reviews can be easily found elsewhere.
Also, you will not find the experimental comparisons with other models because this book does not intend to provide evidence that the proposed models are much better. In fact, sometimes they do outperform, but in other cases may be inferior to, the known counterparts. The proposed models and modelling frameworks simply differ from and possess very attractive learning capabilities in comparison to the more traditional ones.
Generally, each mathematical model has its own place in describing images, and all of them are too simple to represent even a tiny part of such a natural miracle as (human) vision. So, when a model does emulate a small part of natural images, it deserves to be known and used in practice.
Now, after this brief review, let us enter the exciting world of image textures, structures, and multiple pairwise pixel interactions...
A hardbound copy of this book, ISBN 0-7923-5961-5, costs NLG 215.00 / US$ 130.00 / GB£ 75.00
You may send the order to your supplier or:
| For the Americas: | ||
|---|---|---|
|
Kluwer Academic Publishers
Order Department, P.O.Box 322 3300 AH Dordrecht, The Netherlands |
Kluwer Academic Publishers
Order Department, P.O.Box 358 Accord Station, Hingham, MA 02018-0358 USA | |
| Fax: +31-78-6546474
Phone: +31-78-6392392 Email: orderdept@wkap.nl | Fax: (781) 681-9045
Phone: (781) 871-6600 Email: kluwer@wkap.com |