Future Work

Although the deformation of texels might be difficult to be modelled using global image statistics, the comparative study on texture synthesis by non-parametric sampling suggests that the property might be replicated as is by directly transferring training texels from the training into the synthetic texture. An extension to the existing method that incorporates both global and local texture feature measures might be considered as a prospective direction of future work in the pursuit of a complete image model and more realistic and efficient texture synthesis.

Figure 8.1: The difference image (c) between weakly-homogeneous texture D3 [8] (a) and its homogenised prototype synthesised by bunch sampling (b).

$\includegraphics[scale=0.65]{d3.bmp.eps}$	$\includegraphics[scale=0.65]{d3-g128.bmp.eps}$	$\includegraphics[scale=0.65]{d3-diff.bmp.eps}$
(a)	(b)	(c)

Intuitively, a possible solution is to decompose a weakly-homogeneous texture into a component related to the basic periodicity structure and an extra component containing the information on local deformation, and to process two components separately by different algorithms. Some investigations into this direction have recently been carried out in [66]. The method considers a weakly-homogeneous texture formed by applying local deformation onto a homogeneous prototype texture. Therefore, a weakly-homogeneous texture can be separated into a regular and an irregular components. The regular component is related to a homogenised prototype of the original texture, which is very similar to a rectified synthetic texture by the bunch sampling. The irregular component is represented by a deformation field, i.e. a function of the geometric distortion between the underlying translation lattices of the original texture and the homogeneous prototype. The decomposition allows to use different algorithms for the regular and irregular comports. The homogeneous component is synthesised as a regular texture using the method in [67], while the deformation field is synthesised as a stochastic texture using pixel-based non-parametric sampling [29]. Then, two synthesised images are combined using image warping to create a final synthetic texture. The obtained texture is expected to preserve the global regularity and the local variation of the training image. However, the need for interactively identifying the translation lattice from an input weakly-homogeneous texture makes the algorithm less applicable practically. In addition, the method implies a perhaps less justified assumption that local deformation in a texture could be modelled by a stochastic, continuous field that is independent of the homogenous prototype. But as shown in Fig 8.1, for example, the difference image, containing information about local deformation, is still more or less texture-alike which might not be simply represented by a random deformation field independent of the homogeneous prototype. So, future research in this direction would be to investigate whether texture local deformation could be extracted without human intervention and be modelled in a more generic way. In many textures, slight local deformation of repeated elements is approximately related to certain affine transformation [62,88]. This indicates another possible direction of approximating local image distortion via spatial coordinate transformation. That is, the representation of a weakly-homogeneous texture can also involves an extra field of certain geometric transformations.

The proposed bunch sampling can generate a homogenous prototype from an input weakly-homogeneous texture, and the differences between two images might provide a cue for recovering local deformation in a texture. If the homogeneous prototype is called the reference image, the goal is to find certain geometric transformations that aligns an input weakly-homogeneous texture with the reference one, e.g., mapping locations in the input texture to new locations in the reference image. This task could be formulated as a problem of image registration [10]. But, in this case, a single spatial transformation might not be adequate for aligning two images, so piecewise transformation should be considered which subdivides the images into pieces and uses different transformation parameters for each piece. Since texture synthesis might extends the image plane to an arbitrary size, the resulting transformation field should be extensible.

The first attempt to find piecewise affine transformations that align a weakly-homogeneous texture with its homogeneous prototype was conducted by taking the following procedure,

$\displaystyle \left( \begin{array}{c} x^{'}\\ y^{'}\end{array} \right) = \left... ...nd{array} \right) \cdot \left( \begin{array}{c} x\\ y\\ 1\end{array} \right)$

A combined search method based on Hooke-Jeeves optimisation [49] is tested, which exhaustively searches for affine parameters that align two patches so that SSD or normalised correlation between them is minimised. Meanwhile, an affine warping algorithm [69] is also tested. Experiment results suggest the main practical challenge is to find a feasible technique that allows to reliably estimate the affine parameters for especially rather small image patches. As shown in Fig 8.2, the piecewise affine transformations obtained using the mentioned methods produce unsatisfactory alignment between two images (the image (c) should be similar to the image (a), if correct affine parameters are obtained.)

Figure 8.2: Affine transformation that relates a weakly-homogeneous texture with its prototype. (a) Texture D3 (b) Homogeneous prototype of D3 (c) An image obtained by applying obtained piecewise affine transformation onto the homogeneous prototype.

$\includegraphics[scale=0.65]{d3.bmp.eps}$	$\includegraphics[scale=0.65]{d3-g128.bmp.eps}$	$\includegraphics[scale=0.65]{d3-affine.bmp.eps}$
(a)	(b)	(c)