Patch-based Sampling

[65], proposed by Liang et al., also transfers image signals directly from the training to the synthetic textures. Similar local neighbourhood matching is also involved for finding proper image signals for each output location.

However, the method considers a rectangular image patch instead of a single pixel at each step of the synthesis. Each rectangular patch has a boundary zone, being the area surrounding four borders inside the patch (See Fig 5.6 (a)). The difference between boundary zones provides a measure of similarity for two related patches.

Algorithmically, the patched-based sampling is also very similar to the pixel-based algorithm in Algorithm 4. At each step, a patch, which has the closet boundary zone to the patch at the current location, is selected from the training image and is then stitched into the output image such that its boundary zone overlaps with that of the last synthesised patches (See Fig 5.6 (b)). A blending algorithm has to be used in order to smooth the transition between overlapping patches.

This method is generally faster than pixel-based non-parametric sampling, but it has the similar limitations. First, an image patch is usually restricted to be in a rectangular shape, because otherwise a boundary zone is practically difficult to define and to use, i.e. it could be difficult to compare and stitch two irregular boundary zones. Second, the size and the boundary zone of an image patch are two key parameters to the synthesis; the former should be related to the size of local structures for each particular texture, while the latter implies the level of statistical constraints imposed on the sampling process. But, it is still an open challenge to automate the process of finding good parameters for different textures. Third, the method blends overlapping areas, which could blur resulting textures and cause visual artifacts.

Figure 5.7: Minimum error boundary cut in Image Quilting. (a) The overlapping area of patches A and B between the dash lines. (b) The minimum error path (the irregular dash line) determines which portion of each patch contributes to the synthetic texture.

(a)	(b)