Approximate Potential Estimates

Recall Eq. (6.2.5). Given the neighbourhood $\mathcal{N}$, the specific log-likelihood of the potential:

$$\ell(\mathbf{V}\vert g^\circ) = \frac{1}{\vert\mathcal{R}\vert}\l... ...m\limits_{g\in\mathcal{G}} \exp\left( \mathbf{V}^\mathsf{T}\mathbf{F}(g)\right)$$ (A.0.1)

has the following gradient ( $\nabla\ell$) and the Hessian matrix ($\nabla^2$):

$$\begin{array}{lll} \nabla\ell(\mathbf{V}\vert g^\circ) & = & ... ... & = & -\mathbf{C}\{\mathbf{F}(g)\vert\mathbf{V}\} \end{array}$$ (A.0.2)

where $\E\{\ldots\vert\mathbf{V}\}$ denotes the math expectation under the GPD of Eq. (6.2.5) and $\mathbf{C}\{\mathbf{F}(g)\vert\mathbf{V}\}$ is the covariance matrix of the scaled empirical probability vectors. Because the covariance matrix is non-negatively defined, the log-likelihood is unimodal in the potential space [2]. The vector $\E\{\mathbf{F}(g)\vert\mathbf{V}\}$ of the scaled marginal co-occurrence probabilities in the vectorial equation $\nabla\ell(\mathbf{V}\vert g^\circ) = \mathbf{F}(g^\circ)- \E\{\mathbf{F}(g)\vert\mathbf{V}\} = 0$ for the exact MLE of $\mathbf{V}$ and the covariance matrix $\mathbf{C}\{\mathbf{F}(g)\vert\mathbf{V}\}$ are typically unknown except in the cases when the MGRF of Eq. (6.2.5) is reduced to an independent random field (IRF).

Starting from a potential $\mathbf{V}_0$ producing an IRF, the maximum likelihood estimate (MLE) of the potential is approximated by generalising the analytical approach proposed in [38].

PROPOSITION A.0.1   If the gradient and Hessian of Eq. (A.0.2) are known for an image $g^\circ$ and potential $\mathbf{V}_0$, the first approximation of the MLE:

$$\mathbf{V}^\ast = \mathbf{V}_0 + \lambda^\ast\nabla \ell(\mathbf{... ...nabla^2 \ell(\mathbf{V}_0\vert g^\circ) \nabla \ell(\mathbf{V}_0\vert g^\circ)}$$ (A.0.3)

maximises the second-order Taylor series expansion of the log-likelihood

$$\ell(\mathbf{V}\vert g^\circ) \approx \ell(\mathbf{V}_0\vert g^\c... ...)^\mathsf{T} \nabla^2 \ell(\mathbf{V}_0\vert g^\circ)(\mathbf{V}-\mathbf{V}_0)$$

along the gradient from $\mathbf{V}_0$.

It is easily proved by substituting $\mathbf{V}=\mathbf{V}_0 + \lambda\nabla \ell(\mathbf{V}_0\vert g^\circ)$ to the expansion and maximising the latter with respect to $\lambda$.

The approximate solution in [38] presumes the simplest IRF (denoted below $\mathrm{IRF}_0$) with zero potential $\mathbf{V}=\mathbf{0}$. It results in equal marginal probabilities $p(q) = \frac{1}{Q}$ of independent signals $g_{x,y}=q$; $q\in\mathcal;{Q}$, over $\mathcal{R}$ and equiprobable images in Eq. (6.2.5): $P_\mathbf{0}(g^\circ)=\frac{1}{Q^\vert\mathcal{R}\vert}$. In this case $\ell\{\mathbf{0}\vert g^\circ\}=-\ln Q$ and all pairwise co-occurrence probabilities are equal: $p_{\xi,\eta}(q,s)=\frac{1}{Q^2}$.

Let $\mathbf{P}_{0}$ be the vector of the scaled marginal co-occurrence probabilities for the $\mathrm{IRF}_0$: $\mathbf{P}_{0}^\mathsf{T} = \frac{1}{Q^2}\left[ \rho_{\xi,\eta}\mathbf{u}: (\xi,\eta)\in\mathcal{N} \right]$ where $\mathbf{u}^\mathsf{T}=[1,1,\ldots,1]$ is the $Q^2$-vector of unit components. Let $\mathbf{F}_\mathrm{cn}(g^\circ)$ be the vector of the centred scaled empirical co-occurrence probabilities $f_{\mathrm{cn};\xi,\eta}(q,s)=f_{\xi,\eta}(q,s\vert g^\circ)-\frac{1}{Q^2}$ for the image $g^\circ$, i.e. $\mathbf{F}_\mathrm{cn}^\mathsf{T}(g^\circ) = \left[ \rho_{\xi,\eta}[f_{\mathrm... ...(q,s\vert g^\circ) : (q,s)\in\mathcal{Q}^2 ]: (\xi,\eta)\in\mathcal{N} \right]$,

where $\sum_{(q,s)\in\mathcal{Q}^2}f_{\mathrm{cn};\xi,\eta}(q,s\vert g^\circ)=0$ for all $(\xi,\eta)\in\mathcal{N}$.

Then the log-likelihood gradient is $\nabla\ell(\mathbf{0}\vert g^\circ)=\mathbf{F}_\mathrm{cn}(g^\circ)\equiv \mathbf{F}(g^\circ)-\mathbf{P}_{0}$ and the covariance matrix $\mathbf{C}\{\mathbf{F}(g)\vert\mathbf{0}\}$ is closely approximated by the scaled diagonal covariance matrix $\mathbf{C}_\mathrm{ind}$ for the independent co-occurrence distributions: $\mathbf{C}\{\mathbf{F}(g)\vert\mathbf{0}\}\approx\mathbf{C}_\mathrm{ind}$ where $\mathbf{C}_\mathrm{ind} = \frac{1}{Q^2}\left(1 - \frac{1}{Q^2}\right)\mathsf{Diag} \left[ \rho_{\xi,\eta} \mathbf{u}: (\xi,\eta)\in\mathcal{N} \right]$.

PROPOSITION A.0.2   The first approximation of the potential MLE in the vicinity of zero point $\mathbf{V}_0 = \mathbf{0}$ is

$$\mathbf{V}^\ast=\frac{ \mathbf{F}_\mathrm{cn}^\mathsf{T}(g^\circ)... ...rc) }\mathbf{F}_\mathrm{cn}(g^\circ) \approx Q^2\mathbf{F}_\mathrm{cn}(g^\circ)$$ (A.0.4)

Proof. In line with Eq. (A.0.3), the maximising factor $\lambda^\ast$ is equal to

$$\frac{ \mathbf{F}_\mathrm{cn}^\mathsf{T}(g^\circ) \mathbf{F}_\mat... ...\in\mathcal{Q}^2}\left(f_{\xi,\eta}(q,s\vert g^\circ)-\frac{1}{Q^2}\right)^2 }$$

If $Q\gg 1$ and the lattice $\mathbf{R}$ is sufficiently large to assume that $\rho_{\xi,\eta}\approx 1$ for all clique families, it is simplified to $\lambda^\ast\approx Q^2$. $\qedsymbol$