Pattern Analysis in Image Processing Essay

essay A+
  • Words: 1828
  • Category: Database

  • Pages: 7

Get Full Essay

Get access to this section to get all the help you need with your essay and educational goals.

Get Access

Abstract-Finite mixture theoretical account plays a critical function in the field of Image Processing. Earlier due to miss of computational resources, methods like Maximum Likelihood and Bayesian theoretical accounts were non executable. The method therefore predominating were Graphical Method, Method of Moment etc. They are easy to implement but do non supply good consequences. Maximal Likelihood appraisal therefore came into consideration with Expectation Maximization algorithm. As it is non applicable for complex Bayesian information, EM algorithm is so extended to utilize Bayesian belongingss called as Variational Bayesian illation.

Index footings – Finite mixture theoretical account, graphical method, method of minutes, Expectation maximization algorithm, variational Bayesian Inference,


The importance of finite mixture theoretical accounts in the statistical analysis of informations is emphasized by the of all time increasing demand of pattern analysis in the field of Image Processing. The purpose of this paper is to supply an up-to-date history of the theory and applications of different patterning strategies via finite mixture distributions.

One of the first major analysis affecting the usage of mixture theoretical account was undertaken over 100 old ages ago by a celebrated biometrician Karl Pearson [ ] , who fitted a mixture of two normal chance denseness maps with different average discrepancy and weights. The chief progresss in finite mixture modeling has been done in past 50- 60 old ages. Many methods emerged and one of them is EM algorithm. It is extremely convergent and assures convergence to existent upper limit [ ] . It is easy to implement but it has many disadvantages besides, such as, initial parametric quantities are to be given apriori. Besides choice of the parametric quantities are slippery. Number of constituents are besides needed.

These issues are so minimized by Bayesian technique for the analysis of complicated statistical theoretical accounts.

This paper begins with a treatment of mixture theoretical accounts and their applications and gives a brief overview of the current province of the country followed by maximal likeliness adjustment of mixture theoretical accounts via the EM algorithm, Issues sing get downing value, halting standards etc. After that, Bayesian attack to the adjustment of mixture theoretical accounts has been considered. Bayesian attack is now executable utilizing posterior simulations via late developed MCMC methods. Bayes calculators for mixture theoretical accounts are good defined so long as the anterior distributions are proper.


Letbe ‘’ independent and identically random vectors with dimensionthat follow aconstituent Gaussian mixture distribution. TheGaussian constituent in the mixture is defined by its mean, covarianceand the commixture coefficient(& A ; gt ; 0 and) . These parametric quantities together are represented as the parametric quantity vector.

The posterior pdf is given by,


An illustration of two normal homoscedastic constituent mixture is shown

There are some applications for the appraisal of finite mixture theoretical account such as bunch of mixture informations utilizing mixture likeliness attack, decision-Theoretic attack which provides a convenient frame work for the building of discriminant regulations in the state of affairs where an allotment of an unclassified entity is required, constellating of IID informations, image cleavage etc.

A assortment of attacks have been used to gauge mixture distributions. They are graphical method, methods of minute, Minimum distance methods, Maximum Likelihood and Bayesian attacks.


A probabilistic graphical theoretical account is a household of chance distributions that can be defined in footings of a directed or adrift graph. Nodes in the graph represents random variables and the borders represent their dependences among the variables [ ] , [ ] . A directed border from A towards B denotes the stochastic dependence of node B on node A.

Fig. 2. Example of Directed Graph. Circles represent random variables and squares represent parametric quantities [ ]


The method of minutes is defined as the appraisal of parametric quantities by associating the population minutes. A minute is the expected values of powers of the coveted random variable to the parametric quantities of involvement. It represents the different parametric quantities on which the random variables are dependent. Model parametric quantities are chosen to stipulate a distribution whoseJThursdayorder minutes, for several values ofJ, are equal to the corresponding empirical minutes observed in the information.

The method of minutes offers relative simpleness in calculation when compared to others, such as maximal likeliness appraisal. The appraisal of these minutes merely requires work outing a multinomial equation and a system of additive equations, which can be accomplished even manually. On the other manus, the estimations are biased and may non supply sufficient statistics. It fails if the information size is big. Besides ML estimation provide higher chance of being close to the measures to be estimated and are more frequently indifferent.


The Minimum Distance method was developed by Wolfowitz [ ] . Let X=X1, Ten2, …XNbe a random sample of distribution map F ( x, ? ) and vitamin D ( * , * ) be any non-negative distance map. Let ? ?be an unknown parametric quantity of F ( x, ? ) and FN( ten ) be the empirical distribution map based on X. The minimal distance estimation is so defined if there exist ain, such that. [ ]


Maximal Likelihood appraisal is defined as the appraisal of value of ? that maximizes the likeliness map. Foras ‘’ independent and identically random vectors with dimensionthat follow aconstituent Gaussian mixture distribution, the joint denseness map is defined as

Hereis called as likeliness of the map X. Taking log of the likeliness outputs

The above map is called as the log-likelihood map. As it is known that log map is monotonically increasing map, so by maximising the log-likelihood means maximising the chance of acquiring a nice end product.

EM algorithm

In the model of EM algorithm, the mixture informations is formulated as if it consist of uncomplete observed informations, as the associated constituent label vectorsare non available. Forbe ‘’ independent and identically random vectors with dimensionthat follow aconstituent Gaussian mixture distribution. TheGaussian constituent in the mixture is defined by its mean, covarianceand the commixture coefficient(& A ; gt ; 0 and) . These parametric quantities together are represented as the parametric quantity vector. The end is to gauge theoretical account parametric quantities ? given the information. It is assumed that the constituent labelsare distributed unconditionally harmonizing to the distribution. Letbe the losing informations in which. If, it means that the IThursdayinformations point belongs to the JThursdaycategory. Therefore, the joint pdf of the complete informationsbecomes

Where,, by taking the log of the above equation, the log likeliness map is given as

EM algorithm follows two stairss: –

Expectation ( E ) measure

This measure is similar to the computations of MLE. In this measure, old parametric quantity values are used to cipher the posterior pdf. It is given by,


Heredenotes the denseness offromcategory with matching parametric quantitiesBy Bayes theorem, posterior distribution on the mixture constituents can be calculated as

Here omegaJemaah Islamiyahdenotes the chance ofinformations point belonging tobunch.

Maximization ( M ) measure

In this measure, the posterior pdfis used to cipher the new parametric quantity values.


All the above calculated parametric quantities are used to maximise the log likeliness map

EM algorithm can be summarized in the undermentioned mode.

Set initial values ofand ? & A ; gt ; 0.

  1. Calculatewithutilizing ( 3 ) .
  2. Calculateandwithutilizing ( 4 ) and ( 5 ) .
  3. Calculatewithandutilizing ( 6 ) .
  4. Updatewithutilizing ( 3 ) .
  5. Comparisonandby ciphering |||| .

If |||| & A ; lt ; ? so halt, else t = t+1 and travel to step 1.

Convergence belongingss of EM algorithm had been widely studied [ 7 ] . Its convergence belongingss are studied for Gaussian Mixture theoretical account besides [ 8, 9 ] . There are some drawbacks of EM algorithm stated as follows:

Initial parametric quantities are to be given apriori. That is, figure of bunchs and their initial conjecture of parametric quantities must be known. Besides, its convergence is slow and the consequence varies harmonizing to the initial parametric quantity values and halting standards.


Bayesian attack of finite mixture theoretical account follows Bayes regulation. The methodological analysis used is called as Variational Bayesian illation. It is a procedure in which anterior chances and observations along with the normalizing invariable are used to deduce posterior chance. This variational estimate method can be used to work out complex Bayesian theoretical accounts where the EM algorithm fails. In these instances, the appraisal of likeliness map is either really complex or non possible to calculate it straight.

EM algorithm is really a particular instance of Bayesian illation [ ] . EM algorithm first assumes the posterior chance and so maximizes the likeliness map, but it may non ever be possible to avail the posterior PDF. In such instances variational Bayesian illation is used where it approximates the posterior PDF. As perennial sampling or big premises are non required for Bayesian statistics, makes it convenient to utilize. It removes the low-level formatting job of EM algorithm by maximising the fringy likeliness chance by incorporating the concealed variables. Besides, EM algorithm can non be applied on complex Bayesian theoretical accounts. For these instances, if decently constructed, it has the ability to pattern important belongingss i.e. parametric quantities of the informations coevals theoretical account and supply good solutions.


Up till now, many finite mixture theoretical accounts have been proposed. Some of them are discussed here. Graphic method provides a simpler solution to the parametric quantity appraisal but is by and large a colored calculator. It can non supply minimal discrepancy estimations even for a big information set. Method of Moment are besides easy to implement but are non ever available they besides do non supply optimum solution like maximal likeliness and least squares error estimations. The above disadvantages are removed by the Maximum likeliness in Expectation maximization algorithm. EM algorithm is guaranteed to meet to a local upper limit of the informations log-likelihood as map of its parametric quantities [ 7 ] . It gives accurate consequences, provided initial parametric quantities are known. Expectation Maximization algorithm has attractive characteristics such as dependable planetary convergence, low cost per loop, economic system of storage, and easiness of programming. However it has additive convergence rate [ 6 ] and it is by experimentation proved that EM converges easy in presence of overlapping bunchs. One of its chief drawback is that the initial parametric quantity value must be known apriori. It besides fails for the complex Bayesian information sets.

All these defects are removes by Variational Bayesian illation. It is really an extension of Expectation Maximization Algorithm. It works by taking the normalizing factor of Bayes expression into the history. Here the parametric quantities are taken as variables which is solved by ciphering their ain PDFs. One of the drawback is that improper priors yield improper posterior distributions. Besides it is non ever possible to avail the estimate of the initial parametric quantities.


J. Guo, E. Levina, G. Michailidis and J. Zhu, “Graphical Models for Ordinal Data” , Department of Statistics, University of Michigan, Ann Arbor, Oct 2012.

M. I. Jordan, “Graphical Models” , Statistical Science, Institute of Mathematical Statistics, Vol. 19, No. 1, pp. 140-155, Feb. , 2004

A. Anandkumar, D. Hsu, S.M. Kakade, “A Method of Moments for Mixture Models and Hidden Markov Models” , 25th Annual Conference on Learning Theory, Workshop and Conference Proceedings vol 23, pp. 33.1–33.34, 2012

P. J. Farrell, A. K. Md. E. Saleh and Z. Zhang, “Methods of Moments Estimation in Finite Mixtures” , The Indian Journal of Statistics, Volume 73-A, Part 2, pp. 218-230, 2011.

J. Wolfowitz, “Estimation by the minimal distance method” , Annual Math. Statist. 28, pp. 75-87, 1957.

C.A.Drossos, A.N.Philippou, “A note on minimal distance estimates” , Ann. Inst. Statist. Math.32, pp. 121-123, 1980.

Get instant access to
all materials

Become a Member