Fitting finite mixtures of multivariate t-distributions via the em algorithm.

Finite mixture models are an important tool in modelling the distribution
of a wide class of statistical problems. Mathematically, a g−component
finite mixture model of multivariate t-distributions is specified by a convex
combination (mixture distribution) of g multivariate t-distributions, which
is given by
F(x; Ψ) =
Xg
j=1
τjf(x; μj ,Σj , νj)
where Ψ =
��
τ1, · · · , τg−1, θT
1 , · · · , θT
g
T
is the parameter vector containing
all the parameters of the mixture model, θj =
��
μj ,Σj , νj
T
is the parameter
of the jth component of the mixture model whose location parameter is
μj , scale matrix Σj , degrees of freedom νj and the jth component mixing
proportion τj , with
Pg
j=1 τj = 1 for j = 1, 2, · · · , g. In the computation of
maximum likelihood estimates of Ψ, the EM algorithm often outshines other
iterative estimation procedures. The main shortcoming of the EM process
is that convergence to the global mode is not guaranteed due to dependence
on the starting point Ψ(0). To address the problem of convergence to a
local mode when fitting data to mixture models via the EM algorithm,
optimized EM initialization methods such as k-means algorithm have been
developed in the selection of Ψ(0), especially when the underlying mixture
model features Gaussian distributions. Compared to mixtures of Gaussian
distributions however, mixtures of t-distributions have been identified as
more robust modelling tools due to their heavier tails. The pitfall is that
the later have more parameters to be estimated. This may make common
EM initialization methods insufficient at attaining the global mode. Hence
the need for more refined initialization methods. In this study, we extend
the application of the burn-in techniques to mixtures of multivariate t -
distributions. The performance of the burn-in scheme is compared with kmeans
algorithm, hierarchical clustering and random start based methods.
The implementation of the EM algorithm using the package EMMIXskew
in the statistical software R, shows that global convergence percentage is
highest with the burn-in scheme initialized EM algorithm. With examples
using various data sets, we show that the burn-in scheme is a competitive
EM initialization method, even when the underlying mixture model features
multivariate t-distributions.
Key Words: Finite Mixture models, Mixture distributions, EM algorithm,
Multivariate t-distributions, the k-means algorithm, Hierarchical clustering,
Random starts, Burn-in scheme, EMMIXskew.