Compute the the maximum likelihood estimates of a complete Gaussian statistical model. More...

#include <STK_IGaussianModel.h>

Inheritance diagram for STK::IGaussianModel< Array >:

Public Member Functions
virtual	~IGaussianModel ()
	destructor.

RowVector const &	mean () const

Public Member Functions inherited from STK::IStatModel< IGaussianModel< Array > >
	~IStatModel ()
	destructor

Data const *const	p_dataij () const

void	setData (Data const &data)
	Set the data set of the model.

void	setData (Data const *p_data)
	Set the data set of the model.

Public Member Functions inherited from STK::IStatModelBase
int	nbSample () const

Real	lnNbSample () const

int	nbVariable () const

Real	lnLikelihood () const

Real	likelihood () const

int	nbFreeParameter () const

Real	computeBIC () const

Real	computeAIC () const

Real	computeML () const

Protected Member Functions
	IGaussianModel (Array const *p_data)
	constructor.

	IGaussianModel (Array const &data)
	constructor.

void	compMean ()
	compute the empirical mean

void	compWeightedMean (ColVector const &weights)
	compute the empirical weighted mean

virtual void	compCovariance ()=0
	compute the empirical covariance matrix.

virtual void	compWeightedCovariance (ColVector const &weights)=0
	compute the empirical weighted covariance matrix.

Protected Member Functions inherited from STK::IStatModel< IGaussianModel< Array > >
	IStatModel (Data const &data)
	Constructor with data set.

	IStatModel (Data const *p_data)
	Constructor with a ptr on the data set.

Protected Member Functions inherited from STK::IStatModelBase
	IStatModelBase ()
	Default constructor.

	IStatModelBase (int nbSample)
	Constructor with specified dimension.

	IStatModelBase (int nbSample, int nbVariable)
	Constructor with specified dimension.

	IStatModelBase (IStatModelBase const &model)
	Copy constructor.

	~IStatModelBase ()
	destructor

void	setNbFreeParameter (int const &nbFreeParameter)
	set the number of free parameters of the model

void	setNbSample (int const &nbSample)
	set the number of samples of the model

void	setNbVariable (int const &nbVariable)
	set the number of variables of the model

void	setLnLikelihood (Real const &lnLikelihood)
	set the log-likelihood of the model

void	initialize (int nbSample, int nbVariable)
	set the dimensions of the parameters of the model

Protected Attributes
RowVector	mean_
	Vector of the empirical means.

Protected Attributes inherited from STK::IStatModel< IGaussianModel< Array > >
Data const *	p_dataij_
	A pointer on the original data set.

Private Types
typedef IStatModel< IGaussianModel< Array > >	Base

typedef hidden::Traits< Array >::Row	RowVector

typedef hidden::Traits< Array >::Col	ColVector

Additional Inherited Members
Public Types inherited from STK::IStatModel< IGaussianModel< Array > >
typedef hidden::ModelTraits< IGaussianModel< Array > >::Data	Data

typedef hidden::ModelTraits< IGaussianModel< Array > >::ParamHandler	ParamHandler

typedef Data::Type	Type
	Type of the data contained in the container.

typedef hidden::Traits< Data >::Row	Row
	Type of the row of the data container (a sample)

Detailed Description

template<class Array>
class STK::IGaussianModel< Array >

Compute the the maximum likelihood estimates of a complete Gaussian statistical model.

A random vector $X \in; \mathbb{R}^p$ is Gaussian if

$X\ \sim\ \mathcal{N}(\mu,\ \Sigma).$

The likelihood function of a gaussian sample is:

$L(\mu,\Sigma)=[\mbox{constant}] \prod_{i=1}^n \det(\Sigma)^{-1/2} \exp\left(-\frac{1}{2} (X_i-\mu)^T \Sigma^{-1} (X_i-\mu)\right)$

The maximum likelihood estimate can be performed via matrix calculus formulae Re-write the likelihood in the log form using the trace trick:

$\ln L(\mu,\Sigma) = \operatorname{const} - \frac{n}{2} \ln \det(\Sigma) -\frac{1}{2} \operatorname{tr} \left[ \Sigma^{-1} \sum_{i=1}^n (X_i-\mu) (X_i-\mu)^T \right].$

The differential of this ln-likelihood is

$d \ln L(\mu,\Sigma) = -\frac{n}{2} \operatorname{tr} \left[ \Sigma^{-1} \left\{ d \Sigma \right\} \right] -\frac{1}{2} \operatorname{tr} \left[ - \Sigma^{-1} \{ d \Sigma \} \Sigma^{-1} \sum_{i=1}^n (X_i-\mu)(X_i-\mu)^T - 2 \Sigma^{-1} \sum_{i=1}^n (X_i - \mu) \{ d \mu \}^T \right].$

It naturally breaks down into the part related to the estimation of the mean, and to the part related to the estimation of the variance. The first order condition for maximum, $d \ln L(\mu,\Sigma)=0$ , is satisfied when the terms multiplying $d \mu$ and $d \Sigma$ are identically zero. Assuming (the maximum likelihood estimate of) $\Sigma$ is non-singular, the first order condition for the estimate of the mean vector is

$\sum_{i=1}^n (X_i - \mu) = 0,$

which leads to the maximum likelihood estimate

$\hat{\mu} = \bar{X} = \frac{1}{n} \sum_{i=1}^n X_i.$

This lets us simplify $\sum_{i=1}^n (X_i-\mu)(X_i-\mu)^T = \sum_{i=1}^n (X_i-\bar{X})(X_i-\bar{X})^T = S$ . Then the terms involving $d \Sigma$ in $d \ln L$ can be combined as

$-\frac{1}{2} \operatorname{tr} \left( \Sigma^{-1} \left\{ d \Sigma \right\} \left[ nI_p - \Sigma^{-1} S \right] \right).$

The first order condition $d \ln L(\mu,\Sigma)=0$ will hold when the term in the square bracket is (matrix-valued) zero. Pre-multiplying the latter by $\Sigma$ and dividing by $n$ gives $\hat{\Sigma} = \frac{1}{n} S,$