is statsmodels' MLEModel class using Expectation Maximization for fitting?

user_4003308 0
Viewed 179

I am building a custom stat space model using statsmodels' MLEModel class, and fitting the unknown parameters with the .fit() method. I thought that it was using Expectation Maximization, but I am not sure of that and I cannot find any hint to this in the documentation. Moreover, the verbose output of the .fit() method shows the steps of a single optimization, and this makes me even more doubtful.

If it is not using EM, what is it doing? maybe I am missing something here, but I thought that for models with hidden variables (like state space models) you cannot directly minimize the likelihood (since you don't know and don't observe the hidden states).

thanks for any suggestion

expectation-maximization statsmodels python
2 Answers

By default, MLEModel uses a quasi-Newton approach to numerically find parameters that maximize the likelihood function. Ultimately, it relies on the Scipy minimize function. The default algorithm is the BFGS algorithm, but you can select others using the method argument. For example, method='nm' uses the Nelder-Mead algorithm.

The Statsmodels state space framework provides all the tools (e.g. the appropriate smoothers) to use the EM algorithm, but you would have to implement it yourself. For example, the EM algorithm is the default method used to fit the DynamicFactorMQ model, because it has a large number of parameters, and quasi-Newton methods can be slow and unreliable for such problems.

user_6039620

In general, fit() is a method of the model class. Depending on the class object the method fit() from that class will be called. In general, fit() estimates parameters via maximum likelihood and return a results object.

The expectation maximization (EM) algorithm requires and uses the Kalman smoother for models with hidden variables if I'm not mistaken.

EDIT: fit() will cal the fit() of its parent class which in this case is tsbase.TimeSeriesModel as seen in class MLEModel(tsbase.TimeSeriesModel). In turn, class TimeSeriesModel(base.LikelihoodModel) will call its parent class which is base.LikelihoodModel. class LikelihoodModel(Model)'s fit() function default method argument is 'newton'. Thus, Maximum likelihood (ML) estimation will be estimated using Newton's method in this custom state space model.

user_29874880
Related