Optimization MT

Optimization MT 1.0

OPMT is intended for the optimization of functions. It has many features, including a wide selection of descent algorithms, step-length methods, and "on-the-fly" algorithm switching. Default selections permit you to use Optimization with a minimum of programming effort. All you provide is the function to be optimized and start values, and OPMT does the rest.

Special Features in Optimization MT 1.0

  • Internally threaded.
  • Uses structures.
  • Allows for placing bounds on the parameters.
  • Allows for computing a subset of the derivatives analytically, and for combining the calculation of the function and derivatives, thus reducing calculations in common between function and derivatives.
  • More than 25 options can be easily specified by the user to control the optimization
  • Descent algorithms include: BFGS, DFP, Newton, steepest descent, and PRCG
  • Step length methods include: STEPBT, BRENT, and a step-halving method
  • A "switching" method may also be selected which switches the algorithm during the iterations according to two criteria: number of iterations, or failure of the function to decrease within a tolerance

Threading in OPTMT

If you have a multi-core processor you may take advantage of this capability by selecting threading. Activate threading by setting the useThreads member of the optmtControl structure to 1.

struct optmtControl c0; /* Instantiate a structure, c0 */
c0 = optmtControl Create; /* set c0 to default values. */
c0.useThreads = 1; /* activate threading */



An important advantage of threading occurs in computing numerical derivatives. If the derivatives are computed numerically, threading will significantly decrease the time of computation.

Speed increases are similar to those observed with CMLMT, COMT and MLMT, approximately 300% faster on a quad-core machine for medium-small or larger problems.

Structures

OPTMT uses the DS and PV structures that are available in the GAUSS Run-Time Library.

The DS Structure

The DS structure is completely flexible, allowing you to pass anything you can think of into your procedure. There is a member of the structure for every GAUSS data type.

struct DS {
scalar type;
matrix dataMatrix;
array dataArray;
string dname;
string array vnames;
};



The PV Structure

The PV structure revolutionizes how you pass the parameters into the procedure. No longer do you have to struggle to get the parameter vector into matrices for calculating the function and its derivatives, trying to remember, or figure out, which parameter is where in the vector.

If your log-likelihood uses matrices or arrays,you can store them directly into the PV structure and remove them as matrices or arrays with the parameters already plugged into them. The PV structure can handle matrices and arrays in which some of their elements are fixed and some free. It remembers the fixed parameters and knows where to plug in the current values of the free parameters. It can also handle symmetric matrices in which parameters below the diagonal are repeated above the diagonal.

b0 - Mean paramters.
garch - GARCH parameters.
arch - ARCH parameters.
omega - Constant in variance equation.



There is no longer any need to use global variables. Anything the procedure needs can be passed into it through the DS structure. And these new applications uses control structures rather than global variables. This means, in addition to thread safety, that it is straightforward to nest calls to OPTMT inside of a call to OPTM, QNewtonmt, QProgmt, or EQsolvemt.

Improved Algorithm

Optimization implements the numerically superior Cholesky factorization, solve and update methods for the BFGS, DFP and Newton algorithms. The Hessian, or its estimate, are updated rather than the inverse of the Hessian, and the descent is computed using a solve. This results in better accuracy and improved convergence over previous methods.

Simple Bounds

Bounds may be placed on parameters. This can be very important for models with a limited parameter space outside of which the log-likelihood is not defined.

Platform: Windows, Mac, and Linux.

Requirements: GAUSS/GAUSS Light version 10 or higher.