{"id":253,"date":"2017-04-18T03:27:13","date_gmt":"2017-04-18T03:27:13","guid":{"rendered":"http:\/\/sites.warnercnr.colostate.edu\/gwhite\/?page_id=253"},"modified":"2017-04-18T03:27:13","modified_gmt":"2017-04-18T03:27:13","slug":"optimization-method","status":"publish","type":"page","link":"https:\/\/sites.warnercnr.colostate.edu\/gwhite\/optimization-method\/","title":{"rendered":"Optimization Method"},"content":{"rendered":"

Contents<\/a> – Index<\/a><\/span><\/p>\n

\n

Optimization Method<\/b><\/span><\/p>\n

The default method of optimization of the likelihood function in Program MARK is the Newton-Raphson method with numerical derivatives.\u00a0 The VA09AD algorithm from Harwell library is the default method.\u00a0 However, Program MARK has available a second numerical optimization algorithm to maximize the likelihood function and obtain parameter estimates available in the Run Window<\/a>.\u00a0 Sometimes, the default optimization routine does not converge properly and using the second algorithm works.\u00a0 Select the second algorithm by clicking on the “Use Alt. Opt. Method” check box in the Run Window<\/a> dialog.<\/span><\/p>\n

The second method of optimization is simulated annealing.\u00a0 Simulated annealing is a global optimization method that distinguishes between different local optima. Starting from an initial point, the algorithm takes a step and the function is evaluated. When minimizing a function, any downhill step is accepted and the process repeats from this new point.\u00a0 An uphill step may be accepted.\u00a0 Thus, simulated annealing can escape from local optima.\u00a0 This uphill decision is made by the Metropolis criteria.\u00a0 As the optimization process proceeds, the length of the steps decline and the algorithm closes in on the global optimum. Since the algorithm makes very few assumptions regarding the function to be optimized, it is quite robust with respect to non-quadratic surfaces.\u00a0 Simulated annealing can be used as a local optimizer for difficult functions.\u00a0 The\u00a0 implementation of simulated annealing in Program MARK was used in “Global Optimization of Statistical Functions with Simulated Annealing,” Goffe, Ferrier and Rogers, Journal of Econometrics, vol. 60, no. 1\/2, Jan.\/Feb. 1994, pp. 65-100.<\/span><\/p>\n

The difficulty with using simulated annealing is that the algorithm is very inefficient compared to the Newton-Raphson method that is the default optimization method in MARK.\u00a0 However, some data types, specifically the multi-strata data types<\/a>, may have multiple maxima, i.e., local maxima may exist.\u00a0 As a result, the default optimization method may end up at local maximum of the likelihood instead of the global maximum.\u00a0 Simulated annealing is more likely to find the global maximum.<\/span><\/p>\n

Simulated annealing searches within a bounded range of the parameters.\u00a0 In MARK, none of the link functions<\/a> put constrains on the beta parameters<\/a>, i.e., the parameters that are actually optimized.\u00a0 The main problem occurs with the logit link function<\/a> because extreme values of beta parameters can become fixed in the tails (extremes) of this function.\u00a0 That is, the derivative of the inverse logit function appears to be zero because the parameter is apparently constant.\u00a0 Because the simulated annealing optimization method requires a range over which parametgers may vary, the constrains on the beta parameters in the range -20 to 20 have been set, so that the logit link (and the MLogit and CLogit link functions) will function reasonably well.\u00a0 This range is not a problem for the sin link, but may occasionally be a problem with population estimation parameters (N) for closed capture models<\/a> that use the log link<\/a><\/span><\/p>\n

As a consequence of the constraint that beta parameters must be in the [-20, 20] interval, some models may not fit will with the simulated annealing optimization method.\u00a0 I suggest that you only use this optimization method to get near the global maximum, and then use the parameter values from the resulting model as initial values<\/a> for the default optimization method.<\/span><\/p>\n

Warning:<\/b> the simulated annealing optimization method can take as much as 10 to 100 times the computer time of the default optimization method.\u00a0 Only use this alternative method if you suspect a problem with local maxima.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

Contents – Index Optimization Method The default method of optimization of the likelihood function in Program MARK is the Newton-Raphson method with numerical derivatives.\u00a0 The VA09AD algorithm from Harwell library is the default method.\u00a0 However, Program MARK has available a …<\/p>\n

Optimization Method<\/span> Read More »<\/a><\/p>\n","protected":false},"author":117,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-253","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/sites.warnercnr.colostate.edu\/gwhite\/wp-json\/wp\/v2\/pages\/253","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sites.warnercnr.colostate.edu\/gwhite\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/sites.warnercnr.colostate.edu\/gwhite\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/sites.warnercnr.colostate.edu\/gwhite\/wp-json\/wp\/v2\/users\/117"}],"replies":[{"embeddable":true,"href":"https:\/\/sites.warnercnr.colostate.edu\/gwhite\/wp-json\/wp\/v2\/comments?post=253"}],"version-history":[{"count":1,"href":"https:\/\/sites.warnercnr.colostate.edu\/gwhite\/wp-json\/wp\/v2\/pages\/253\/revisions"}],"predecessor-version":[{"id":254,"href":"https:\/\/sites.warnercnr.colostate.edu\/gwhite\/wp-json\/wp\/v2\/pages\/253\/revisions\/254"}],"wp:attachment":[{"href":"https:\/\/sites.warnercnr.colostate.edu\/gwhite\/wp-json\/wp\/v2\/media?parent=253"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}