Profile Likelihood Confidence Intervals


Profile Likelihood Confidence Intervals

The Run Window includes a check box to request profile likelihood confidence intervals be computed for the real parameter estimates.  The default confidence intervals for real parameter estimates in the 0-1 interval are based on the standard error and the logit transformation.  That is, a 95% confidence interval is computed on the logit estimate, and then these intervals are transformed to the real scale.  Use of the logit transformation precludes confidence interval boundaries outside the 0-1 interval.  However, problems are encountered with this approach when the parameter estimate is on the boundary, e.g., when a survival rate is estimated as 1, and/or the standard error is estimated as zero.

Profile likelihood confidence intervals are based on the log-likelihood function.  For a single parameter, likelihood theory shows that the 2 points 1.92 units down from the maximum of the log-likelihood function provide a 95% confidence interval when there is no extrabinomial variation (i.e. c = 1)..  The value 1.92 is half of the chi-square value of 3.84 with 1 degree of freedom.  Thus, the same confidence interval can be computed with the deviance by adding 3.84 to the minimum of the deviance function, where the deviance is the log-likelihood multiplied by -2 minus the -2 log likelihood value of the saturated model.

MARK will compute profile likelihood confidence intervals with extrabinomial variation assumed, i.e., c > 1.  To do so, the amount dropped below the maximum of the likelihood is c-hat*1.92, or the amount to increase above the minimum deviance is c-hat*3.84.

The method used in MARK to compute the profile likelihood lower bound for parameter i is to minimize the following function:

[-2 log likelihood of current parameter value  –  (-2 log likelihood of maximum likelihood estimates + c-hat*3.84) ]**2 – (maximum likelihood estimate of parameter i  –  current parameter value of i).

The first portion of this expression finds the value of the deviance that is c-hat*3.84 units larger than the deviance for the maximum likelihood parameter estimates.  The second portion of the expression maximizes the difference between the maximum likelihood parameter estimate and the lower bound.

For the upper bound, the function has to be modified maximize the difference between the maximum likelihood estimate and the upper bound:

[-2 log likelihood of current parameter value  –  (-2 log likelihood of maximum likelihood estimates + c-hat*3.84) ]**2 – (current parameter value of i  –  maximum likelihood estimate of parameter i).

This procedure appears to be different from the typical approach to estimate profile likelihood intervals because in MARK, the parameters that are optimized are the beta estimates, but we are interested in the profile intervals for the real parameters.  Thus, there is not always a 1 to 1 conversion from the beta estimates to the real estimates, e.g., trend models.  The approach described above accommodates this additional complexity.

Because of the additional optimization required to obtain the profile likelihood interval estimates, considerably more computer time is required to compute these intervals.

The following is an example comparing the default confidence intervals with the profile likelihood confidence intervals.  The example is from the North Park sage grouse data distributed with Program MARK in the NPMALES.DBF file.  First is the default output for the 30 survival estimates from the {S(a*T) r(a*T)} model.

                      North Park Male Sage Grouse Band Recovery Data
                       Real Function Parameters of {S(a*T) r(a*T)}
                                                              95% Confidence Interval
Parameter                  Estimate       Standard Error      Lower           Upper
————————-  ————–  ————–  ————–  ————–
    1:S                     0.5212251       0.0463568       0.4306641       0.6104122     
    2:S                     0.5017107       0.0406542       0.4226535       0.5806825     
    3:S                     0.4821911       0.0351372       0.4140880       0.5509623     
    4:S                     0.4627257       0.0300246       0.4046607       0.5218196     
    5:S                     0.4433733       0.0256337       0.3938728       0.4940243     
    6:S                     0.4241912       0.0224038       0.3809848       0.4685882     
    7:S                     0.4052348       0.0208173       0.3651713       0.4466016     
    8:S                     0.3865568       0.0211150       0.3460756       0.4286692     
    9:S                     0.3682068       0.0230520       0.3242870       0.4144267     
   10:S                     0.3502306       0.0260782       0.3009809       0.4028942     
   11:S                     0.3326698       0.0296737       0.2772552       0.3931355     
   12:S                     0.3155620       0.0334756       0.2538794       0.3845078     
   13:S                     0.2989398       0.0372553       0.2313377       0.3766175     
   14:S                     0.2828314       0.0408711       0.2099224       0.3692287     
   15:S                     0.2672600       0.0442350       0.1898019       0.3621974     
   16:S                     0.8083942       0.0596942       0.6646855       0.8997973     
   17:S                     0.7757114       0.0608475       0.6353812       0.8728417     
   18:S                     0.7392520       0.0609946       0.6039334       0.8405448     
   19:S                     0.6991644       0.0602202       0.5700751       0.8028948     
   20:S                     0.6557833       0.0587967       0.5334585       0.7604383     
   21:S                     0.6096393       0.0571881       0.4936989       0.7143893     
   22:S                     0.5614463       0.0559738       0.4505298       0.6665446     
   23:S                     0.5120654       0.0556664       0.4041007       0.6189161     
   24:S                     0.4624480       0.0564830       0.3552982       0.5731824     
   25:S                     0.4135633       0.0582287       0.3057997       0.5302933     
   26:S                     0.3663247       0.0603937       0.2577011       0.4904807     
   27:S                     0.3215234       0.0623774       0.2129511       0.4535516     
   28:S                     0.2797822       0.0636788       0.1729395       0.4191781     
   29:S                     0.2415309       0.0639865       0.1383693       0.3870549     
   30:S                     0.2070061       0.0631856       0.1093399       0.3569485     

Next is the output for the same model, but with profile likelihood intervals computed for the 30 survival estimates. The value of c-hat used to compute the profile likelihood confidence intervals is reported in the output, 

                      North Park Male Sage Grouse Band Recovery Data
                       Real Function Parameters of {S(a*T) r(a*T)}
                                                              95% Confidence Interval
Parameter                  Estimate       Standard Error      Lower           Upper
————————-  ————–  ————–  ————–  ————–
    1:S                     0.5212251       0.0463568       0.4305389       0.6107000       Profile c-hat=1.0000      
    2:S                     0.5017107       0.0406542       0.4226006       0.5809796       Profile c-hat=1.0000       
    3:S                     0.4821911       0.0351372       0.4140988       0.5512675       Profile c-hat=1.0000       
    4:S                     0.4627257       0.0300246       0.4047206       0.5221246       Profile c-hat=1.0000       
    5:S                     0.4433733       0.0256337       0.3939565       0.4943047       Profile c-hat=1.0000       
    6:S                     0.4241912       0.0224038       0.3810508       0.4687985       Profile c-hat=1.0000       
    7:S                     0.4052348       0.0208173       0.3651681       0.4467037       Profile c-hat=1.0000       
    8:S                     0.3865568       0.0211150       0.3459763       0.4286955       Profile c-hat=1.0000       
    9:S                     0.3682068       0.0230520       0.3241126       0.4144641       Profile c-hat=1.0000       
   10:S                     0.3502306       0.0260782       0.3007706       0.4030047       Profile c-hat=1.0000       
   11:S                     0.3326698       0.0296737       0.2770367       0.3933405       Profile c-hat=1.0000       
   12:S                     0.3155620       0.0334756       0.2536662       0.3848095       Profile c-hat=1.0000       
   13:S                     0.2989398       0.0372553       0.2311351       0.3770121       Profile c-hat=1.0000       
   14:S                     0.2828314       0.0408711       0.2097321       0.3697119       Profile c-hat=1.0000       
   15:S                     0.2672600       0.0442350       0.1896241       0.3627656       Profile c-hat=1.0000       
   16:S                     0.8083942       0.0596942       0.6637455       0.8966843       Profile c-hat=1.0000       
   17:S                     0.7757114       0.0608475       0.6360563       0.8699084       Profile c-hat=1.0000       
   18:S                     0.7392520       0.0609946       0.6062872       0.8382203       Profile c-hat=1.0000       
   19:S                     0.6991644       0.0602202       0.5740674       0.8017697       Profile c-hat=1.0000       
   20:S                     0.6557833       0.0587967       0.5388831       0.7611976       Profile c-hat=1.0000       
   21:S                     0.6096393       0.0571881       0.5001267       0.7176146       Profile c-hat=1.0000       
   22:S                     0.5614463       0.0559738       0.4573229       0.6724149       Profile c-hat=1.0000       
   23:S                     0.5120654       0.0556664       0.4105786       0.6270169       Profile c-hat=1.0000       
   24:S                     0.4624480       0.0564830       0.3609972       0.5826686       Profile c-hat=1.0000       
   25:S                     0.4135633       0.0582287       0.3106001       0.5403518       Profile c-hat=1.0000       
   26:S                     0.3663247       0.0603937       0.2617290       0.5007203       Profile c-hat=1.0000       
   27:S                     0.3215234       0.0623774       0.2164036       0.4640376       Profile c-hat=1.0000       
   28:S                     0.2797822       0.0636788       0.1759763       0.4301850       Profile c-hat=1.0000       
   29:S                     0.2415309       0.0639865       0.1410853       0.3988211       Profile c-hat=1.0000       
   30:S                     0.2070061       0.0631856       0.1117793       0.3695837       Profile c-hat=1.0000       

The difference between the 2 sets of intervals is basically negligible when c-hat = 1.  

To view the profile likelihood confidence intervals when c-hat is not equal to 1, the user has to open up the full output file for a model.  Profile likelihood confidence intervals are displayed in the real parameters window only when c-hat = 1.  Because the user can change the value of c-hat, and profile likelihood confidence intervals are only computed when the log-likelihood for the model is optimized, the profile likelihood values reported in the full output are defined by the c-hat in use at the time.  If the user changes the value of c-hat, the profile intervals in the output will not change.  Thus, to avoid potential mistakes, the value of c-hat used to compute the interval is reported beside the interval.  

Warning: The approach described above does not always work correctly because of numerical problems, notably for parameters estimated on the boundary.  I have found that often the optimization routine is not always able to move the parameter estimate away from the boundary.  As an example, consider the 3rd real parameter in the {phi(g*t) p(g*t)} model of the AFS Monograph example distributed with Program MARK and described by Burnham et al. (1987).  The following results are for the first version of the profile likelihood confidence interval code.  Since that version, I have modified the code to provide better results, but I am not confident that the code will always produce correct results.  Thus, if you should check the profile likelihood confidence interval estimates to be sure that proper optimization is achieved.  

With the identity link, the following estimate and default confidence interval is obtained:

                 Real Function Parameters of {Phi(g*t) p(g*t) PIM coding}
                                                              95% Confidence Interval
Parameter                  Estimate       Standard Error      Lower           Upper
————————-  ————–  ————–  ————–  ————–
    3:Phi                   1.0734483       0.2038269       0.6739477       1.4729490     

Note that the confidence interval is computed as the estimate +/- 1,96 times the standard error in the above output, because the real parameter estimate is outside the 0-1 interval.  Had the estimate been in the 0-1 interval, the confidence interval would have been computed on the logit scale, and then back-transformed.  When a profile likelihood confidence interval is requested with the identity link, the following estimates are obtained:

    3:Phi                   1.0734483       0.2038269       0.7269042       1.5390461       Profile c-hat=1.0000      

and are not particularly different from the default values shown above.  Next, consider the estimate for parameter 3 when the logit link is used to constrain the estimate to the 0-1 interval, and the following default output is obtained.

    3:Phi                   0.9999728       0.0032660       0.5107488E-06   1.0000000                     

The confidence interval is nonsensical because the standard error is estimated as zero, and so the confidence interval extends from 0 to 1.  Now, when a profile likelihood interval is requested, the following results are obtained:

    3:Phi                   0.9999728       0.0032660       0.9999728       0.9999728       Profile c-hat=1.0000       

Note that the optimization procedure was not able to move the parameter estimate away from the boundary, so the profile interval is not correct.  Similar results are obtained if the sin link is used:

    3:Phi                   1.0000000       0.1856222E-03   1.0000000       1.0000000       Profile c-hat=1.0000       

Interestingly enough, when the confounded parameters phi5 and phi10 are fixed at one, a correct profile likelihood interval is obtained with the sin link:

    3:Phi                   0.9999997       0.6026331E-03   0.7725536       0.9999997       Profile c-hat=1.0000       

Evidently fixing these confounded parameters makes the optimization more stable, and proper convergence is obtained.

In this particular case, because the parameter at the boundary is a function of a single beta (i.e., the parameter is not a function of multiple beta values because an identity link is being used), you can compute the profile likelihood parameter manually by fixing the parameter to your guess of the profile likelihood interval endpoint.  Thus, run the {phi(g*t} p(g*t)} model with phi3 fixed to 0.7725536, and check that the deviance of this new model is approximately 3.84 units larger than the deviance of the model with none of the unconfounded parameters fixed.

As noted above, the code has been modified to provide correct estimates for this example.  However, be suspicious!

In summary, care should be taken in using the profile likelihood capability because of the numerical problems caused with parameters at the boundary.  Unfortunately, parameters estimated at the boundary is the primary reason that the prof