Profile Likelihood Confidence Intervals
Profile Likelihood Confidence Intervals
The Run Window includes a check box to request profile likelihood confidence intervals be computed for the real parameter estimates. The default confidence intervals for real parameter estimates in the 0-1 interval are based on the standard error and the logit transformation. That is, a 95% confidence interval is computed on the logit estimate, and then these intervals are transformed to the real scale. Use of the logit transformation precludes confidence interval boundaries outside the 0-1 interval. However, problems are encountered with this approach when the parameter estimate is on the boundary, e.g., when a survival rate is estimated as 1, and/or the standard error is estimated as zero.
Profile likelihood confidence intervals are based on the log-likelihood function. For a single parameter, likelihood theory shows that the 2 points 1.92 units down from the maximum of the log-likelihood function provide a 95% confidence interval when there is no extrabinomial variation (i.e. c = 1).. The value 1.92 is half of the chi-square value of 3.84 with 1 degree of freedom. Thus, the same confidence interval can be computed with the deviance by adding 3.84 to the minimum of the deviance function, where the deviance is the log-likelihood multiplied by -2 minus the -2 log likelihood value of the saturated model.
MARK will compute profile likelihood confidence intervals with extrabinomial variation assumed, i.e., c > 1. To do so, the amount dropped below the maximum of the likelihood is c-hat*1.92, or the amount to increase above the minimum deviance is c-hat*3.84.
The method used in MARK to compute the profile likelihood lower bound for parameter i is to minimize the following function:
[-2 log likelihood of current parameter value – (-2 log likelihood of maximum likelihood estimates + c-hat*3.84) ]**2 – (maximum likelihood estimate of parameter i – current parameter value of i).
The first portion of this expression finds the value of the deviance that is c-hat*3.84 units larger than the deviance for the maximum likelihood parameter estimates. The second portion of the expression maximizes the difference between the maximum likelihood parameter estimate and the lower bound.
For the upper bound, the function has to be modified maximize the difference between the maximum likelihood estimate and the upper bound:
[-2 log likelihood of current parameter value – (-2 log likelihood of maximum likelihood estimates + c-hat*3.84) ]**2 – (current parameter value of i – maximum likelihood estimate of parameter i).
This procedure appears to be different from the typical approach to estimate profile likelihood intervals because in MARK, the parameters that are optimized are the beta estimates, but we are interested in the profile intervals for the real parameters. Thus, there is not always a 1 to 1 conversion from the beta estimates to the real estimates, e.g., trend models. The approach described above accommodates this additional complexity.
Because of the additional optimization required to obtain the profile likelihood interval estimates, considerably more computer time is required to compute these intervals.
The following is an example comparing the default confidence intervals with the profile likelihood confidence intervals. The example is from the North Park sage grouse data distributed with Program MARK in the NPMALES.DBF file. First is the default output for the 30 survival estimates from the {S(a*T) r(a*T)} model.
North Park Male Sage Grouse Band Recovery Data
Real Function Parameters of {S(a*T) r(a*T)}
95% Confidence Interval
Parameter Estimate Standard Error Lower Upper
————————- ————– ————– ————– ————–
1:S 0.5212251 0.0463568 0.4306641 0.6104122
2:S 0.5017107 0.0406542 0.4226535 0.5806825
3:S 0.4821911 0.0351372 0.4140880 0.5509623
4:S 0.4627257 0.0300246 0.4046607 0.5218196
5:S 0.4433733 0.0256337 0.3938728 0.4940243
6:S 0.4241912 0.0224038 0.3809848 0.4685882
7:S 0.4052348 0.0208173 0.3651713 0.4466016
8:S 0.3865568 0.0211150 0.3460756 0.4286692
9:S 0.3682068 0.0230520 0.3242870 0.4144267
10:S 0.3502306 0.0260782 0.3009809 0.4028942
11:S 0.3326698 0.0296737 0.2772552 0.3931355
12:S 0.3155620 0.0334756 0.2538794 0.3845078
13:S 0.2989398 0.0372553 0.2313377 0.3766175
14:S 0.2828314 0.0408711 0.2099224 0.3692287
15:S 0.2672600 0.0442350 0.1898019 0.3621974
16:S 0.8083942 0.0596942 0.6646855 0.8997973
17:S 0.7757114 0.0608475 0.6353812 0.8728417
18:S 0.7392520 0.0609946 0.6039334 0.8405448
19:S 0.6991644 0.0602202 0.5700751 0.8028948
20:S 0.6557833 0.0587967 0.5334585 0.7604383
21:S 0.6096393 0.0571881 0.4936989 0.7143893
22:S 0.5614463 0.0559738 0.4505298 0.6665446
23:S 0.5120654 0.0556664 0.4041007 0.6189161
24:S 0.4624480 0.0564830 0.3552982 0.5731824
25:S 0.4135633 0.0582287 0.3057997 0.5302933
26:S 0.3663247 0.0603937 0.2577011 0.4904807
27:S 0.3215234 0.0623774 0.2129511 0.4535516
28:S 0.2797822 0.0636788 0.1729395 0.4191781
29:S 0.2415309 0.0639865 0.1383693 0.3870549
30:S 0.2070061 0.0631856 0.1093399 0.3569485
Next is the output for the same model, but with profile likelihood intervals computed for the 30 survival estimates. The value of c-hat used to compute the profile likelihood confidence intervals is reported in the output,
North Park Male Sage Grouse Band Recovery Data
Real Function Parameters of {S(a*T) r(a*T)}
95% Confidence Interval
Parameter Estimate Standard Error Lower Upper
————————- ————– ————– ————– ————–
1:S 0.5212251 0.0463568 0.4305389 0.6107000 Profile c-hat=1.0000
2:S 0.5017107 0.0406542 0.4226006 0.5809796 Profile c-hat=1.0000
3:S 0.4821911 0.0351372 0.4140988 0.5512675 Profile c-hat=1.0000
4:S 0.4627257 0.0300246 0.4047206 0.5221246 Profile c-hat=1.0000
5:S 0.4433733 0.0256337 0.3939565 0.4943047 Profile c-hat=1.0000
6:S 0.4241912 0.0224038 0.3810508 0.4687985 Profile c-hat=1.0000
7:S 0.4052348 0.0208173 0.3651681 0.4467037 Profile c-hat=1.0000
8:S 0.3865568 0.0211150 0.3459763 0.4286955 Profile c-hat=1.0000
9:S 0.3682068 0.0230520 0.3241126 0.4144641 Profile c-hat=1.0000
10:S 0.3502306 0.0260782 0.3007706 0.4030047 Profile c-hat=1.0000
11:S 0.3326698 0.0296737 0.2770367 0.3933405 Profile c-hat=1.0000
12:S 0.3155620 0.0334756 0.2536662 0.3848095 Profile c-hat=1.0000
13:S 0.2989398 0.0372553 0.2311351 0.3770121 Profile c-hat=1.0000
14:S 0.2828314 0.0408711 0.2097321 0.3697119 Profile c-hat=1.0000
15:S 0.2672600 0.0442350 0.1896241 0.3627656 Profile c-hat=1.0000
16:S 0.8083942 0.0596942 0.6637455 0.8966843 Profile c-hat=1.0000
17:S 0.7757114 0.0608475 0.6360563 0.8699084 Profile c-hat=1.0000
18:S 0.7392520 0.0609946 0.6062872 0.8382203 Profile c-hat=1.0000
19:S 0.6991644 0.0602202 0.5740674 0.8017697 Profile c-hat=1.0000
20:S 0.6557833 0.0587967 0.5388831 0.7611976 Profile c-hat=1.0000
21:S 0.6096393 0.0571881 0.5001267 0.7176146 Profile c-hat=1.0000
22:S 0.5614463 0.0559738 0.4573229 0.6724149 Profile c-hat=1.0000
23:S 0.5120654 0.0556664 0.4105786 0.6270169 Profile c-hat=1.0000
24:S 0.4624480 0.0564830 0.3609972 0.5826686 Profile c-hat=1.0000
25:S 0.4135633 0.0582287 0.3106001 0.5403518 Profile c-hat=1.0000
26:S 0.3663247 0.0603937 0.2617290 0.5007203 Profile c-hat=1.0000
27:S 0.3215234 0.0623774 0.2164036 0.4640376 Profile c-hat=1.0000
28:S 0.2797822 0.0636788 0.1759763 0.4301850 Profile c-hat=1.0000
29:S 0.2415309 0.0639865 0.1410853 0.3988211 Profile c-hat=1.0000
30:S 0.2070061 0.0631856 0.1117793 0.3695837 Profile c-hat=1.0000
The difference between the 2 sets of intervals is basically negligible when c-hat = 1.
To view the profile likelihood confidence intervals when c-hat is not equal to 1, the user has to open up the full output file for a model. Profile likelihood confidence intervals are displayed in the real parameters window only when c-hat = 1. Because the user can change the value of c-hat, and profile likelihood confidence intervals are only computed when the log-likelihood for the model is optimized, the profile likelihood values reported in the full output are defined by the c-hat in use at the time. If the user changes the value of c-hat, the profile intervals in the output will not change. Thus, to avoid potential mistakes, the value of c-hat used to compute the interval is reported beside the interval.
Warning: The approach described above does not always work correctly because of numerical problems, notably for parameters estimated on the boundary. I have found that often the optimization routine is not always able to move the parameter estimate away from the boundary. As an example, consider the 3rd real parameter in the {phi(g*t) p(g*t)} model of the AFS Monograph example distributed with Program MARK and described by Burnham et al. (1987). The following results are for the first version of the profile likelihood confidence interval code. Since that version, I have modified the code to provide better results, but I am not confident that the code will always produce correct results. Thus, if you should check the profile likelihood confidence interval estimates to be sure that proper optimization is achieved.
With the identity link, the following estimate and default confidence interval is obtained:
Real Function Parameters of {Phi(g*t) p(g*t) PIM coding}
95% Confidence Interval
Parameter Estimate Standard Error Lower Upper
————————- ————– ————– ————– ————–
3:Phi 1.0734483 0.2038269 0.6739477 1.4729490
Note that the confidence interval is computed as the estimate +/- 1,96 times the standard error in the above output, because the real parameter estimate is outside the 0-1 interval. Had the estimate been in the 0-1 interval, the confidence interval would have been computed on the logit scale, and then back-transformed. When a profile likelihood confidence interval is requested with the identity link, the following estimates are obtained:
3:Phi 1.0734483 0.2038269 0.7269042 1.5390461 Profile c-hat=1.0000
and are not particularly different from the default values shown above. Next, consider the estimate for parameter 3 when the logit link is used to constrain the estimate to the 0-1 interval, and the following default output is obtained.
3:Phi 0.9999728 0.0032660 0.5107488E-06 1.0000000
The confidence interval is nonsensical because the standard error is estimated as zero, and so the confidence interval extends from 0 to 1. Now, when a profile likelihood interval is requested, the following results are obtained:
3:Phi 0.9999728 0.0032660 0.9999728 0.9999728 Profile c-hat=1.0000
Note that the optimization procedure was not able to move the parameter estimate away from the boundary, so the profile interval is not correct. Similar results are obtained if the sin link is used:
3:Phi 1.0000000 0.1856222E-03 1.0000000 1.0000000 Profile c-hat=1.0000
Interestingly enough, when the confounded parameters phi5 and phi10 are fixed at one, a correct profile likelihood interval is obtained with the sin link:
3:Phi 0.9999997 0.6026331E-03 0.7725536 0.9999997 Profile c-hat=1.0000
Evidently fixing these confounded parameters makes the optimization more stable, and proper convergence is obtained.
In this particular case, because the parameter at the boundary is a function of a single beta (i.e., the parameter is not a function of multiple beta values because an identity link is being used), you can compute the profile likelihood parameter manually by fixing the parameter to your guess of the profile likelihood interval endpoint. Thus, run the {phi(g*t} p(g*t)} model with phi3 fixed to 0.7725536, and check that the deviance of this new model is approximately 3.84 units larger than the deviance of the model with none of the unconfounded parameters fixed.
As noted above, the code has been modified to provide correct estimates for this example. However, be suspicious!
In summary, care should be taken in using the profile likelihood capability because of the numerical problems caused with parameters at the boundary. Unfortunately, parameters estimated at the boundary is the primary reason that the prof