We earlier used this simulation-based approach to evaluate model fit for closed CMR models. Here I go through the steps of parametric bootstrap for a standard CJS data structure, typified by the Dipper example. The basic steps are as follows:
Fit a model to the field data and compute a fit statistic such as deviance/ deviance.df. Usually this will be the "global" model, e.g., for the Dipper example Phi(sex*time) p(sex*time)
Take the parameter estimates (estimates of Phi and p by time and group combinations) as given and use these to simulate a data set having the same structure. Since CJS models are conditional on releases each occasion this means we want to simulate data based on the identical number of occasion specific releases
Fit the same model to the simulated data and compute the chosed fit statistic
Repeat the simulation-estimation steps a large (e.g., 1000) number of times and compare the data-based fit statistic to the resulting mean and empirical distribution
If the data value is unusually large (> 97.5% quantile) take this as evidence of violation of model assumptions (overdispersion) and adjust AIC accordingly.
I wrote a series of functions that performs parametric bootstrapping for the Dipper example. For full disclosure, the core of this procedure is the cjs simulation code written by Mark Kery and Michael Schaub (Chapter 7). The code takes the results from the Dipper data and simulates replicated data sets with the same structure (same number of released per occasion for 7 occasions for 2 groups), estimates the CJS model for each set, computes deviance/df, and compares that result to the data result. For 100 simulations (more would be better, but would take a long time!) the results of simulations were
> sim.out
$c.hat.mean
[1] 3.019743
$c.hat.025
2.5%
2.106522
$c.hat.975
97.5%
3.977396
By comparision the data results was
> #compare to data c.hat
> global$results$deviance/global$results$deviance.df
[1] 3.761788
One could reasonably conclude from this that there is little evidence of overdispersion, since although the data value is greater than the simulated mean (3.76 vs. 3.01) it is still less than the upper 5% value of 3.977. However, a "conservative" suggestion is to take the ratio of the data value to the mean simulated value as a measure of oversdispersion.
> #modified c.hat
> data.c.hat/sim.out$c.hat.mean
[1] 1.245731
This can be used as a new c.hat value for quasilikelihood adjustment. RMark has a built in function that will automatically adjust AIC to produce QAIC for a specified list of models. Applying this to the early list of covariate models for Dipper we have:
> cov.models<-collect.models()
> adjust.chat(chat=1.245731,cov.models)
The modified QAIC table is here and the R script to conduct the bootstrap is here.