Abstract
One challenge an analyst often encounters when dealing with complex mark–recapture models is how to limit the number of a priori models. While all possible combinations of model structures on the different parameters (e.g., ϕ, p) can be considered, such a strategy often results in a burdensome number of models, leading to the use of ad hoc strategies to reduce the number of models constructed. For the Cormack–Jolly–Seber data type, one example of an ad hoc strategy is to hold a general ϕ model structure constant while investigating model structures on p, and then to hold the resulting best structure on p constant and investigate structures on ϕ. Many comparable strategies exist. The effect of following ad hoc strategies on parameter estimates as well as for variable selection and whether model averaging can ameliorate any problems are unknown. By means of a simulation study, we have investigated this informational gap by comparing the all-combinations model building strategy with two ad hoc strategies and with truth, as well as considering the results of model averaging. We found that model selection strategy had little effect on parameter estimator bias and precision and that model averaging did improve bias and precision slightly. In terms of variable selection (i.e., cumulative Akaike’s information criterion weights), model sets based on ad hoc strategies did not perform as well as those based on all combinations, as less important variables often had higher weights with the former than with the all possible combinations strategy. Increased sample size resulted in increased variable weights, with an infinite sample size resulting in all variable weights equaling 1 for variables with any predictive influence. Thus, the distinction between statistical importance (dependent on sample size) and biological importance must be recognized when utilizing cumulative weights. We recommend that all-combinations model strategy and model averaging be used. However, if an ad hoc strategy is relied upon to reduce the computational demand, parameter estimates will generally be comparable to the all-combinations strategy, but variable weights will not correspond to the all-combinations strategy.
Similar content being viewed by others
References
Anderson DR (2008) Model based inference in the life sciences—a primer on evidence. Springer, New York
Barbieri MM, Berger JO (2004) Optimal predictive model selection. Ann Stat 32:870–897
Barker RJ (1997) Joint modeling of live-recapture, tag-resight, and tag-recovery data. Biometrics 53:666–677
Burnham KP, Anderson DR (2002) Model selection and multimodel inference—a practical information-theoretic approach, 2nd edn. Springer, New York
Burnham KP, Anderson DR (2004) Multimodel inference—understanding AIC and BIC in model selection. Sociol Methods Res 33:261–304
Doherty PF, Nichols JD, Tautin J, Voelzer JF, Smith GW, Benning DS, Bentley VR, Bidwell JK, Bollinger KS, Brazda AR, Buelna EK, Goldsberry JR, King RJ, Roetker FH, Solberg JW, Thorpe PP, Wortham JS (2002) Sources of variation in breeding-ground fidelity of mallards (Anas platyrhynchos). Behav Ecol 13:543–550
Dugger KM, Ballard G, Ainley DG, Barton KJ (2006) Effects of flipper bands on foraging behavior and survival of Adelie Penguins (Pygoscelis adeliae). Auk 123:858–869
Lebreton JD, Burnham KP, Clobert J, Anderson DR (1992) Modeling survival and testing biological hypotheses using marked animals—a unified approach with case-studies. Ecol Monogr 62:67–118
Lebreton JD, Nichols JD, Barker RJ, Pradel R, Spendelow JA (2009) Modeling individual animal histories with multistate capture-recapture models. Adv Ecol Res 41:88–173
Link WA, Barker RJ (2006) Model weights and the foundations of multimodel inference. Ecology 87:2626–2635
Lukacs PM, Burnham KP, Anderson DR (2010) Model selection bias and Freedman’s paradox. Ann Inst Stat Math 62:117–125
Murray K, Conner MM (2009) Methods to quantify variable importance: implications for the analysis of noisy ecological data. Ecology 90:348–355
Sandercock BK, Jaramillo A (2002) Annual survival rates of wintering sparrows: assessing demographic consequences of migration. Auk 119:149–165
White GC (2008) Closed population estimation models and their extensions in program MARK. Environ Ecol Stat 15:89–99
Acknowledgments
We thank Evan Cooch, David Anderson, Kate Huyvaert, and Larissa Bailey for interesting discussions and two reviewers for valuable comments.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by M. Schaub.
Rights and permissions
About this article
Cite this article
Doherty, P.F., White, G.C. & Burnham, K.P. Comparison of model building and selection strategies. J Ornithol 152 (Suppl 2), 317–323 (2012). https://doi.org/10.1007/s10336-010-0598-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10336-010-0598-5