PT  - JOURNAL ARTICLE
AU  - Oberpriller, Johannes
AU  - de Souza Leite, Melina
AU  - Pichler, Maximilian
TI  - Fixed or random? On the reliability of mixed-effect models for a small number of levels in grouping variables
AID  - 10.1101/2021.05.03.442487
DP  - 2021 Jan 01
TA  - bioRxiv
PG  - 2021.05.03.442487
4099  - http://biorxiv.org/content/early/2021/05/04/2021.05.03.442487.short
4100  - http://biorxiv.org/content/early/2021/05/04/2021.05.03.442487.full
AB  - Biological data are often intrinsically hierarchical. Due to their ability to account for such dependencies, mixed-effect models have become a common analysis technique in ecology and evolution. While many questions around their theoretical foundations and practical applications are solved, one fundamental question is still highly debated: When having a low number of levels should we model a grouping variable as a random or fixed effect? In such situation, the variance of the random effect is presumably underestimated, but whether this affects the statistical properties of the fixed effects is unclear.Here, we analyze the consequences of including a grouping variable as fixed or random effect and possible other modeling options (over and underspecified models) for data with small number of levels in the grouping variable (2 - 8). For all models, we calculated type I error rates, power and coverage. Moreover, we show the influence of possible study designs on these statistical properties.We found that mixed-effect models already for two groups correctly estimate variance for two groups. Moreover, model choice does not influence the statistical properties when there is no random slope in the data-generating process. However, if an ecological effect differs among groups, using a random slope and intercept model, and switching to a fixed-effect model only in case of a singular fit avoids overconfidence in the results. Additionally, power and type I error are strongly influenced by the number of and the difference between groups.We conclude that inferring the correct random effect structure is of high importance to get correct statistical properties. When in doubt, we recommend starting with the simpler model and using model diagnostics to identify missing components. When having identified the correct structure, we encourage to start with a mixed-effects model independent of the number of groups and only in case of a singular fit switch to a fixed-effect model. With these recommendations, we allow for more informative choices about study design and data analysis and thus make ecological inference with mixed-effects models more robust for low number of groups.Competing Interest StatementThe authors have declared no competing interest.