Existing caribou habitat and demographic models are poorly suited for Ring of Fire impact assessment: A roadmap for improving the usefulness, transparency, and availability of models for conservation

Environmental impact assessments often rely on best available information, which may include models that were not designed for purpose and are not accompanied by an assessment of limitations. We reproduced available models of boreal woodland caribou resource selection and demography and evaluated their suitability for projecting impacts of development in the Ring of Fire (RoF) on boreal caribou in the Missisa range (Ontario, Canada). The specificity of the resource selection model limited usefulness for predicting impacts, and high variability in model coefficients among ranges suggests responses vary with habitat availability. The aspatial demographic model projects decreasing survival and recruitment with increasing disturbance, but high variability among populations implies the importance of these impacts depends on the status of the Missisa population, and there no current status estimate. New models that are designed for forecasting the cumulative effects of development and climate change are required to better inform RoF decisions. To demonstrate how open-source tools and reproducible workflows can improve the transparency and reusability of models we developed an R package for data preparation, resource selection, and demographic calculations. Open-source tools, reproducible workflows, and reuseable forecasting models can improve our collective ability to inform wildlife management decisions in a timely manner.

.2, S1.3). To reproduce the original RSFs, we acquired data sets for the 148 predictor variables that were publicly available and, where possible, used information on the 149 timing of the event or the feature's construction to recreate the data that was available when the 150 original model was produced (Table S1.2). We used current versions of these same datasets to 151 project the RSFs on the landscape. 152

RSF Model Validation 153
To assess our ability to reproduce the original model, we acquired the original projected 154 surfaces for comparison from the authors (R. Rempel pers. comm, 2021). We validated our 155 models visually and quantitatively. The original model used a nested hexagon grid to generate 156 predictions. We opted to approximate this approach with distance-weighted moving windows on 157 a rectangular grid, which is easier to implement using standard raster-processing tools. To 158 compare these approaches, we extracted the values from our rectangular grid to the hexagonal 159 grid using the weighted mean. We mapped the difference in predictions and explanatory 160 variables between the models and used scatterplots to compare the value of the model 161 response for each grid cell produced by the two models. A perfect reproduction would produce a 162 Pearson correlation coefficient of 1, and any deviation from the original prediction would reduce 163 this coefficient. We expected some differences in the predictions as a result of the type of grid 164 and predictor data sets used. 165

RSF Model Projection Under Disturbance Scenarios 166
The RSF includes road density as a predictor and would require additional information 167 on roads within mining areas that we do not have; for the RSF we only compare the base and 168 road-only scenarios. To understand how the 'roads only' scenario would affect the caribou RSF, 169 we projected the original RSFs across updated landscape conditions as represented by i) 170 temporal changes in forest structure between 2010 and 2020 (e.g., fires; Table S1.2) and ii) the 171 new proposed roads only scenario (Fig. 2). We also assessed the potential for borrowing 172 information from other ranges by using the coefficients from the top original RSFs from the 173 adjacent James Bay, Nipigon, and Pagwachuan ranges transferred to Missisa. We visualized 174 the spatial transferability of the models and examined their sensitivity to changing habitat 175 availability using scatterplots to compare the value of the model response for each grid cell 176 produced by the Missisa model and each respective adjacent range. 177

Demographic Model 178
Canada's national demographic boreal caribou models were developed from adult 179 female survival (S), calf recruitment (R), and landscape data across 58 boreal caribou study 180 areas, including 13 study areas in Ontario (Johnson et al., 2020; Table S1). We used these 181 models (Johnson et al., 2020) to predict changes in S and R; in the roads-only and roads-and-182 mines development scenarios. The demographic model is aspatial and all types of 183 anthropogenic disturbances are combined into a single measure of disturbed area, so the 184 'roads-and-mines' footprint is sufficient and there is no need to specify the location of roads 185 within mining claim areas. We calculated the relevant predictor variables for the Missisa range 186 (e.g., % anthropogenic disturbance buffered by 500 m; % wildfire within the last 40 years; Table  187 1), and calculated expected R and S as a function of disturbance according to the beta 188 regression models with highest support (

RSF Model Reproduction and Validation 238
We were able to produce a reasonably accurate representation of the original RSFs for 239 the Missisa range, as evidenced by high Pearson's r values across all seasons (r > 0.935; Fig.  240 S1.1). Maps highlighting the differences between predictions and some variation in the input 241 data layers are provided in Supplementary Material ( Fig. S1.2, S1.3). In the Missisa range, the 242 model predicted the highest relative use probabilities in the northwest of the study area during 243 winter (Fig. 3), consistent with the original RSFs. During the summer and spring, the eastern 244 portion of the range was used more than the northwest (Fig. 3). 245

RSF Model Projection 246
We observed a lower relative probability of use in areas associated with proposed roads 247 (Fig. 4). Lower relative probability of use along the road corridors was strongest in the spring 248 and summer, consistent with seasonal changes in the response to roads described in the 249 original RSFs (Fig. 4). There was high variability in the estimated response to roads among 250 ranges (Fig. 4). The James Bay range prediction appeared the most similar to the Missisa range 251 prediction (Fig. 4); however it varied by season (Fig S1.4). The Nipigon and Pagwachuan range 252 projections, which visually differed substantially from the Missisa projection, did not show a 253 strong response to proposed roads ( Fig. 4; Fig S1.4).  (Table 1), and the corresponding range of variability in 258 demographic rates among sample populations is high (Fig. 5). The model predicts increasing 259 anthropogenic disturbance will decrease both survival and recruitment (Fig. 5), but the 260 importance of that decrease for self-sustainability of the population is highly uncertain and 261 depends on initial population status (Fig. 5). A 2014 assessment informed by data from 2008-262 2012 indicated lower than expected recruitment, survival, and population growth rate in the 263 Missisa range (diamonds in Fig. 5; MNRF, 2014a); however, we lack recent information on the 264 survival, recruitment and status of this population to project the impacts of disturbance. 265

Discussion 266
We examined the applicability of two existing caribou models for projecting impacts of 267 development in the RoF on boreal caribou in the Missisa range. We highlight limitations of 268 existing tools and point out possible solutions. We found that the original RSFs are poorly suited 269 for projecting the impacts of development in the RoF because they are specific to current 270 conditions. In contrast to the RSFs, the existing aspatial demographic model is too general to 271 project the impacts of development on this particular population. Ideally, forecasting models 272 used to inform environmental decisions should be designed for the purpose, identify and 273 account for uncertainty, be updateable with new information, and be transparent and 274 reproducible (Bodner et al., 2021a). However, in practice, environmental impact assessments 275 often rely on best available information, which may include tools and models that were not 276 designed for the purpose. We highlight limitations of existing tools and point out possible 277

solutions. 278
The RSF models for each caribou range were fit independently using data from that 279 range. This is a reasonable approach when the objective is to characterize current habitat use, 280 but not for projecting responses to changing landscape conditions. There has been very little (2020). We also opted to use a different method for ensuring whole numbers of animals, which 321 lead to different estimates of population growth rate for small populations (see Supplementary  322 Material Part 2). We assumed that demographic rates (i.e., recruitment and survival) vary with 323 disturbance according to the best supported national models (Table 2 of Johnson et al., 2020) 324 but note that other competing models in that candidate set were nearly as well supported by the 325 data. This simple population model also assumed no variation in recruitment or survival with age 326 or other parameters (Supplementary Material Part 2). Most female caribou reproduced at 2-3 327 years of age rather than at 1 year, as assumed in the model. This likely results in overestimating 328 demographic parameter values under changing conditions. A more thorough investigation of the 329 sensitivity of results, parameters, and representation of uncertainty and stochasticity in aspatial 330 caribou demographic models is beyond the scope of this paper but seems warranted. 331 We lack region-specific information on disturbance history, vegetation, and vegetation is not always an easily usable tool. In this project, we were able to reproduce existing models 375 because the developers of those models were willing to share their code. Code does not have 376 to be flawless to enable others to build on previous work, and there are many ways that 377 researchers can shift to more open and reproducible workflows that reduce the chance of 378 errors, increase efficiency, improve reproducibility, and increase our ability to generalize across 379 studies (Alston and Rick, 2021; Lewis et al., 2018). We are hopeful that improving the 380 transparency, reproducibility and decision-relevance of wildlife response models will improve 381 our collective ability to inform decisions for SAR and improve conservation outcomes.    Bay, Nipigon, and Pagwachuan range to estimate the relative probability of use (0-1) by boreal 740 caribou during spring, summer, fall, and winter. Scale ranges from dark blue to yellow with 741 yellow representing a higher relative probability of use. 742