On the efficacy of restoration in stream networks: comment and critique

2 Abstract 14 Swan and Brown (2017) recently addressed the effects of restoration on stream 15 communities under the meta-community framework. Using a combination of headwater and 16 mainstem streams, Swan and Brown (2017) evaluated how position within a stream network 17 affected the outcome of restoration on invertebrate communities. Ostensibly, their hypotheses 18 were partially supported as restoration had stronger effects in headwater streams: invertebrate 19 taxonomic richness was increased and temporal variability decreased in restored reaches; 20 however, these results were not consistent upon closer scrutiny for both the original paper (Swan 21 and Brown 2017) and the later erratum (Swan and Brown 2018). This is due to issues with 22 experimental design, improper use of statistical analyses, and discrepancies between written 23 methods and what was actually conducted. Here, I provide a secondary analysis of the data, with 24 hypotheses and interpretations in the context of stream, restoration, and metacommunity ecology.


Introduction 29
In a recent study, Swan and Brown (2017)  . CC-BY-NC-ND 4.0 International license It is made available under a (which was not peer-reviewed) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint . http://dx.doi.org/10.1101/611939 doi: bioRxiv preprint first posted online Apr. 18, 2019; expected to have a greater impact on communities in headwaters relative to mainstems (Swan 37 and Brown 2017). 38 Although Brown (2017, 2018) noted that restoration techniques can vary in 39 intrusiveness on stream ecosystems, they did not account for this in their experimental design 40 and statistical analyses. Restored streams in their study received various combinations of bank 41 stabilization, in-channel manipulation, and riparian reforestation (i.e. tree planting) treatments, 42 and these treatments were not applied in a consistent or systematic manner (Swan and Brown 43 2017: Table 2). Swan and Brown (2017) did not set a restoration criterion for site inclusion in 44 their study, instead including all sites regardless of the combination of applied restoration 45 treatments. I suggest that this oversight leads to unnecessary assumptions about the efficacy of 46 restoration by assuming the effects of all treatments combinations are equivalent, and this issue 47 could have been partially resolved a priori by hypothesizing how each restoration reach would 48 affect headwater and mainstem streams and then setting requirements for site inclusion in the 49 analyses. 50 I contend that the various restoration treatments differ not only in their overall effects but 51 also if the treatment is applied in headwater or mainstem streams, and, for these reasons, criteria 52 for site selection could be set. I suggest that bank stabilization and in-channel manipulation 53 treatments are more likely to have consistent effects in both headwaters and mainstems, while 54 riparian reforestation would likely have stronger effects in headwater compared to mainstem 55 streams. A similar argument was made by Swan and Brown, though it was not explicitly noted 56 until the erratum (Swan and Brown 2018). Bank stabilization and in-channel manipulation can 57 increase bed stability and substrate availability and diversity in both headwater and mainstem 58 streams; however, the effects of riparian reforestation could act on a gradient from headwaters to 59 . CC-BY-NC-ND 4.0 International license It is made available under a (which was not peer-reviewed) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint . http://dx.doi.org/10.1101/611939 doi: bioRxiv preprint first posted online Apr. 18, 2019; Shannon's diversity) and compared using an analysis of variance (ANOVA). The model was 83 constructed to examine the individual effects of reach (restored or adjacent), order (headwater or 84 mainstem), and season (spring, summer, fall, and winter) and all two-and three-way interactions, 85 with individual ANOVAs for richness and diversity; I also fit the full and reduced taxonomic 86 richness models proposed in the erratum ( Temporal variability was then compared using an ANOVA with the individual effects of order 96 and reach and their interaction. All ANOVAs were performed for both the full and revised sites, 97 with stream identity fitted as a random blocking factor in each ANOVA; all ANOVAs were 98 fitted by restricted maximum likelihood. 99 Exploratory data analysis was conducted prior to any model fitting to determine if the 100 data met test assumptions (Zuur et al. 2009). For the full sites analyses, numerical summaries 101 demonstrated an unbalanced design, with equal representation of restored and unrestored reaches 102 but a large disparity in the number of samples between headwaters and mainstems for each of the 103 taxonomic richness and diversity (headwater n = 38, mainstem n = 62), spatial dissimilarity 104 (headwater n = 19, mainstem = 31), and temporal variation (headwater n = 10, mainstem n = 16) 105 . CC-BY-NC-ND 4.0 International license It is made available under a (which was not peer-reviewed) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint . http://dx.doi.org/10.1101/611939 doi: bioRxiv preprint first posted online Apr. 18, 2019; 6 analyses. Additionally, the assumption of homogeneity of variance was violated for the 106 taxonomic richness and diversity and the spatial dissimilarity analyses. The unbalanced design 107 was greatly reduced for the revised sites analyses: taxonomic richness and diversity (headwater n 108 = 30, mainstem n = 40), spatial dissimilarity (headwater n = 15, mainstem = 20), and temporal 109 variation (headwater n = 8, mainstem n = 10); however, the assumption of homogeneity of 110 variance was still violated. To better meet the assumption of equal variance, taxonomic richness 111 was ln-transformed, taxonomic diversity was square root-transformed, and spatial dissimilarity 112 . CC-BY-NC-ND 4.0 International license It is made available under a (which was not peer-reviewed) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity.

Results and Discussion 129
There were no main or interactive effects of season, order, or reach on taxonomic 130 richness for either the full or revised sites analyses (Table 1, Figure 1). The full model of 131 taxonomic richness proposed in the erratum (Swan and Brown 2018) did not show any main or 132 interactive effects of season, order, or reach (Table 2); however, the reduced model of taxonomic 133 richness demonstrated an interaction between order and season (F 3, 80 = 4.105, P = 0.009) and 134 significant main effects of season (F 3, 80 = 4.358, P = 0.007) and treatment (F 1, 80 = 4.844, P = 135 0.031). In contrast to taxonomic richness, taxonomic diversity varied by season for the full (F 3, 80 136 = 12.267, P < 0.001) and revised (F 3, 80 = 10.999, P < 0.001) sites (Table 1, Figure 2). There 137 were no further main or interactive effects of season, order, or reach on taxonomic diversity for 138 either the full or revised sites (Table 1, Figure 2). Spatial dissimilarity did not vary by any of the 139 main or interactive effects of season and order for both the full and revised sites (Table 1, Figure  140 3). Additionally, temporal variation did not vary by the main effects of or interaction between 141 reach and order for the full and revised sites (Table 1, Figure 4). 142 Differences in significant main effects or interactions within the full and revised sites in 143 the re-analysis did not seem to be the result of increased variation in the revised sites. In fact, 144 variance, as measured by 95% confidence intervals, was either similar or even reduced for each 145 of local diversity, spatial dissimilarity, and temporal variability for the revised sites compared to 146 the full sites (Figures 1-4). It is therefore unlikely that that revised sites analysis was unable to 147 detect effects due to increased variation and more likely due to reduced statistical power 148 associated with a smaller sample size or the true lack of an effect of restoration in this system. 149

Effectiveness of Local Restoration 150
. CC-BY-NC-ND 4.0 International license It is made available under a (which was not peer-reviewed) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint . http://dx.doi.org/10.1101/611939 doi: bioRxiv preprint first posted online Apr. 18, 2019; I hypothesized that stream-channel manipulations would have a more consistent effect 151 between headwaters and mainstems relative to the effects of riparian reforestation, with stronger 152 effects of restoration in headwaters relative to mainstems. As there were no significant effects of 153 restoration on any of the community metrics between headwaters and mainstems, this hypothesis 154 could be invalid or, at a minimum, revised and re-tested. I should note that I was unable to 155 directly test this hypothesis because I was re-analyzing data from a previous study and the 156 experimental design precluded any robust test to isolate the effects; however, the hypothesis was limitation of their study, they failed to acknowledge they could have better controlled for this 173 . CC-BY-NC-ND 4.0 International license It is made available under a (which was not peer-reviewed) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint . http://dx.doi.org/10.1101/611939 doi: bioRxiv preprint first posted online Apr. 18, 2019; variation by setting strict criteria for site selection and inclusion, which was the foundation for 174 my re-analysis. This concern was briefly acknowledged in the erratum (

Contradicting Results and Questionable Research Practices 194
Restoration was not found to have a significant effect on local diversity, spatial 195 dissimilarity, or temporal variability of stream invertebrate communities between paired restored 196 . CC-BY-NC-ND 4.0 International license It is made available under a (which was not peer-reviewed) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. . CC-BY-NC-ND 4.0 International license It is made available under a (which was not peer-reviewed) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint . http://dx.doi.org/10.1101/611939 doi: bioRxiv preprint first posted online Apr. 18, 2019; Finally, and of greatest concern, is the wholesale disagreement between the reported 220 analytical procedure and what was actually conducted when analyzing temporal variability. 221 Temporal variability was reportedly quantified as the mean distance to the group centroid 222

Concluding Remarks 239
I intended to re-analyze the data provided by Brown (2017, 2018) to determine 240 if the same patterns were observed if only sites receiving at least both intensive, stream-channel 241 restoration treatments were included. I then evaluated the same hypotheses as proposed by Swan 242 . CC-BY-NC-ND 4.0 International license It is made available under a (which was not peer-reviewed) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint . http://dx.doi.org/10.1101/611939 doi: bioRxiv preprint first posted online Apr. 18, 2019; and Brown (2017), as the hypothesized mechanisms were consistent with metacommunity 243 theory. I did not find any effects of restoration on local diversity, spatial dissimilarity, and 244 temporal variability, let alone differential effects between headwaters and mainstems. This is 245 contrary to results presented by Brown (2017, 2018), who reported that restoration 246 increased taxonomic richness and decreased temporal variability in restored headwater streams. I 247 have demonstrated that the statistical tests from which those results were derived were invalid, 248 and, therefore, recommend use of the results presented here. I also have some evidence 249 demonstrating questionable research practices conducted by Brown (2017, 2018), 250 whereby written methods were not properly followed when analyzing the data. I have concluded 251 that, given restoration had no effect on any community diversity metric, local restoration of . CC-BY-NC-ND 4.0 International license It is made available under a (which was not peer-reviewed) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity.

326
. CC-BY-NC-ND 4.0 International license It is made available under a (which was not peer-reviewed) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint . http://dx.doi.org/10.1101/611939 doi: bioRxiv preprint first posted online Apr. 18, 2019; Note: N/A indicates a factor or interaction that was removed in the reduced model. 331 . CC-BY-NC-ND 4.0 International license It is made available under a (which was not peer-reviewed) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint . http://dx.doi.org/10.1101/611939 doi: bioRxiv preprint first posted online Apr. 18, 2019; . CC-BY-NC-ND 4.0 International license It is made available under a (which was not peer-reviewed) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity.