Applying And Promoting Open Science In Ecology - Surveyed Drivers And Challenges

Open Science (OS) comprises a variety of practices and principles that are broadly intended to improve the quality and transparency of research, and the concept is gaining traction. Since OS has multiple facets and still lacks a unifying definition, it may be interpreted quite differently among practitioners. Moreover, successfully implementing OS broadly throughout science requires a better understanding of the conditions that facilitate or hinder OS engagement, and in particular, how practitioners learn OS in the first place. We addressed these issues by surveying OS practitioners that attended a workshop hosted by the Living Norway Ecological Data Network in 2020. The survey contained scaled-response and open-ended questions, allowing for a mixed-methods approach. Out of 128 registered participants we obtained survey responses from 60 individuals. Responses indicated usage and sharing of data and code, as well as open access publications, as the OS aspects most frequently engaged with. Men and those affiliated with academic institutions reported more frequent engagement with OS than women and those with other affiliations. When it came to learning OS practices, only a minority of respondents reported having encountered OS in their own formal education. Consistent with this, a majority of respondents viewed OS as less important in their teaching than in their research and supervision. Even so, many of the respondents’ suggestions for what would help or hinder individual OS engagement included more knowledge, guidelines, resource availability and social and structural support; indicating that formal instruction can facilitate individual OS engagement. We suggest that the time is ripe to incorporate OS in teaching and learning, as this can yield substantial benefits to OS practitioners, student learning, and ultimately, the objectives advanced by the OS movement.

Introduction 1 trends in science and society broadly aiming to enhance openness about and free and 5 inclusive access to all aspects of science, driven mainly by networks of individual OS 6 practitioners [2,3]. Different facets of OS, each emerging from slightly different 7 assumptions and goals, focus on promoting diverse and equitable access and contribution 8 to knowledge; innovation and efficiency through collaboration; quality and credibility 9 through transparency; access to open research platforms, new and efficient tools and 10 services; and alternative metrics for assessing research contribution and impact [3,4]. 11 Together, the rapidly evolving principles and practices associated with OS are expected 12 to revolutionise how research is done and shared in the not-too-distant future [5]. 13 The transition towards OS has required the development of necessary infrastructure, 14 including platforms for collaboration and large-scale interactive databases, and is 15 currently re-defining publishing models (e.g. Plan S, https://www.coalition-s.org). This 16 transition is supported by developments in licencing, data and metadata standards, and 17 by requirements for open publishing and data sharing set by research funding bodies. 18 While many practitioners adopt a subset of OS principles and practices for idealistic or 19 pragmatic purposes, institutional support has been crucial for developing OS 20 infrastructure and mainstreaming OS. Moreover, OS principles conceptualised by 21 practitioners can be adopted and implemented by institutions, as exemplified by the 22 inclusion of the FAIR guiding principles [6] by the European Open Science Cloud [7] 23 and the newly adopted UNESCO Recommendation on Open Science [8]. Through such 24 efforts, many of the key institutional, economic and infrastructure-related challenges in 25 the transition to Open Science have been addressed. 26 Despite these developments, the practices associated with OS are not widely 27 implemented across research communities, but rather within select groups, networks or 28 events involving OS practitioners, or in a sub-optimal piecemeal manner [9]. Thus, a 29 major challenge for fully utilising the potential of OS involves its widespread uptake by 30 diverse members of the scientific community, which will require a major cultural and 31 behavioural shift among scientists. Such a transition is not straightforward, considering 32 the variability in how individual researchers perceive OS in terms of values, required 33 skills and pragmatic trade-offs between benefits and costs [10]. For some practitioners, 34 the interest and entry point to adopting OS practices may be driven by the necessity of 35 reproducibility and replicability. In addition, parallel networks of researchers can differ 36 in their emphasis of these and other OS aspects as well as the extent of collaboration 37 among peers [4]. Therefore, considerable variability in how OS is understood and 38 practiced can be found both between individual researchers and between groups or 39 networks. 40 In the light of these challenges, an understanding of how OS practices and principles 41 are learned, understood, and transmitted among researchers can better inform 42 institutions and policymakers invested in implementing OS in full. As the OS movement 43 is still relatively young, we are particularly interested in practitioners' thoughts on the 44 role of OS in teaching and supervision. Arguably, for OS to become an integral part of 45 mainstream science, its inclusion in how science is taught and learned may prove highly 46 effective and necessary. Our investigative team included individuals involved in organising the colloquium itself 71 and associated collaborators. We met twice before the colloquium to clarify research 72 questions, develop survey items, and subject the items to talk-aloud refinement. This 73 resulted in a questionnaire that we structured into three parts where each part was 74 distributed to colloquium participants as follows: Part I three days before the event 75 started, Part II at the end of the first day, and Part III after the second and final day of 76 the event. We split the survey partially to distribute the effort of respondents taking 77 the survey, to focus on different themes, and to gather data on whether participants' 78 understanding of OS evoloved during the event.    Email addresses submitted by respondents were deleted within two weeks and remaining 95 data were uploaded to the GitHub repository together with the R code. We clarified  consensus on the coding for all responses. Importantly, all coding was done before any 116 coders were aware of the results of the quantitative analyses (see below), to protect the 117 integrity of the coding process. We then discussed overarching themes that resulted 118 from the coding analysis during writing of the manuscript. We have lightly edited some 119 of the quotes reported for grammar and clarity.

136
We performed all quantitative analyses using R software (R Core Team 2020) and 137 the code is available on the GitHub project repository. 138 We conducted analyses of quantitative data as follows. We treated the scaled 139 responses as ordinal data in our analyses by using cumulative link models (clms,  For both clms and clmms, final models were checked for violation of the proportional 157 odds assumption, namely if any of the fixed term estimates varied with the response 158 categories. As these fixed terms were nominal factors, nominal tests were conducted by 159 using the function nominal test for clms. As the same function does not apply to clmms, 160 we performed nominal tests for these models through likelihood ratio tests, comparing 161 the most parsimonious model and a model with the same structure except having the Gender was used as a fixed term, the latter category was omitted as a factor level as it 182 was represented by a single observation.

183
How do respondents define open science? 184 We asked colloquium participants to respond to the following open-ended question: 185 People define 'Open Science' in many ways, and it is a multi-faceted concept. We are 186 interested in how you define Open Science, especially as it pertains to your own work.  Table). Quantifying the occurrence of these codes allows us to infer shared, as well as 189 less common and less salient, perceptions of the meaning of OS (Table 1). In the 190 pre-event responses, some patterns are evident, in that shared data was the most  Very few (3 of 60 respondents) defined OS in a way that specifically referenced Among the 60 respondents to Part I of the survey, 49 were engaged in primary research, 225 and for these we found strong evidence that those having an academic affiliation 226 interacted with OS practices more frequently than colloquium participants with other 227 affiliations (P = 0.003) (Fig. 2, Supporting Information S5 Table). Results also 228 indicated that the frequency of interaction with OS practices in general among 229 respondents was higher for men than for women (P = 0.004) (Supporting Information 230 S5 Table). However, we did not find evidence for higher OS engagement among 231 early-carreer researchers, as the term was discarded in the model selection process 232 (Suppoprting Information S5 Table).

233
Respondents stated Read open access publications as the OS aspect that was most 234 frequently engaged with, followed by Used open code and Used open data (Fig. 1). 235 Further, we found strong evidence for respondents using open data and code more 236 frequently than sharing data and code (P < 0.001), and men did so more frequently 237 than women (P = 0.002) (Supporting information S5 table). separate codebooks for the "hinders" (Supporting information S7 Table) and "helps" 249 (Supporting information S8 Table) responses. It is important to note that many of the 250 perceived hinders reflect the opposite of what is perceived to help and vice versa.

251
Of the sixty responses to the "hinders" item, 7 colloquium participants wrote either 252 nothing or N/A. Many respondents reported lack of guidelines (n=15), lack of time 253 (15), or insufficient knowledge (15) as barriers for engaging in OS (Table 2). One 254 respondent exemplifies these sentiments (along with fear of critique) with the following: 255 "lack of familiarity with relevant online platforms, software, methods. . . perception 256 that the landscape of the above tools changes very quickly, and keeping up is a big time 257 commitment. . . fear of doing it wrong."

258
Of the sixty responses to the "help" item, 20 participants referenced social support 259 as something that helped them to engage in OS (Table 3). Twenty people also cited 260 resource availability. One respondent covered both of these codes saying:     Information S6 table).

282
How do OS practitioners involved in higher education value OS 283 in teaching and supervision?

284
In terms of perceived importance of OS practices in research, teaching and supervision, 285 we found strong evidence for such aspects being given less importance in teaching as 286 compared to research ( P <0.001), but our data did not reveal any such contrast 287 between supervision and research ( Fig. 3; Supporting Information S9 Table). Surveying attendees at the Living Norway 2020 Colloquium gave us a view of how OS 293 can be understood, applied and promoted in a network of dedicated practitioners. We 294 identified two emergent ideas that cut across the responses from the survey, namely 1) 295 that OS is mainly understood in terms of shared and accessible data, code and 296 publications and 2) that individual OS engagement may be further facilitated through 297 education. Firstly, the respondents' own definitions of OS and the stated frequencies of 298 engagement in related practices revealed that the most emphasised aspects were sharing 299 data and code openly in addition to open access publishing. Secondly, our analyses 300 indicate that formal instruction may provide incentives to OS engagement as indicated 301 by the participants, while also offsetting disincentives as these are often identified as 302 practical and/or due to lack of knowledge. Still, OS was considered significantly less 303 important in the context of teaching compared to research and supervision. Taken 304 together, these results suggest that an inclusion of OS in teaching and learning can aid 305 in facilitating wide-scale implementations of OS, even though it is clear from our data 306 that this potential may not be evident to educators currently engaging in OS. Thus, a 307 major implication of our study is that by integrating OS principles and practices more 308 formally into higher education, we can naturally address the implementation barriers 309 that depend on individual experience with OS.

310
Definitions of concepts can reveal how they are perceived and understood, and 311 gathering such definitions is a method that has been applied in previous research 312 (e.g. [14]). Colloquium participants were asked to define OS before the first day of the 313 colloquium, and those definitions helped identify the meaning attributed to OS by 314 attendees ahead of the event. The most frequent codes emerging from those definitions 315 largely mirror practices and principles associated with OS in relevant literature [15], 316 indicating that the respondents were familiar with the associated terms and their 317 meanings. Although there were less responses to the final part of the survey, distributed 318 after the colloquium, where participants were asked to define OS a second time, we 319 interpret the high consistency between initial and final definitions (Fig. 2) as evidence 320 that the event did not substantially change respondents' definition of OS. Meanwhile, 321 we argue that the most frequent codes emerging from the definitions of OS illustrate a 322 shared understanding of the concept among respondents.

323
Taking a closer look on the participants' definitions of OS, the most frequent codes 324 reflect aspects of OS that have a strong practical relevance in ecology [16]. While the 325 methods and approaches of ecological field research reflect the complex variability in the 326 science of ecology and in nature, it is both possible and critically important to ensure 327 the epistemological and computational reproducibility by adhering to the FAIR 328 principles, namely that metohds, protocols and data should be findable, accessible, 329 interoperable and reusable. FAIR was explicitly mentioned in definitions of 11 330 participants and two of these principles were frequently mentioned separately: 331 accessibility (N=38) and -in less explicit terms -replication/reproducibility (N=19).

332
This frequent understanding of OS may not be representative of the ecological research 333 community at largem, however, as the survey was carried out amongst attendants 334 associated with Living Norway, a collaborative structure that promotes FAIR principles 335 in ecology and OS in general terms.

336
Collaborative grassroot structures similar to Living Norway have emerged in recent 337 years [17]. Novel programming tools and enhanced computational power better enable 338 ecologists to address high-level complexity in nature and to generate data across studies 339 and systems. As such methods are highly data-intensive, they require improved 340 alignment of data documentation, management and access across research 341 collaborations and institutions, yielding bigger and more complex datasets. Through 342 analyses of complex systems and research synthesis, ecologists can better inform 343 communities, governments and stakeholders through more accurate predictions that 344 address global concerns, such as ecosystem change [17,18] and functional relationships 345 between environments and organisms [19]. Thus, the wide scale enactment of FAIR 346 principles in ecological research is a means to build robust datasets and analythical 347 pathways that can be put to wider use in the service of science and society. assume that this mirrors a wider tendency for the OS movement, we argue that higher 358 education can address both incentives and disincentives to OS engagement, most 359 importantly through facilitating learning of OS tools, principles and practices, offering a 360 OS supportive social environment, and providing structural OS support.

361
Even though we suggest a potential role for higher education in facilitating OS, an 362 interesting observation from our data is that this possibility may not necessarily be 363 evident to OS practitioners, even not those engaged in teaching. Considering 364 respondents' perceptions of the importance of OS in research, as well as the more 365 frequent engagement with OS for those having a university affiliation, it is remarkable 366 that such practices and principles were deemed less relevant in educational settings.

367
The reasons behind this discrepancy are probably complex and may involve lack of a 368 tradition in higher education for both teaching and learning the relatively new practices 369 and principles of OS, as well as a lack of learning materials and associated uncertainty 370 on how to incorporate OS in teaching and learning activities [20,21]. Furthermore, OS 371 may be perceived by instructors as more relevant for more advanced students, such as 372 those engaged in thesis work, as the respondents gave similar scores for perceived 373 importance of OS in supervision and research. We suggest that a wider adoption of OS 374 in undergraduate teaching could significantly leverage student engagement and learning 375 [22], and thus speed up the implementation of OS in the wider community. Guidance Given the increasing impact of OS on the wider practices and principles of science, 379 including applied science and science communication, early engagement in OS may 380 prove highly beneficial to a growing number of students along the lines described for 381 early career researchers (ECRs) [24][25][26]. Through the careful inclusion of OS practices 382 in higher education study programmes, educators can offer students a range of activities 383 that increase familiarity with OS and its impact on science itself and the science-society 384 interface, while strengthening the acquisition of domain-specific content knowledge.

385
This is likely to promote students' future carreers not only in research per se, but also in 386 other professions that are informed by or interact with research, such as natural 387 resource management, climate science, medicine, engineering and the science-policy 388 interface, more generally.

389
Efforts aimed at implementing OS practices in academic institutions involve a 390 variety of agents acting at different levels, namely practitioners (grassroots), institutions 391 (meso level), or political regulation (top-down). While grassroot collaborative structures 392 can be found globally, the most substantial institutional and political efforts are seen in 393 Europe. The League of European Research Universities (LERU) recommends 394 universities to "integrate Open Science concepts, thinking, and its practical applications 395 in educational and skills development programmes" [27]. Further, such implementations 396 are likely affected by political influence manifested as initiatives such as the European 397 Open Science Cloud [28] stemming from the European Commission Single Market 398 strategy [7]. While a majority of academic institutions in Europe are aiming for the 399 adoption of OS practices in strategic terms, successful implementations are still limited 400 [29]. We suggest that such challenges can be addressed through an interplay between 401 the agents that are invested in OS across different levels. Since grassroot practitioners 402 involved in research and teaching are on the frontline for implementing OS in academic 403 institutions, political efforts can generate the necessary large-scale incentives and 404 structural support.

405
As OS is promoted by a variety of agents, the lack of a unifying definition gives 406 room for diverse interpretations or even skepticism towards OS [30,31]. Other barriers 407 to OS engagement can be practical, such as lacking the required skills, or concerns with 408 the trade-offs pertaining to data sharing [32]. For educators intending to introduce OS 409 in teaching and learning, our main advice is to consider successful initiatives that share 410 a similar purpose. As an example, Project EDDIE (Environmental Data-Driven Inquiry 411 and Explorations) engages students in STEM education by applying active learning 412 methods combined with the use of data repositories that follow the FAIR principles [33]. 413 Further, the International Plant Functional Traits Courses offer training in trait-based 414 ecology through a field campaign grounded in FAIR open science practices, including 415 planning and conducting reproducible fieldwork and data management, and experience 416 with publishing data papers [26,34]. Moreover, educators can obtain formal support for 417 implementing OS in teaching and learning provided through workshops, courses and 418 online-tutorials, and Bossu & Heck [35] offer recommendations on the topic.

419
In conclusion, our study provide insights into how OS can be understood, applied 420 and promoted within a cluster of practitioners. Respondents seemed to understand and 421 practice OS mainly in terms of providing and/or re-using data and code in addition to 422 open access publishing, but are less aware of how OS can support and promote 423 education. Further, statements pertaining to what helps and hinders individual 424 engagement in OS revealed aspects that can be addressed directly through building a 425 collaborative OS science culture, including through higher education and post-graduate 426 training. Even though we can expect variation in terms of experiences and attitudes 427 across the broader ecological and OS communities, we believe that our results are 428 indicative of some trends that deserve closer consideration. In particular, the differential 429 emphasis of OS in research vs. teaching reflects a prolonged schism in academia where 430 these two scholarly activities are typically regulated by dissimilar mechanisms.

431
Therefore, implementing OS holistically in both research and higher education offers a 432 unique opportunity to bring teaching and research closer together, ultimately advancing 433 knowledge and its applications to the most pressing challenges of our time.