Disparities in information on Long-Acting Reversible Contraceptives available to college students on student health center websites in USA

Long Acting Reversible Contraceptive (LARC) methods are among the most effective birth control approaches for adolescent and young adults yet information on these methods is not widespread. We examine LARC information provided by Student Health Centers (SHC) websites from Universities across the USA to document disparities in access to information on these important contraception methods for college students. We find that compared to EC, Condoms, (plus Pap smear as control), LARC is mentioned less frequently than the others and 73% of schools have no LARC content on their SHC websites. There is no standardization in how the sexual and reproductive health information is organized on SHC websites, which might hinder access. When LARC information does exist, readability and accessibility vary. Universities having high rates of the student body who are African American or female are less likely to provide LARC information on their SHC website and universities situated in more rural settings are less likely to post LARC information on their websites.

the Affordable Care Act (ACA) in 2010 required coverage of contraception as an 23 essential health benefit, thereby changing the contraception care landscape in the U.S., 24 including at universities. This led to schools either dropping student insurance plans 25 altogether (such as Brigham Young University) or complying with the mandate of 26 providing contraception coverage. The ACA prioritized reproductive health care in an 27 effort to reduce existing socioeconomic barriers to essential preventive care for women, 28 in the hope of thereby addressing associated health-related disparities [2]. There is now 29 evidence that impacts of the ACA include narrowing the gap in prescription 30 contraception access between black and white women, where black women have 31 historically accessed prescription forms of contraception at lower rates than white 32 women [3]. 33 SHCs must both provide services and adequately inform the student population 34 about these services in order to ensure equitable access to reproductive health care for 35 college students [4]. While much progress has been made in increasing basic access to 36 contraception thorough SHC clinics and provider appointments [5,6], new barriers are 37 emerging. Rather than physically visiting an SHC location on a university campus, 38 most students now make their first contact with an SHC through the internet [7,8]. high-quality information to adolescent and young adult students [9]. As of 2016, 42 approximate 41% of 18 to 24-year-olds were enrolled in college with a higher proportion 43 of female than male attendees (43% female vs. 38% men) and growing racial/ethnic 44 diversity of the student population. Moreover, in studies of sources of health information 45 amongst college students, black and Hispanic students were more likely than white the student bodies of universities diversify, SHCs may take on a larger role in supporting 48 student health through the equitable provision of appropriate education and services. 49 Given the central role that SHCs play in providing health information and services 50 to adolescents and young adults, there is growing interest in documenting the services 51 and information that they provide. A recent study that surveyed college SHCs about 52 their provision of sexual health services had a response rate of about 55% [6], reflecting 53 a strong level of interest in the issue of providing sexual health services through SHCs. 54 However, a key limitation in survey-based studies is that response rates may be biased 55 by issues such as the role of the survey respondent, the level of support for 56 contraceptive care at the SHC, or the resources devoted to reproductive health services 57 at a particular campus. To assess SHCs' role in contraceptive access and education for 58 students, an alternative strategy to administering a survey would be to systematically 59 assess information that SHCs provide to students through their websites. This approach 60 encompasses the online nature of most students' searches for contraceptive information 61 and services. It may allow for the assessment of factors which impact students' ability to 62 access high-quality information about effective and appropriate contraceptive options. Long-acting reversible contraceptive (LARC) methods are among the most effective 66 birth control approaches [11]. LARC contraceptive methods include intrauterine 67 devices (IUD), the hormonal implant, and the shot. Nationally their use as primary 68 source of contraception has been rapidly increasing from 2.4% in 2002 to 11.6% in 2012, 69 a 5-fold increase in 10 years [12,13]. Updated guidelines recommend these methods for 70 use in adolescents and young adults, as they are considered to be extremely effective 71 and safe for this population at high risk of unintended pregnancy [11]. The increase in 72 use has been most rapid for low-income adolescents, increasing 18 and interest in LARC methods among the overall population of the country [16].

79
The question remains as to whether the increase in LARC use has contributed to the 80 decline in teen birth rates. In spite of the last decade's increase in LARC usage, overall 81 LARC adoption rates remain relatively low, especially among adolescents. Less than 6% 82 of U.S. adolescents have used LARC methods [17,18]. This is even more perplexing 83 when the following factors are considered: 1. Adolescents and young adults are at a 84 high risk of unintended pregnancy [15,19,20], 2. The U.S. continues to have the highest 85 adolescent birth rates of any developed country [20], and 3. Unintended pregnancy has 86 extremely severe lifetime consequences for adolescents, including the interruption or 87 prevention of meaningful education [15,19,20].

88
Accurate information and adequate support in decision-making have been shown to 89 significantly influence women's successful use of contraception [8,21]. Clinician bias or 90 misinformation can be a barrier to adequate LARC access for adolescents and young 91 adults [22,23]. Lack of knowledge was identified as the most common barrier to LARC 92 use in one survey of 1,982 female undergraduate students. LARC usage among college 93 students, while increasing, continues to trail usage of older and less reliable 94 methods [4,24]. Many college students report concern over unplanned pregnancy as well 95 as a lack of sufficient LARC education and access [4,25], as well as a desire for greater 96 access to more effective methods including LARC [26]. Misconceptions about infertility 97 and other harms have been identified as a common barrier to IUD uptake among female 98 college students [27]. SHCs and their websites are well positioned to provide 99 high-quality LARC information to diverse populations of students [9].

100
In this study, we take an equity lens and systematically document the information 101 provided by SHC websites at 4-year public universities across United States. We focus 102 on public institutions as they are obligated under Title IX to limit sex-related barriers 103 and required under ACA to provide coverage to contraception if they offer a student 104 health plan. We undertake 3 distinct studies/analyses that build on one another to 105 examine access to online information about LARC. In our first study we examine 106 information on SHC websites on LARC relative to other forms of contraception and 107 reproductive health services. In our second study we develop and deploy an algorithm to 108 search for LARC information on SHC website in a systematic way. In a third study, we 109 examine how the online information for LARC varies by university-level characteristics.  For this study we focused on four categories: LARC methods, Emergency Contraception 117 (EC) methods, Condoms, and Pap smear. Information about EC and condoms was The set of educational institutions for this study were chosen by first querying the 134 National Center for Educational Statistics 135 (https://nces.ed.gov/collegenavigator/) to identify all 4-year, public institutions 136 that grant bachelor's degrees in the U.S. This retrieved a set of 591 universities. Next, a 137 subset of 200 universities was randomly sampled from the 591 universities to define the 138 final set for this study (File S1 File.).

139
The SHC websites of the selected 200 universities were then analyzed manually for 140 presence of the four categories under study (LARC, EC, Condoms, Pap smear).

141
Specifically, annotators were instructed to: 1. locate the SHC website for a given 142 university, and 2. then search for the specified category's keywords on the SHC website. 143 Presence of any of the keywords from the category was recorded as value 1 for that 144 category, and absence of all keywords was recorded as value 0 for that category. For 145 example, if a SHC website mentioned Mirena on one or more web-pages, then the 146 annotator would report 1 for the LARC category. Every category was annotated by two 147 independent coders. Cohen's kappa coefficient was computed for each category to 148 measure inter-annotator reliability.

149
For subsequent analysis we computed Average Annotation Value (AAV) for each 150 data point (university) which is the average of the coders' response value. For example, 151 if both annotators agreed that one or more keywords were present for a category, then 152 then average was 1, if they agreed that keywords were absent, the average was 0, and if 153 they disagreed, the average was 0.5. The AAV essentially reflects the uncertainty 154 coming from the disagreement, while allowing us to keep the data point. This leads to 155 generation of 200 AAVs for each category. Using these AAVs, the LARC category were 156 compared with each of the remaining three categories (EC, Condoms, Pap smear) for

159
The results of the comparative analysis of the four categories are reported in Table 1.

160
The average of the 200 AAVs for each category is reported under the column Mean. 161 These values suggest that the overall prevalence of all four categories is low on SHC 162 websites. The lowest prevalence is for LARC methods (0.29), closely followed by EC 163 (0.30), Condoms (0.35), and Pap smear (0.49). The presence of Pap smear is 164 significantly more prevalent that that of LARC methods on SHC websites (p<0.01). The 165 difference between LARC and Condoms prevalence is marginally significant (p<0.1).

170
The inter-annotator reliability results are given next. Cohen's kappa coefficients 171 were 0.64, 0.60, 0.44, and 0.51 for LARC, EC, Condoms, and Pap smear, respectively. 172 These values are typically interpreted as moderate agreement. It could be argued that 173 the coefficient for LARC (0.64) is bordering on substantial agreement. 174 We had expected much higher coefficients since we believed that the annotation task 175 (searching for specified keywords on a website) was straightforward and objective.

176
Neither of these assumptions were found to be true. Annotators reported that the 177 placement of relevant information was not always intuitive, which leads to annotation 178 errors. There is high variability in how the content on the SHC websites is organized, 179 which makes the annotation task longer, tedious, and further amplifies the possibility to 180 human error. The highest agreement on LARC methods also suggests that unconscious 181 bias might have been introduced into the annotated data -annotators might have been 182 extra careful when searching for LARC methods since it is the focus of the study. These 183 trends motivate the next study which employs computational approaches to avoid 184 human error and bias. The findings from Study 1 and our experience of conducting Study 1, motivates this 188 next study where the central goal is to expand the LARC data gathering efforts in order 189 to analyze national-level trends. Computational approaches lend well to this goal since 190 the LARC data that needs to be gathered is available in digital format on the World 191 Wide Wed (WWW), specifically the university student health center websites.

192
Employing computational approaches offers two important benefits: 1. efficient 193 scaling-up, and 2. effective tracking. Once the computational approaches are designed 194 and developed they can be applied to as many universities as needed without any 195 additional cost (efficient scaling-up). These approaches can also be re-applied as many 196 times as needed to track changes in the data (effective tracking). Achieving either of 197 this with traditional data gathering instruments such as, surveys or annotation efforts 198 (Study 1) is extremely difficult, inefficient, and expensive.

199
In this study we have developed computational approaches for 1. identifying the 200 student health center website for a given university, 2. assessing the accessibility of 201 LARC information available on SHC websites, and 3. assessing the quality of the LARC 202 information available on SHC websites. 203 1 If instead of using 0.5 as an uncertainty value in the rows with a disagreement we simply treat them as missing values, the overall statistical t-test is t(156) = -1.84, p<.05, one-tailed.

204
We commence the study by defining the set of universities that will be analyzed using 205 the computational approaches.  Since the goal of this study is to analyze the reproductive health and contraceptive 215 information provided on SHC websites, the first task we undertook was to find the SHC 216 website, more specifically, the web address (URL) of the SHC website for a given 217 university. We have designed and developed an algorithmic approach for this task that 218 consists of four simple steps:    Once the SHC website is identified for a university, the LARC information provided on 250 the SHC website is studied next. To guide this study, a rubric was developed by a 251 certified nurse-midwife (one of the authors) and a nursing student research assistant.

252
Two key aspects of the LARC information on SHC websites -accessibility and quality -253 were chosen for the rubric. These aspects of LARC information can empower college 254 students to pursue informed decisions about LARC, ultimately making a decision with 255 the support of a qualified healthcare provider. Accessibility of any information directly 256 impacts its use and application. In case of SHC websites, information that is posted on 257 their homepage has much higher accessibility than information on a web-page that is 258 several clicks away from the SHC homepage. The quality of the provided information 259 also impacts its use and application. In the context of this study, quality is defined as 260 understandability of the information. If the LARC information provided on a SHC 261 website uses simple, easy to understand language then it is more likely to be understood 262 and in turn used. In contrast, if the LARC information on SHC website uses specialized 263 medical terminology, then an average college student is unlikely to find it useful. There 264 is an extensive body of research in the context of doctor-patient communication that 265 transfers over this study [28][29][30][31]. The details of how these two aspects were quantified 266 for a given SHC website are provided in the next two subsections. metric. The pseudo-code for our approach is given in Algorithm 1. At a high-level, the 275 algorithm is designed to automate the website navigation and LARC information search 276 process starting from a given SHC homepage. In order to find the LARC content that is 277 closest to the SHC homepage (minimum number of clicks), this exploration is conducted 278 in a breadth-first manner where all the web-pages at the same level/depth are explored 279 before web-pages at deeper levels. To operationalize this logic, a queue is used to prioritize the web-pages (URLs) that have to be explored iteratively. As is shown in Algorithm 1, the expected input to this approach is the SHC website. 283 The URL of the SHC website is the first URL that is added to the queue (Line 2). At 284 each iteration, the URL at the head of the queue is obtained (Line 4), and the web-page 285 content at this URL is searched for LARC terms (Line 5). These LARC terms and 286 phrases used in Study 1 were reused here. Every time a URL is popped from the queue 287 and searched, the #Clicks is increased by one (Line 6) because these steps emulate the 288 action of user clicking a link and exploring the new web-page for LARC content. The 289 algorithm terminates as soon as the first instance of any of the LARC terms is found on 290 a web-page (Line 8). For instance, if SHC homepage has LARC content, then the 291 algorithm stops right after exploring the first URL (SHC homepage), and returns value 292 0 for #Clicks since no clicks were needed to reach LARC content. When a web-page 293 being explored does not contain LARC content, URLs on that web-page that may lead 294 to LARC content are identified and added to the queue (Line 9 and 10). To identify 295 such URLs, the text of the hyperlinks on the web-page is leveraged. Specifically, if the 296 hyperlink text contains any of the predefined keywords then the corresponding URL is 297 added to the queue. The set of keywords was defined based on the observed trends such 298 as SHC websites tend to provide LARC information under sections titled 'Clinical 299 Service', 'Women's health', 'Reproductive health'. The program is terminated if LARC 300 content has not been located even after exploring 100 URLs. It is assumed that LARC 301 content is not present on this SHC website, and a special value of -1 is returned for 302 #Clicks metric to indicate the same. The accuracy of this algorithmic approach was 303 also analyzed and found to be 91% (Details in File S3 File.).      Access to LARC information can be associated with a myriad of factors ranging from 390 the institutional factors to contextual factors related to location [16,17]. We examine a 391 few institutional level demographic variables (racial composition, gender composition) 392 and urbanicity at the county level as key factors related to university providing LARC 393 information on SHC website. Institutional level data is from Integrated Postsecondary 394 Education Data System and county level urbanicity is based on designations from the 395 Center of Disease Control 396 (https://www.cdc.gov/nchs/data_access/urban_rural.htm).

398
Our main analyses examine the associations between institutional demographic 399 variables and county urbanicity and having LARC information on the SHC website. We 400 estimate the association using standard logistic regression models. We present a series 401 of bivariate associations and multivariate association where we account for demographic 402 and urbanicity characteristics simultaneously and control for census region. This 403 analysis was conducted on same set of universities (549) as Study 2.

404
For the subset of 171 universities where LARC information was found on their SHC 405 website, we also analyzed the associations between demographic and urbanicity factors 406 with #Clicks (the minimum number of clicks to the LARC information on SHC website) 407 and the readability scores of the LARC information. Given the small sample we 408 examine difference using Chi-squared test for this analysis.  Table 4 presents odds ratios and 95% confidence intervals for the association between  For the subset of universities that provide LARC information on their SHC websites, 421 we examine #Clicks values and readability scores. LARC information was usually found 422 between 0 to 4 clicks. Most schools with LARC information have details within 1 click 423 from the SHC homepage. There were limited systematic differences, except that at  websites compared to other common reproductive health services such as EC, condoms, 431 and Pap smears. In addition to finding that LARC methods were mentioned less often 432 than the other methods, Study 1 findings suggest that manual annotation may not be 433 the gold standard for research questions regarding online health information access.

434
Study 2 shows that while only 27.5% of the SHCs at U.S. public universities mention 435 LARC methods on their websites, at SHCs where LARC information is provided it has 436 fair accessibility in terms of navigation and readability. In addition, in Study 3 we 437 found that public universities with higher proportions of African American students and 438 female students were less likely to mention LARC information on their SHC websites. 439 Our results suggest that institutional demographic characteristics are associated with 440 access to LARC information for college students. Most importantly, informational 441 disparities may hinder access to and use of LARC methods in ways that could 442 potentially exacerbate pre-existing disparities by putting students at increased risk for 443 unplanned pregnancy.

444
Systematic manual and automated reviews as we have done here may offer an 445 innovative alternative to surveys in the assessment of reproductive health information 446 provided by SHCs. When previous studies have relied on surveys, there has been the 447 risk of low response rates and likely highly select responses from universities with strong 448 SHCs. In addition, surveys of staff and students at SHCs may not accurately reflect the 449 experiences of the full range of students attempting to access contraceptive information 450 through the SHCs. Our results suggest that human annotation is variable even with 451 simple predetermined rubrics, and thus automated methods may promise greater 452 reliability, especially for online information. The above studies demonstrate that the 453 data gathering process for online LARC information can be automated with high 454 accuracy. Study 3 results are supported by literature showing that African American 455 women access prescription contraceptive methods such as LARC at lower rates than 456 white women even after passage of the ACA [3]. There is also evidence that care 457 provided at rural public health clinics may pose unnecessary barriers to LARC access or 458 simply not provide LARC services [32].

Strengths and limitations 460
This study has several important strengths, the first being that it applies an 461 interdisciplinary approach to a traditional public health question, leveraging the 462 strengths of several disparate fields for a new and innovative approach. Moreover, our 463 results show that automated approaches can be used to efficiently scale data gathering 464 efforts in a systematic way, which can support longitudinal tracking studies that observe 465 the changes in online informational health data provided by organizations. Specifically, 466 while the results from the SHC website identification task revealed a lack of 467 standardization for SHC web addresses, the now-vetted algorithm could be deployed  This study also has several limitations. Firstly, there is the contextual issue of ACA 473 coverage, which mandates contraceptive provision by SHCs but does not require that all 474 types of contraceptives be available and covered at SHCs. An inherent assumption of 475 our study is that one could expect accurate, accessible information about LARC 476 methods to be offered on an SHC website even if the services are not provided by the 477 SHC. The rationale for this is that posting the information on the website has a positive 478 impact on choice and safety for students [4] and is essentially low-cost or free. Yet given 479 the information available on the websites (or lack thereof), it was not possible for us to 480 fully ascertain which schools may be providing these services in their clinics. Additional 481 research is needed to understand the link between LARC information offered on SHC 482 website and service provision in SHC clinics. Another key limitation is that only a few 483 variables were examined as factors associated with LARC information and the 484 evaluation occurred at one point in time. To understand whether these associations are 485 casual, future studies would be needed in order to build a panel of relevant changes in 486 student demographics and LARC information. SHCs have the potential to positively impact health-related outcomes for U.S. college 489 students by providing access to high-quality contraceptive information on their websites, 490 including information on LARC. Improving the low rate at which SHC websites 491 currently provide LARC information would be an effective and low-cost method of 492 improving health outcomes and well-being for college students. In addition, 493 interdisciplinary automated approaches for data collection such as the one we have 494 developed here hold promise for the future study of public health questions involving 495 online information. With repeated use, an automated approach could provide a reliable 496 method for monitoring changes in online health-related information over time. In 497 contrast to traditional survey research, our systematic approach revealed that even 498 groups perceived to be socioeconomically privileged, such as U.S. college students, can 499 experience different levels of access to basic health information and services such as 500 LARC methods based on predictors such as race and urban or rural geographic area.

501
This hidden disparity is often overlooked based on the commonly held assumption that 502 the provision of online information related to a health service can be interpreted as 503 evidence of the provision of that service, without taking into account the accessibility of 504 the online information regarding services for the target population. Future research is 505 needed to test that assumption and explore the role of online information provision and 506 accessibility in health service access, particularly for under-served populations.