ABSTRACT
Editorial decision-making is a fundamental element of the scientific enterprise. We examined whether contributions to editorial decisions at various stages of the publication process is subject to gender disparity, based on analytics collected by the biomedical researcher-led journal eLife. Despite efforts to increase women representation, the board of reviewing editors (BRE) was men-dominant (69%). Moreover, authors suggested more men from the BRE pool, even after correcting for men’s numerical over-representation. Although women editors were proportionally involved in the initial editorial process, they were under-engaged in editorial activities involving reviewers and authors. Additionally, converging evidence showed gender homophily in manuscripts assignment, such that men Senior Editors over-engaged men Reviewing Editors. This tendency was stronger in more gender-balanced scientific disciplines. Together, our findings confirm that gender disparities exist along the editorial process and suggest that merely increasing the proportion of women might not be sufficient to eliminate this bias.
INTRODUCTION
Women remain underrepresented in science, technology, engineering, mathematics and medicine (STEMM), and are also prone to experiencing bias and discrimination (1–5). This gender gap in representation and career advancement is present across all career stages (1, 6–9). For example, beyond the clear disproportionate representation of men over women in senior investigator categories, women receive fewer and less prestigious awards (10–14), obtain fewer grants (15–17), are less frequently invited to write review or comment papers (18–21), and have lower salaries relative to men (6, 7, 22). Gender disparities at senior levels are also noticeable for services to the broader scholarly community, where men are more likely to provide higher status external service, whereas women tend to perform lower status internal service (11, 23). Moreover, although women and men spend comparable time at work, differences in how they fulfil their various responsibilities outside research (e.g., teaching and service compared with research) (24, 25) may contribute to differences in productivity and ultimately to other markers of career success (2, 8, 26, 27). Due to these and other factors, women benefit from less prominence and eminence at senior levels, relative to men (2, 5, 11, 28). These disparities can arise from structural, institutional, and systemic sexism as well as pervasive bias (whether implicit or explicit) harboured by colleagues of any gender (29–31), and can have multiple adverse implications (e.g., for women’s pay (6, 7, 22) and promotion (1, 2, 6–8, 22)).
Scientific publishing is a central aspect of academia, with critical implications for hiring decisions and career advancement. Inequalities, based on an author’s gender, have been systematically documented along different stages of the scientific publishing process (4, 20). First, the proportion of women as first and senior authors in peer-reviewed publications is lower than expected given their prevalence in the field (4, 20, 32–40). Moreover, across different fields, women tend to submit fewer papers than men (41), with larger imbalances in journals with higher impact factors (42). A higher publication standard for women authors, which in turn leads to decreased productivity, could contribute to this gap (43). Gender inequities are also evident once women cross the submission hurdle, in the evaluation of women-led manuscripts. For example, in several studies manipulating authors’ identity, reviewers evaluated conference abstracts, papers, and fellowship applications supposedly written by men as better than when they were supposedly written by women (44, 45). Moreover, a recent analysis of peer review outcomes of 23,876 initial submissions and 7,192 full submissions that were submitted to the journal eLife showed a homophily effect between reviewers and authors (46). In particular, the acceptance rate for manuscripts with men senior authors was greater than for women senior authors and this disparity was greatest when the team of reviewers only comprised men (46). After publication, women are less cited than expected (47–56). This imbalance is mainly due to a homophily effect in men authors, wherein men under-cite women’s publications compared to men’s publications (47, 57).
Gender disparities in the scientific publishing process may be further exacerbated by the underrepresentation of women among journal reviewers and editors. Editorial service is an essential element of the scientific enterprise. Editors and editorial boards are tasked with establishing benchmarks for scientific publishing, and do so by engaging with a wide network of authors, reviewers, and other members of editorial boards. Insofar as editorial service has the potential to influence the progress and direction of a given scientific field, appointment to an editorial board reflects the high regard and trust of a community towards individual editors (5, 57). Despite repeated calls for making deliberate effort to incorporate gender diversity into editorial board structures (5, 58), gender disproportions remain pervasive (59–65). Presently, little is known about gender disparities in the editorial process itself. Here we address this knowledge gap by examining whether the involvement of individuals in an editorial board and along the different stages of the editorial process is subject to gender disparities.
We focused on the journal eLife, a non-profit open-access journal led by researchers, that aims to accelerate discovery by operating a platform for research communication that encourages and recognises the most responsible behaviours (https://elifesciences.org/about). eLife is a selective journal that publishes promising research in all areas of biology and medicine, and its Editorial Board is structured to contain this broad expertise required to evaluate research quality. eLife employs over 600 researchers in their Board of Reviewing Editors (BRE) and from 2019 onwards in particular have made a concerted effort to increase the representation of women editors towards the goal of gender equality. For these reasons, eLife provides a rich case example to evaluate gender imbalance along the editorial process of a STEMM journal. eLife’s review process broadly involves two main stages: initial evaluation of submissions by the eLife editorial team, and evaluation of full submissions together with external reviewers. While the initial evaluation of submissions involves an internal consultation among eLife editors, the ensuing step of handling the review of full submissions includes community-facing interactions with external experts.
The aim of this study was to determine whether the involvement of individuals in eLife’s BRE is subject to gender disparities at various stages of the editorial process. Specifically, we sought to determine whether women eLife editors are proportionally involved in the editorial decision process. To address this question, we explored fully anonymous analytics collected by eLife’s editorial platform. This data was collected for monitoring purposes with the explicit aim to help improve eLife’s submission and review process. The analytics provided binary gender information (“man” or “woman” as assigned by the editorial office based on scientists’ names and perceived gender expression) relating to the handling of submissions. We assessed the presence of gender imbalance at different stages of editors’ participation, starting with the external influence of authors who are invited to nominate potential editors (and appeal their decisions), through to the engagement of Reviewing Editors (REs) by Senior Editors, and then ending with the responsiveness of REs to editorial assignments. We predicted that despite efforts to increase the involvement of women in the BRE, women’s editorial activities would be lower in comparison to men, even after taking into consideration their proportional disparity in the editorial system. Based on related research (46, 47, 59), we further predicted that decreased engagement would be exacerbated by a homophily effect, where men Senior Editors are more likely to engage men REs. By elucidating the editorial actions where gender imbalance is more prominent, we hope that this study will motivate the scientific community to work towards greater equity in this important process.
METHODS
In this methods section, we first provide a detailed description of eLife’s peer review process, before describing the data we study and the statistical methods we employ.
eLife’s peer review process
eLife holds a unique two-stage evaluation process, as detailed in Figure 1A. The first stage is the initial assessment, and the second stage is peer review. We will describe each in turn, along with the series of actions it comprises.
Initial Assessment stage
In the first stage, submitted manuscripts are evaluated by a team of editors with related expertise. A Senior Editor solicits the advice of one or several REs in order to determine whether the manuscript is suitable for peer review. The process of soliciting and receiving advice is carried out in an interactive consultation forum between all involved participants. Thus, the role of the RE at this stage is internal. The outcome of this process is communicated to the author in a letter signed by the Senior Editor. As such, the identity of the advising RE(s) is only known internally. To help the Senior Editor identify the most relevant members of the BRE to solicit as an advising RE, the authors are invited to suggest REs as part of their initial submission.
Peer Review stage
For papers that are invited for full review, an RE is chosen to manage the process by overseeing the reviewer selection and by coordinating an open discussion between the reviewers, the handling Senior Editor, and the RE once all individual reviewer reports have been submitted. The RE is also encouraged to provide their own independent review as one of the peer reviewers. The RE facilitates the discussion and drafts a final decision either rejecting the paper or requesting the necessary revisions to support the acceptance of the paper. The identity of the RE is revealed not only to the reviewers in the discussion, but also to any other experts that were invited to take part in the peer-review process. Both Senior and Reviewing Editors sign the decision letter, and if the paper is published with eLife, they are also named as editors on the published manuscript. As such, the role of the RE at this stage is community-facing.
Post-rejection
In the event that a paper is rejected at either stage of the editorial process, the author(s) can appeal the editorial decision.
Data
Data accumulated by eLife’s platform for science publishing over the years 2017-2019 were organised into two datasets, as summarised in Table 1. The first dataset will be referred to as the BRE dataset, and the second will be referred to as the Manuscript dataset. We will describe each in turn. But first we make a note on assigned gender.
Gender assignment
In all cases, Editor gender was assigned by eLife’s staff based on the editor’s name and gender expression. Note that staff (i) assigned a binary “man” or “woman” gender, (ii) did not distinguish between trans and cis identities, and (iii) did not assign other genders such as nonbinary, genderqueer, agender, or genderfluid. Note that any editor could have a gender different from the one that was assigned, and that true gender may or may not be more widely known by the community for several reasons: (i) scientists might be closeted due to the pervasive violence and discrimination faced by gender minorities, (ii) scientists might share their true gender identity only with a few close colleagues or friends, or (iii) scientists might share their identity freely but because of the complexity of the social network landscape in science, that information may not have reached all other scientists in their field. Accordingly, the staff’s assignment of gender therefore reflects not self-identity but rather the perceived binary gender of the person. This perception is likely to also be held by the majority of the broader community, and hence is particularly relevant to understanding how the editor might be treated by that community (e.g., the frequency with which they might be suggested as a Reviewing Editor by authors). We also note that since early 2020, eLife has given all Senior and Reviewing Editors the option of sharing their self-reported gender identity via a confidential survey. However, the current response rate (~40%) precludes a comprehensive analysis of gender disparities using the data at this stage (66).
BRE dataset
This dataset includes anonymous information relating to the engagement of individual REs in the editorial process. This information includes the start and end dates of their editorial contracts, the number of consultations in which they have been invited to participate, how responsive they are to consultation requests (number of responses and response rate), the number of full submissions assigned, and how many days they take to make an editorial decision. In addition, the editorial staff asks REs to provide a set of keywords that reflect the scope of their research, which was also included in this dataset; for some REs, additional keywords are added by the editorial staff based on the information publicly available on the editors’ academic websites.
Manuscript dataset
This dataset includes information relating to each manuscript submission, detailing the manuscript’s outcome in each of the submission stages. This dataset also contains the assigned gender (as described above) of those BRE members that were suggested by the authors, the recorded gender of the handling RE, and the recorded gender of the assigned Senior Editor. Note that here our information regarding gender pertains only to the editorial team handling the manuscripts and not to the manuscript authors, whose identities were not made available for the present study due to ethical considerations (though we note that the authors’ identity, but not necessarily their self-defined gender, was known to the editors involved in the assessment). Manuscripts with appeals received after the Initial Submission and without a Full Submission decision were most likely rejected prior to review. It is possible that a small fraction of manuscripts were withdrawn prior to evaluation; however, we did not have access to such data.
Additionally, this dataset contains up to two (out of 18) disciplines that the authors assigned to their manuscript upon submission. Options included ‘Neuroscience’, ‘Cell Biology’, ‘Developmental Biology’, ‘Structural Biology and Molecular Biophysics’, ‘Microbiology and Infectious Disease’, ‘Biochemistry and Chemical Biology’, ‘Chromosomes and Gene Expression’, ‘Genetics and Genomics’, ‘Computational and Systems Biology’, ‘Immunology and Inflammation’, ‘Cancer Biology’, ‘Medicine’, ‘Evolutionary Biology’, ‘Physics of Living Systems’, ‘Plant Biology’, ‘Ecology’, ‘Epidemiology and Global Health’, and ‘Sa Cells and Regenerative Medicine’ (Manuscript Dataset; see Tables 1&2). In order to analyse the manuscript data across disciplines, we assigned to each discipline all the manuscripts in which a discipline was chosen at submission. This process created some overlap between disciplines (6289 fully submitted manuscripts; 1979 manuscripts were assigned to two disciplines out of 8268 assigned manuscripts, or 23.9%).
Ethics statement
eLife’s submission guidelines notifies authors that eLife undertakes research and surveys relating to the submission and review process periodically, and that participation does not affect the decision on manuscripts under consideration, or any policies relating to the confidentiality of the review process. Authors who do not wish to participate can opt out of eLife’s research and/or surveys. Ethical approval to analyse and share the anonymised data was given by Goldsmiths, University of London’s Research Ethics Committee.
Data analysis
We applied several exclusion criteria to the data before proceeding with further analysis. In the BRE dataset, we excluded REs who became Senior Editors, or resigned as Senior Editors (and became REs) in a given year, or those who were inactive (i.e., were never contacted on initial submissions). In addition, in the manuscript dataset, we limited the number of author-suggested REs to five per manuscript; and excluded papers handled by guest editors as well as Research Advances, Registered Reports, and formats that go through a different workflow.
Statistical analysis
Results are reported as mean ± standard deviation (StD). Owing to several non-normal distributions in the data, we used non-parametric tests in all analyses. Binomial tests and N-1 χ2 proportion comparison tests were performed to compare one or two proportions using JASP (JASP Team (2020) Version 0.14) and MedCalc online tools (MedCalc Software, Ostend, Belgium), respectively. Contingency table analysis was used for testing the interrelation between binary variables using JASP software. When comparing the means of two groups with unequal sample sizes, we used a permutation-based Welch’s independent t-test (10,000 permutations) in MATLAB (PERMUTOOLS package, The Math Works, Inc. MATLAB. Version 2020a, The Math Works, Inc., 2020. Computer Software. www.mathworks.com/). Pearson correlation coefficients were computed using JASP in order to test the association between continuous scale variables, after checking for normality assumption violations using the Shapiro-Wilk test for bivariate normality. When relevant, all tests were conducted using 2-tailed tests. When effects were close to the critical alpha (p<0.05), we conducted equivalent Bayesian analyses, with default prior settings (Bayesian correlation stretched beta prior width=1; Bayesian Contingency tables, prior concentration=1) using JASP to test whether there was more evidence for H0 or for H1. To investigate whether gender disparities were associated with REs’ expertise, as advertised by eLife to prospective authors, we conducted an analysis of the relative scope and reach of the REs’ keywords broken down by the recorded gender of the RE. Keywords for each RE were extracted and strung together using the ‘OR’ operator and then queried against the PubMed database through NCBI’s public API—‘Entrez Programming Utilities (E-utilities)’ (Entrez Programming Utilities Help [Internet]. Bethesda (MD): National Center for Biotechnology Information (US); 2010-. Available from: https://www.ncbi.nlm.nih.gov/books/NBK25501/). The number of search results for each set of RE keywords was recorded and used as a measure of the reach of the keywords provided by the REs, as evidenced by published papers related to the keywords in the literature. The E-Utilities API was accessed through a script in Python (Python Software Foundation. Python Language Reference, version 3.9.6. Available at http://www.python.org).
Results
Gender imbalance of eLife Reviewing Editors
We first quantified the gender ratio among eLife BRE members (Figure 1B). The contribution for each individual BRE member was measured as the number of months the BRE member was active in each of the individual years included in our dataset (months of BRE service per year). The proportion of RE months of service contributed by men was significantly larger than the proportion contributed by women throughout the entire study period (2017-2019: N=12,518 months, women vs. men BRE service months: 30.60% vs 69.40% ; binomial p<0.001; Cohen’s h=0.40).
We next considered dynamics in gender balance over the three-year window. The gender imbalance observed overall slightly diminished over time due to eLife’s effort to recruit more women to the BRE. The proportion of women in the BRE did not significantly differ between 2017 and 2018 (1.81% difference, χ2(1)=3.10, p=0.078, Cohen’s h=0.86). By contrast, the proportion in 2019 was significantly greater than that in 2018 (N-1 χ2 proportion comparison test; 2018 vs. 2019: 4.42% difference, χ2(1)=19.67, p<0.001, Cohen’s h=0.78). Despite this slight improvement, the BRE gender base rate remained strongly imbalanced (2017: N=3,715 months, women vs. men BRE service months: 27.64% vs. 72.36%; 2018: N=4,047 months, 29.45% vs. 70.55%; 2019: N=4,756 months, 33.87% vs. 66.13%; binomial p-values<0.001; 2017: Cohen’s h=0.46, 2018: Cohen’s h=0.42, 2019: Cohen’s h=0.33). Accordingly, and for all subsequent analyses, the 2017-2019 data were pooled to increase statistical power. Taken together, these results indicate that there exists a pronounced gender imbalance in the BRE gender base rate.
External influence in the Initial Submission – (Action 1)
At the Initial Submission stage, authors suggest potential BRE members that could handle their manuscript (Action 1). We tested if this action was (im)balanced according to gender by comparing the proportion of women REs that were suggested by authors relative to the women BRE member base rate. A N-1 χ2 proportion comparison test revealed that authors suggest significantly fewer women REs than the corresponding proportion among eLife’s BRE (29.08% vs. 30.6%, χ2(1)=11.65, p<0.001, Cohen’s h=0.90; Figure 1C). We next sought to determine whether women’s perceived expertise might be a partial explanation for authors’ imbalanced RE suggestions. Accordingly, we tested whether women and men REs differed in the number of keywords used to showcase their expertise. We found that women and men REs did not differ in their numbers of associated keywords (Women: 5.51±2.19; Men: 5.32±2.39; t(581)=0.932, p=0.352). We next sought to determine whether a difference in the scope and reach of the keywords associated with women and men REs could contribute to authors’ imbalanced RE suggestions. Accordingly, we quantified the number of PubMed search results for women and men BRE members’ keywords. A permutation Welch’s t-test comparing groups in the number of PubMed search results was not significant (women vs. men search results: 1,755,724±2,979,049 vs. 1,920,643±3,307,501; t(488.9475)=0.62; p=0.56; Hedge’s g=−0.052). These data provide no evidence of a gender difference in the overall reach of the keywords provided by BRE members.
Internal processes in the Initial Assessment stage (Actions 2-3)
We next explored the presence of gender imbalances during Initial Assessments. In this action, the Senior Editor invites one or more REs for an initial consultation in order to assess whether to invite a full submission of the manuscript for peer review. To test whether Senior Editors tend to similarly engage women and men REs (Action 2), we compared the average number of consultation requests per month for individual REs. A permutation Welch’s t-test showed no significant difference in the mean number of requests per month between women and men REs (t(809.7)= 0.11, p=0.92; Figure 1D), indicating no evidence for imbalanced engagement solely based on RE gender in this Action. While examining the distributions of requests per month, it appeared that the distribution of the men REs might have a longer tail (kurtosis men RE=2.36; kurtosis women RE =1.28). Intuitively, a gender difference in the distribution of requests per month could be due to the increased involvement of selected men REs. To examine this possibility, we selected the BREs who were disproportionally engaged in initial consultations relative to the BRE; that is, the 43 REs defined as the upper outliers of the population (defined as higher than the 75th percentile+1.5×interquartile range), with an average of 6.9 monthly consultations, relative to 2.24 on average. We find that only 10 of these especially engaged REs (23%) were women. However, Levene’s test for equality of variances did not show significant differences between men and women RE request distributions (F(1,1217)=0.052, p=0.82). As such, we find no evidence for gender differences when approaching REs for initial consultation.
We then evaluated the presence of gender differences in RE responses to the initial consultation request (Action 3). Compared to men REs, the response rate of women REs was significantly lower (0.88±0.17 compared to 0.91±0.14; Welch’s t(651.4)= 3.04, p=0.001, Hedges’s g=0.20; Figure 1D). In addition, women REs took longer to respond compared to men REs (1.83±1.55 days vs. 1.54±1.23 days; Welch’s t(636.3)=−3.24, p=0.002, Hedges’s g=−0.22; Figure 1D). These data provide converging evidence for reduced responsiveness of women REs to engage in initial consultations, in comparison to men REs.
Community-facing processes in the Full Submission stage (Action 4)
For manuscripts that pass the initial assessment, the Senior Editor assigns an RE who handles the reviewing process (Action 4). In order to evaluate the presence of imbalances in RE assignment, we first compared the number of full submissions per month handled by women and men REs. A permutation Welch’s t-test showed that women REs handled slightly, though significantly, fewer submissions per month than men REs (0.40±0.32 vs. 0.44±0.37; t(869.8)=2.22, p=0.026, Hedges’s g=0.13; Figure 2A). We next explored the effect of the Senior Editor’s gender on manuscript assignment to women and men REs. Using a contingency table analysis, we compared the proportion of manuscripts assigned to women and to men REs as a function of Senior Editor gender. Compared to RE gender base rates of manuscript assignment (6,289 manuscripts; women vs. men RE assigned manuscripts: 30.04% vs. 69.96 %), women Senior Editors assigned significantly more manuscripts to women REs (41.41% for women SEs vs. 30.04% for all SEs) and men Senior Editors assigned significantly more manuscripts to men REs (76.57% for men SEs vs. the 69.96% for all SEs; χ2(1)=224.55, p<0.001, contingency coefficient 0.186; Figure 2B). These results demonstrate that both women and men SEs are more likely to assign papers to REs of the same gender relative to the gender base rates.
In order to examine how this manifestation of gender homophily might vary across disciplines, we next divided the manuscripts according to the disciplines the authors assigned to their submission (up to 2 out of 18 suggested discipline categories; see Table 2). We repeated the contingency table analysis for each discipline separately and found a significant homophily effect of Senior Editor gender on the gender of the assigned RE in 14 out of the 18 disciplines (78%; Contingency table analysis with FDR correction for multiple comparisons; see Table 2 for details). Figure 2C shows the gender homophily in RE manuscript assignment across all manuscript discipline categories. These results demonstrate that gender homophily in manuscript assignment is a widespread cross-disciplinary effect.
Previous research suggests that homophily effects negatively associate with the extent of gender imbalance (47). Accordingly, we next explored associations between homophily in manuscript assignments and gender across disciplines (Figure 2D). For each discipline, we first defined Senior Editor homophily as the difference between (i) the proportion of manuscripts assigned to men REs by men Senior Editors and (ii) the proportion assigned to men REs by women Senior Editors. Intuitively, a value of zero indicates no gender difference between men and women Senior Editor manuscript assignments, whereas a value of unity indicates that Senior Editors only assign manuscripts to REs of their own gender. We similarly defined for each discipline an index that we refer to as the manuscript assignment imbalance, which is calculated as the difference between (i) the proportion of manuscripts assigned to men REs and (ii) the proportion assigned to women REs. Intuitively, a value of zero indicates a fully balanced discipline, whereas a value of unity indicates that manuscripts are assigned exclusively to men REs. Across disciplines, the correlation between Senior Editor homophily and the manuscript assignment imbalance index was negative, albeit borderline in statistical significance (r=−0.47, p=0.049). A Bayesian correlation analysis also suggested only anecdotal evidence in favour of a negative association (BF10=1.77). This result provides preliminary evidence that in disciplines with more equal manuscript assignment, Senior Editor homophily is stronger, in line with previous research (47).
Appeals (Action 5)
In our final analysis, we evaluated the presence of gender imbalances in authors’ appeals (Action 5). In the Initial Assessment stage, only the identity of the Senior Editor is revealed to the authors. The difference in the rates of appeals of manuscripts handled by women and men Senior Editors in the Initial Assessment was marginal; we observed a trend towards fewer appeals over women Senior Editors’ assessments, but this trend did not reach statistical significance (Contingency table analysis, χ2(1)=3.781, p=0.052, Contingency coefficient=0.013; Figure 2E). Moreover, a Bayesian Contingency table analysis suggested moderate evidence in favour of the null hypothesis (BF10=0.28), confirming the lack of difference in Senior Editor gender in Initial Assessment appeals. In the Full Submission stage, the identities of both the Senior Editor and the handling RE are revealed to the authors. Dovetailing with the Initial appeals findings, the gender difference in the rates of appeals for the final decision for both Senior Editors and REs did not reach significance (Senior Editor gender: Contingency analysis, χ2(1)=0.34, p=0.58; RE gender: Contingency analysis, χ2(1)=1.69, p=0.19; Figure 1E). These results suggest that in general, authors’ tendency to appeal does not seem to depend on the gender of the Senior Editor and handling RE. It is important to note, however, that the small rate of appeals limits the robustness of this finding: we observed 809 initial assessment appeals out of 24018 initial submissions (3.4%), and 417 final decision appeals out of 6289 fully submitted manuscripts (6.6%).
Discussion
Gender imbalance in the scientific publishing process is already evident when considering simple numerical disparities, starting with women’s representation in scientific editorial boards (59–65), number of invited articles (66–68), frequency of being asked to referee (67–69), and number of publications (33, 34, 70). Here we extend the scope of this disparity by reporting clear under-representation of women in the BRE of a prominent biomedical journal (eLife). Beyond numerical proportions, the eLife dataset allowed us to examine whether the various actions that make up the editorial process are related to RE gender. We find that gender disparity stretches well beyond the known numerical imbalance, hinting at gender biases influencing the editorial process. Moreover, in a number of cases, gender disparity effects were large in magnitude. The gender disparity is first exerted by external influence—authors suggest more men from the pool of REs, even after correcting for men’s numerical over-representation in the BRE. We also see gender disparity within eLife, in terms of the RE’s bidirectional engagement during the internal initial assessment of submissions. Perhaps most strikingly, we find a robust homophily effect when assigning REs to lead the community-facing role of the editorial peer review. Each of these gender disparity effects is compatible with previous research demonstrating systematic biases in STEMM. Where we add to this body of knowledge is by uncovering the internal working of editorial decisions that will impact the participation and contribution of women. By revealing multiple contributing factors that exacerbate the existing imbalance, our findings highlight the need to assess and correct gender disparities in terms of the contribution to the editorial process (equity) and not just in terms of proportional representation (equality). It is our hope that a better understanding of these mechanisms will help reduce the biases that we document.
The eLife dataset
Before we discuss our key findings, it is important to consider our unique dataset and the potential advantages and limitations inherent to it. As detailed in the Methods section, we used anonymous analytics collected by eLife’s editorial platform for monitoring purposes. This rich dataset reflects a real-life process, and spans a relatively large range of biological disciplines and international contributing scientists and editors. During the investigation period, eLife had a similar fraction of women in their BRE relative to other editorial boards (60), suggesting that the issues identified here are likely to be observed in other journals. However, the specific factors that we could study were not pre-determined based on our experimental needs. Accordingly, we were limited in our explanatory power, both in terms of other relevant factors that might be contributing to the observed effects (e.g., the level of seniority of each RE) and in terms of the statistical power (e.g., authors’ appeals are rather infrequent). To mitigate some of these gaps, we can gain some insight from more recent data relating to the heterogeneity of eLife’s BRE (see Supplemental Section), although these recent analytics may not fully represent the dataset we analysed here. It is also important to consider the makeup of the BRE; these are invited roles, and as such, all the REs are established in their subfields. However, due to issues we expand upon below, it is possible that women REs are less senior than men REs, as described in the Supplemental data. We also do not have data on the intersectionality of gender with other primary sources of disparity (e.g., geographic location, race, ethnicity, class, sexual orientation, and ability (71–74)). Yet, the results of a recent eLife self-report survey conducted outside our study period suggest that women serving as editors are more likely to also self-identify as belonging to an underrepresented or minority group based on their race or ethnicity (66). Finally, as described in the Methods section, perceived gender was assigned as “man” or “woman” (without distinguishing trans from cis) based on the REs’ names and public profiles, and hence may or may not reflect the BRE’s true gender identity. Although eLife recent data suggest that the vast majority of the BRE is cis (66), gender identity was not measured along and outside the binary (e.g., nonbinary, genderfluid, etc.). With these points in mind, our gender effects might be modulated by other contributing factors, that should be investigated in future research in greater detail.
Gender disparities
We first considered gender differences in REs bidirectional engagement, including both invitations to contribute to the initial editorial consultation by the Senior Editors and the individuals’ participation in response. We define this process as internally-facing because the identity of the REs involved is only revealed to the other editors engaged in the consultation. We did not find significant differences in the number of invitations of women REs by the Senior Editors to participate in initial consultations relative to men REs. However, we did observe a heavy-tailed, skewed distribution of consultations, such that there is a small group, mostly comprised of men, that disproportionally dominates initial consultations. Even if the differential proportions of these groups are not statistically significant, this small men-dominated group might still skew diversity (75, 76). To distribute the influence more fairly, a potential solution is to cap the number of consultations per individual RE.
Although the number of initial invitations did not differ between women and men, women engaged less with invitations from the SEs, resulting in the under-involvement of women in editorial activities. The delayed responsiveness of women relative to men was slight (women were approximately 7 hours slower to answer emailed invitations), but considering the interactive nature of the consultation process, this delay could be meaningful. In the eLife initial consultation process, this means that men are more likely to set the tone of the discussion by providing their opinion first, making it more difficult for women, on average, to influence the editorial decision (through conformity and anchoring cognitive biases for example (77–79)). It has been previously shown that it is more difficult to voice a different opinion once an opinion has been formed (80, 81). The delayed response, as well as reduced response rate (by approximately 3%) could potentially be attributed to the fact that women have more duties and responsibilities than men REs. There are multiple reasons to suggest this, depending on women’s specific intersecting identities (40, 82, 83). For example, senior women are overburdened by administrative responsibilities due to the institutional need to narrow the gender gap (40). More specifically to our dataset, there is a hint that women REs are at an earlier career stage relative to men (Supplementary section), and hence may be more likely to have children at home than their men colleagues and thus face an added burden on their time (11, 12, 14), or be more laden with obtaining tenure. Another potential contributing factor is the higher standard of communication women are held to in order to receive equal acknowledgment, resulting in an imposed time-consuming quantity/quality trade-off for women, and reducing their productivity (43, 84, 85). Irrespective of the reasons, our results signal that the journal submission and review process needs to shift away from monitoring decisions based on the decision time, which adds time pressure, and instead could potentially delay discussion and/or decisions about submissions until women have contributed.
We did find a significant difference in the engagement of women REs when considering community-facing duties, particularly when leading the peer-review process. Specifically, women were assigned 9% fewer manuscripts relative to men. This effect is likely exacerbated by the reduced responsiveness we observed during initial consultations, as the assignment of the reviewing RE is often determined during the initial consultation. We are hopeful that if the bias in the previous stage is corrected then the under-assignment of full submissions to women REs will be improved. However, it is also important to consider more carefully other potential sources of bias and how to mitigate them. For example, it is also possible that men might volunteer more readily to take up this time-consuming role – our data does not allow us to shed any light on the inner discussions beyond response time. Regardless, our effect is consistent with other studies showing that women are disproportionately engaged in internal-institutional facing duties, whereas men are disproportionately engaged in community-facing roles, which are also more associated with eminence, networking, and other benefits related to the more visible duties of the reviewing RE leading the peer review (11, 13, 19, 21, 67). The reasons underlying this pattern should be further studied, however women’s different time allocation may reflect a purposeful choice to contribute to their institutions. Another potential driver could be inherent biases of the Senior Editor assigning the RE; research shows that women are less frequently approached to apply for awards, write invited reviews, etc. (84). Within the context of editorial assignments, this effect could be potentially corrected by providing gender-specific statistics to the Senior Editors about disproportional engagement by gender. We turn to consider gender-based interactions between the Senior Editor and REs in the next section.
Homophilic Behaviours
Homophily is one of the fundamental patterns underlying human relationships across multiple social systems, influencing how communities form, how status is distributed, and how subgroups evolve in occupations and organizations (86). With respect to the homophily effect of the Senior Editor’s gender on REs assignments, we find that across multiple sub-disciplines, there is a significant tendency for Senior Editors to choose same-gender REs to handle full submission for peer reviews. One might wonder whether the observed homophily effects might be explained by field-specific differences in gender proportions: in a discipline comprised mostly by men, e.g. physics of living systems, the Senior Editor (likely a man) will more often reach out to more men simply because most of the experts are men. To evaluate this possible explanation, we separated our data by discipline. We found that the homophily effect exists quite broadly, across 14 of the 18 disciplines (despite noticeable variability in the proportion of women/men RE across disciplines, see Figure 3C), hence refuting the differently-gendered subdisciplines account. What other drivers could potentially explain the homophily effect? Homophily is driven by various types of associations and dimensions of similarity (87), such as ascribed attributes (e.g. gender (88)), acquired attributes (e.g. occupation (89)), values, attitudes, and beliefs (e.g. activism (90)). Homophily, and gender homophily in particular, are prevalent in academia, for example in shaping interactions in scientific conferences (91), affecting scientific collaboration and scientific societies (11, 92), and biasing the selection of Nobel laureates (59, 91). Thus, we were not surprised to find that men Senior Editors assign more men REs than the women REs, even after taking into consideration the larger numerical proportion of men in the BRE.
It is possible that homophily in women arises from different drivers than homophily in men (59, 93), due to distinct social processes (94, 95) and the roles they play in intersectional power structures (96). Considering the current political climate where there is greater awareness for the under-representation of women in STEMM, it is possible that women Senior Editors adopt an informal policy to engage women REs disproportionately. In this respect, the women homophily offsets to some degree the gender bias we see in the editorial process. Activism-driven homophily among women was demonstrated for example in crowdfunding of start-up projects, whereby a small proportion of women backers disproportionately supported women-led projects in areas where women are historically underrepresented (90). Similarly, gender homophily in reviewer assignment by journal editors was widespread among men editors, while for women only a small number of highly homophilic editors dominated (47). Our data did not allow us to directly explore the prevalence of homophily among individual REs, yet the fact that homophily was widespread across many fields, involving different REs, suggests women homophily is a broad phenomenon in eLife. Additionally, we find that homophily increases with gender balance across sub-disciplines. This echoes the finding that men homophily in article citations increases as the research field gets more gender balanced with time (11, 59, 88, 95, 96). However, given that women Senior Editors are outnumbered by men (for example, 36% (30) women vs. 64% (52) men Senior Editors in 2021), on average we see an over-engagement of the men REs, even after accounting for their numerical dominance in the BRE. One simple candidate intervention is to increase the proportion of women in senior roles, which could also potentially serve to address other aspects of gender disparity that we did not study here. However, for the reasons detailed above, simply increasing representation (e.g., the number of women) might not be sufficient to ensure inclusion, equity, and justice (11, 59, 90, 97, 98).
Despite the fact that women display homophilic tendencies that serve to partly balance the homophilic tendencies of men, we do not in general endorse homophily effects as an appropriate solution to the gender bias observed here, as it can have devastating trickle-down consequences. For example, it was previously shown that scientific journal editors of both genders were more likely to appoint reviewers of the same gender as themselves (46). Moreover, a previous study of eLife editorial decisions focused on how the gender makeup of the participants in the peer-review stage – both editors and reviewers – biases acceptance rates for men and women authors (59, 97, 98). It was observed that all-men reviewer teams are far more likely to accept men-led manuscripts. Therefore, the homophilic behaviour that we observe among men is likely to exacerbate these effects and increase the gender publishing gap. More generally, it was shown that homophilic groups tend to have similar evaluations and mind-sets (59, 99, 100). Hence, the uncontrolled effects of homophily may undermine the impartiality of peer-review, and thus undermine science (59, 101). Instead, solutions should be driven by formal policy that foreground equity and justice. For example, the homophily factor could be monitored to help Senior Editors avoid implicit and explicit biases. Another important candidate intervention for this issue is to diversify the network of the Senior Editors within the BRE.
Conclusion
A summary of our results in provided in Table 3. To conclude, at the time of our analysis, eLife and other scientific journals do not have a formal strategy for engaging women, beyond increasing their numerical proportion. By including more women in the editorial process, the hope is that their voice will be expressed and heard. However, the evidence provided here suggests that simply increasing women’s numbers is not enough to overcome gender bias. Critically, without taking into consideration women’s specific work habits and availability, starting with their potentially different career demands, through different work-life balance and ending with sociological preferences, it is difficult to imagine a future in which the underlying mechanisms for under-engagement of women do not continue to bias the process. We therefore suggest that in order to index gender balance, we need a focus on equity rather than equality. We further suggest that informal policies, such as gender homophily, need to be replaced by formal policies that are based on educating both Senior and Reviewer Editors on how the choices that they make during editorial activities impact the gender gap.
Conflicts of Interest
Tamar Makin is a Senior Editor at eLife. Maria Guerreiro is part of the executive staff team of eLife.
Diversity Statement
Recent work in several fields of science has identified a bias in citation practices such that papers from women and other minority scholars are under-cited relative to the number of such papers in the field (47, 49–56). Here we sought to proactively consider choosing references that reflect the diversity of the field in thought, form of contribution, gender, race, ethnicity, and other factors. First, we obtained the predicted gender of the first and last author of each reference by using databases that store the probability of a first name being carried by a woman (47, 102). By this measure (and excluding self-citations to the first and last authors of our current paper), our references contain 30.62% woman(first)/woman(last), 22.82% man/woman, 18.14% woman/man, and 28.42% man/man. This method is limited in that a) names, pronouns, and social media profiles used to construct the databases may not, in every case, be indicative of gender identity and b) it cannot account for intersex, non-binary, or transgender people. Second, we obtained predicted racial/ethnic category of the first and last author of each reference by databases that store the probability of a first and last name being carried by an author of color (103, 104). By this measure (and excluding self-citations), our references contain 6.21% author of color (first)/author of color(last), 15.01% white author/author of color, 16.03% author of color/white author, and 62.75% white author/white author. This method is limited in that a) names and Florida Voter Data to make the predictions may not be indicative of racial/ethnic identity, and b) it cannot account for Indigenous and mixed-race authors, or those who may face differential biases due to the ambiguous racialization or ethnicization of their names. We look forward to future work that could help us to better understand how to support equitable practices in science.
Supplementary data
Supplementary methods
Datasets
To get some intuition about potential demographic factors that could mitigate gender disparities observed in our data, we analysed two additional datasets.
Senior Editor Dataset
This dataset contains information relating to the assigned gender (as described in the Methods section) of eLife Senior Editors, including Editor-in chief and Deputy Editors, who act as Senior Editors in the reviewing process. These data where extracted from eLife’s website (eLife leadership team (2021), retrieved from https://elifesciences.org/about/people).
BRE demographic dataset
This dataset contains anonymous information relating to the assigned gender of REs (as described above), their continent of residence, as inferred by the location of the institution where they are primarily based, and their career stage (number of years since independence). These analytic data were acquired by eLife during February 2019, January 2020, and December 2020. RE career-stage was divided into three categories: Early career (less than or equal to 5 years of independence), Mid-career (6-15 years of independence), and Late career (more than 16 years of independence).
Statistical analysis
N-1 χ2 proportion comparison test was performed to compare the gender proportions of Senior Editors MedCalc online tools (MedCalc Software, Ostend, Belgium), and contingency table analysis was used for testing the interrelation between RE gender and Career stage, and between RE gender and Continent of residence, using JASP software.
Supplementary results
There was a significant imbalance in Senior Editor’s gender (36% women vs. 64% men, χ2(1)=5.848, p=0.016, Cohen’s h=0.56; Supp Fig.1A).
There was a significant disparity in the career stage distribution between men and women REs: women REs tended to be at earlier career stages than men REs (women: Early 14.95%, Mid 47.27%, Late 37.78%; Men: Early 6.34%, Mid 36.60%, Late 57.06%; χ2(2)=56.04, p<0.001,
Contingency coefficient=0.20; Supp Fig.1B). In contrast, there was no evidence for gender disparity in the geographical representation of women and men REs (χ2(5)=8.36, p=0.14; Supp Fig.1C).
Acknowledgements
We thank the eLife editorial team for facilitating this study at all stages, and in particular, to: Andy Collings and Daniel Ecer for help with preparing the dataset; Jennifer Raymond, Chris Baker and Jon Roiser for discussions about the dataset; Stuart King for feedback on the manuscript; Michael B. Eisen, Tim Behrens and eLife Senior Leadership for feedback and ongoing support of this study. T.S.M was funded by an Agence Nationale de la Recherche grant (ANR-16-CE37-0005); T.R.M. was funded by a Wellcome Trust Senior Research Fellowship (215575/Z/19/Z) and an ERC Starting Grant (715022 EmbodiedTech).
References
- 1.↵
- 2.↵
- 3.
- 4.↵
- 5.↵
- 6.↵
- 7.↵
- 8.↵
- 9.↵
- 10.↵
- 11.↵
- 12.↵
- 13.↵
- 14.↵
- 15.↵
- 16.
- 17.↵
- 18.↵
- 19.↵
- 20.↵
- 21.↵
- 22.↵
- 23.↵
- 24.↵
- 25.↵
- 26.↵
- 27.↵
- 28.↵
- 29.↵
- 30.
- 31.↵
- 32.↵
- 33.↵
- 34.↵
- 35.
- 36.
- 37.
- 38.
- 39.
- 40.↵
- 41.↵
- 42.↵
- 43.↵
- 44.↵
- 45.↵
- 46.↵
- 47.↵
- 48.
- 49.↵
- 50.
- 51.
- 52.
- 53.
- 54.
- 55.
- 56.↵
- 57.↵
- 58.↵
- 59.↵
- 60.↵
- 61.
- 62.
- 63.
- 64.
- 65.↵
- 66.↵
- 67.↵
- 68.↵
- 69.↵
- 70.↵
- 71.↵
- 72.
- 73.
- 74.↵
- 75.↵
- 76.↵
- 77.↵
- 78.
- 79.↵
- 80.↵
- 81.↵
- 82.↵
- 83.↵
- 84.↵
- 85.↵
- 86.↵
- 87.↵
- 88.↵
- 89.↵
- 90.↵
- 91.↵
- 92.↵
- 93.↵
- 94.↵
- 95.↵
- 96.↵
- 97.↵
- 98.↵
- 99.↵
- 100.↵
- 101.↵
- 102.↵
- 103.↵
- 104.↵