Abstract
As the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic was expanding, it was clear that effective testing for the presence of neutralizing antibodies in the blood of convalescent patients would be critical for development of plasma-based therapeutic approaches. To address the need for a high-quality neutralization assay against SARS-CoV-2, a previously established fluorescence reduction neutralization assay (FRNA) against Middle East respiratory syndrome coronavirus (MERS-CoV) was modified and optimized. The SARS-CoV-2 FRNA provides a quantitative assessment of a large number of infected cells through use of a high-content imaging system. Because of this approach, and the fact that it does not involve subjective interpretation, this assay is more efficient and more accurate than other neutralization assays. In addition, the ability to set robust acceptance criteria for individual plates and specific test wells provided further rigor to this assay. Such agile adaptability avails use with multiple virus variants. By February 2021, the SARS-CoV-2 FRNA had been used to screen over 5,000 samples, including acute and convalescent plasma or serum samples and therapeutic antibody treatments, for SARS-CoV-2 neutralizing titers.
1. Introduction
Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), a betacoronavirus with a positive-strand RNA genome, was identified as a novel pathogen and causative agent of coronavirus disease (COVID-19) in humans. The high transmissibility of the virus has led to a pandemic with over 110 million documented infections worldwide and over 2.4 million deaths (as of 22 February 2021).
While initially thought to cause a primarily respiratory disease, time has shown that SARS-CoV-2 infection can cause multiple clinical manifestations, including severe respiratory disease, cardiovascular disease [1], and neurological disease [2, 3]. In addition, convalescence in some people can be complicated by long-term sequelae that can be quite severe [2-4]. Early on, individuals with pre-existing comorbidities were identified as being more severely affected by SARS-CoV-2 infection [5, 6], and there are indications that age, gender, race, and other genetic factors play a role in disease severity and clinical outcome [7, 8].
During acute disease, there appears to be a rapid antibody class switch from immunoglobulin M (IgM) to IgG and IgA [9, 10], although a slow class switch may be predictive of patient prognosis and associated with patients requiring hospitalization [11]. The antibody isotype is important in controlling the disease, and so is the target viral protein. In fact, more robust and prolonged antibody responses to the viral nucleoprotein (N) were associated with more severe disease [11].
The presence of anti-SARS-CoV-2 antibodies in blood is presumed to be a good measure of protective immunity for a vaccine candidate. Hence, methods to reliably, sensitively, and rapidly detect SARS-CoV-2 neutralizing antibodies are needed for pre-clinical vaccine studies and clinical trials. Further, quantifying potent neutralizing antibodies from recovered COVID-19 patients may be useful in identifying potential donors for passive immunization and hyper-immunoglobulin therapeutic applications. The U.S. Food and Drug Administration (FDA) initially approved an Expanded Access Program (EAP) [12] for the treatment of COVID-19 using plasma from individuals with a neutralization titer of 1:160 or higher. This program led to the treatment of over 94,000 patients at participating provider locations across the U.S. Based on data from the EAP [13], in August 2020 the FDA issued an Emergency Use Authorization (EUA) to allow therapeutic plasma treatment of COVID-19 patients outside the context of clinical trials [14]. Additional efforts to develop therapeutic monoclonal antibodies have led to approval of EUAs for antibody cocktails developed by Regeneron [15] and Lilly [16].
Here we describe the development of a semi-high-throughput SARS-CoV-2 neutralization assay that takes advantage of the capabilities of a high-content imaging system to quantify the number of infected cells in individual wells. This assay is devoid of subjective interpretation and more precise than most other wild-type virus neutralization assays. In addition, the assay has been quickly adapted for use with multiple virus variants.
2. Materials and Methods
2.1 Virus and Cells
The 2019-nCoV/USA-WA1-A12/2020 human isolate of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) (Nidovirales: Coronaviridae: Sarbecovirus) was provided by the U.S. Centers for Disease Control and Prevention (CDC; Atlanta, GA, USA). The virus was propagated at the Integrated Research Facility-Frederick in high containment (biosafety level 3 [BSL-3]) by inoculating Vero cells, acquired from the American Type Culture Collection (ATCC #CCL-81; Manassas, VA, USA). The infected cells were incubated for 72 h in Dulbecco’s Modified Eagle Medium with L-glutamine (DMEM; Lonza, Walkersville, MD, USA) containing 2% heat-inactivated fetal bovine serum (FBS; SAFC Biosciences, Lenexa, KS, USA) in a humidified atmosphere at 37°C with 5% carbon dioxide (CO2). The resulting master stock (IRF0394) was quantified by plaque assay using Vero E6 cells (ATCC #CRL-1586) with a 2.5% Avicel overlay and stained after 48 h with a 0.2% crystal violet stain. Working stocks (IRF0395 and IRF0399) were prepared using multiplicity of infections (MOI) of 0.01, harvested after 48 h, and quantified by plaque assay. Virus stocks were sequenced and found to be identical to the published sequence (GenBank #MT020880) for this isolate.
For the neutralization assay, Vero E6 cells (BEI #NR596; Manassas, VA, USA) were plated at a density of 3×104 cells per well in 96-well plates and incubated overnight at 37°C with 5% CO2 so they were approximately 80-90% confluent on the day of infection.
2.2 Sample Dilution
The fluorescence reduction neutralization assay (FRNA) method for measuring neutralizing antibodies was originally developed for Ebola virus and Middle East respiratory syndrome coronavirus (MERS-CoV) [17, 18]. Two changes were made to adapt it to measuring SARS-CoV-2-specific antibodies. First, cell culture dilution media without calcium was used to reduce plasma sample coagulation, particularly when screening plasma samples. Second, statistical evaluations to allow masking of outlier data were developed.
Different positive controls have been used in this assay depending upon the material being tested and its application. Most assays were run using a SARS-CoV-2 antiserum (SAB Biotherapeutics) as a positive neutralizing control, while others used an in-house hyperimmune anti-SARS-CoV-2 immunoglobulin. Virus and antibody-free cell culture medium served as a negative control. During assay development, it was found that there was little difference between using heat-inactivated or untreated serum or plasma samples. Subsequently, in an effort to decrease turnaround time, samples were not heat-inactivated and no complement was added. For standard plasma screening, samples (positive control antibody and test article) were diluted through a six-step two-fold serial dilution (1:40–1:1280) in serum-free DMEM (Gibco #21068028) using a 96-well plate format (Figure 1A). For studies with higher titer monoclonal antibodies, requiring precise calculation of a 50% neutralizing titer (NT50), a 12-step two-fold serial dilution (e.g., 20-40960) was used (Figure 1B). SARS-CoV-2 was diluted in serum-free DMEM to MOI 0.5 (e.g., 15,000 PFU per 30,000 cells). The diluted virus and diluted test samples were mixed 1:1 (vol/vol) and incubated at 37°C in a humidified 5% CO2 atmosphere for 1 h to allow anti-SARS-CoV-2 antibodies to bind the virus. The virus/sample mixtures were then transferred to the wells of a 96-well plate (Greiner Bio-One #655948) containing Vero E6 cells and incubated at 37°C and 5% CO2 for 24 h. For screening activities, samples were tested in duplicate on duplicate plates to allow for four replicates per tested sample. For full plate dilutions, samples were tested as a single replicate per plate on four different plates.
All plasma and serum samples used for developing this assay were deidentified donor samples.
2.3 Cell Staining
After 24 h of incubation, cells were fixed by adding 20% neutral-buffered formalin (NBF) (Thermo Scientific #23-751-800, Kalamazoo, MI) directly to the media for 30 min at room temperature. Plates were stored approximately 24h in 10% NBF at 4°C in accordance with the facility’s safety protocols and then removed from the containment laboratory. Following removal from the laboratory, the NBF was decanted and the cells were washed twice with 1X phosphate-buffered saline (PBS) diluted with purified water from a 10X stock solution (Fisher Scientific #BP3994). The cells were then permeabilized with 0.25% Triton buffer in 1X PBS (Fisher Scientific #PR-H5142) for 5 min at room temperature. Cells were then washed three times with 1X PBS prior to blocking with 3% bovine serum albumin (BSA; Sigma #A7906, Saint Louis, MO) in 1X PBS. Cells were stained with an anti-SARS antibody (SARS-CoV/SARS-CoV-2 nucleocapsid antibody, rabbit monoclonal antibody; Sino Biological #40143-R001), diluted to 0.125 μg/mL in 3% BSA/PBS blocking solution for 1 h at room temperature. The cells were again washed three times with PBS and then stained with an Alexa Fluor 594-conjugated goat anti-rabbit IgG (H+L) highly cross-adsorbed secondary antibody (Thermo Fisher Scientific #A11037), diluted in BSA/PBS for 1 h in the dark at room temperature, and counterstained with Hoechst 33342 nuclear stain (Thermo Fisher Scientific #H3570). The Operetta CLS High-Content Analysis System (Perkin Elmer, Waltham, MA) was used to count the number of virus-infected and non-infected cells in wells containing samples/replicates—specifically those in the four internal (not near well walls) fields with a minimum of 1,000 cells per field.
2.4 Calculation of Standard NT50 Values
For screening relatively low titer samples, a standard dilution scheme of four replicates for each sample were spread across two plates (Figure 1A). The NT50 value was determined individually for each set of two plates based on the virus-positive control wells. To determine the 50% infection rate for a plate, the average of 12 observations of positive control wells was multiplied by 0.50. The output of each step-wise dilution was the average of the four replicates across two plates. The sample dilution output was compared against the calculated NT50 cutoff value for the duplicate plates and the highest dilution to achieve an infection of ≤50% was considered the NT50 titer for the sample. Results are reported as the reciprocal dilution.
2.5 Calculation of NT50 Values by Regression Analysis
For larger dilution series, particularly with known high titer material such as monoclonal antibodies, a more precise calculation was required. Samples were diluted across the plate (Figure 1B) with one replicate per plate in four plates. The fluorescence signal was plotted against the log value of the antibody dilution. A four-parameter logistical analysis was performed on the full dilution series using Prism (GraphPad Software, San Diego, CA). The regression was performed using all four replicates per dilution, and the precise titer was calculated from the regression curve.
2.6 ELISAs
Enzyme-linked immunosorbant assay (ELISA) kits from Euroimmun US, Inc. (#EI 2606-9601 G) and COVID-SeroIndex from USA R&D Systems, Inc. (#DSR200) were used to test anti-SARS-CoV-2 antibody positivity of serum used in FRNA50. The ELISA was performed according to manufacturer instructions.
Briefly, the EUROIMMUN anti-SARS-CoV-2 assay provides a semi-quantitative in vitro determination of human antibodies of immunoglobulin classes IgG. Each kit contains microplate coated with S1 domain of spike protein of SARS-CoV-2. In the first reaction step, diluted patient samples are incubated in the wells. In the case of positive samples, S1-specific antibodies will bind to the viral antigens. To detect the bound antibodies, a second incubation is carried out using an enzyme-labelled antihuman IgG (peroxidase conjugate) catalyzing a color reaction. Results are evaluated semi-quantitatively by calculation of a ratio of the extinction of the control or patient sample over the extinction of the calibrator. This ratio is interpreted as follows:<0.8 negative; ≥ 0.8 to<1.0 borderline; ≥ 1.1 positive. Borderline results were considered positive for analysis [19].
The COVID-SeroIndex Kit from R&D which utilizes a recombinant receptor binding domain (RBD) of the SARS-CoV-2 spike protein. Antigen was pre-coated onto a 96-well microplate in phase 1. When the test sample was added, antibodies specific for the SARS-CoV-2 RBD antigen bind the antigen and are retained in the well. After washing, an enzyme-linked monoclonal antibody specific for human IgG is added to the wells. Following a wash to remove any unbound enzyme-linked antibody, a substrate is added to the wells and color develops in proportion to the amount of IgG antibodies in the sample bound to the SARS-CoV-2 RBD antigen. The color development is stopped, and the intensity of the color is measured. The cut off index is calculated as the ratio of corrected OD of the sample and corrected OD of positive control. The cut off index is interpreted as follows:<0.7 negative; ≥ 0.7 is positive [20].
3. Results
3.1 Immunofluorescence Staining
Development of immunofluorescence assays for Ebola virus and MERS-CoV [18] established fundamental protocols. Early in the COVID-19 pandemic (January 2020), a primary concern was identifying a SARS-CoV-2-specific antibody that reacted with this novel virus. Previous work with MERS-CoV had shown that an N-protein-specific antibody was highly reactive and very specific. Subsequently, an antibody specific for the SARS-CoV N protein proved to cross-react with SARS-CoV-2 with no marked evidence of nonspecific binding to cellular proteins (Figure 2).
3.2 Initial Testing of Human Plasma
Initial sample screening protocols were developed to qualify donor samples for clinical trials supported by the National Institute of Allergy and Infectious Diseases (Clinical Trials NCT04344977, NCT04546581). A donor was eligible if their neutralization titer was ≥1:80. To identify eligible samples, a two-fold serial dilution series was devised to test for titers ranging 1:40–1:1,280. In addition, initial testing evaluated the utility of heat-inactivating the test article and a possible preference for serum or plasma as a preferred matrix for screening activities. This testing found that heat-inactivation did not significantly affect testing, and results between serum and plasma were similar. Thus, in order to retain consistency with the samples that would ultimately be collected from donors and to decrease processing time, subsequent screening was performed on plasma that was not heat-inactivated.
Further testing of plasma, serum, purified IgG, monoclonal antibodies (mAb) and mAb cocktails, polyclonal antibodies and nanobodies has also been completed using this assay. As appropriate for some samples, the assay employed an alternate dilution series (e.g., 1:20– 1:40,960). The fundamental aspects of the assay were unchanged regardless of the dilution series. By February 2021, over 5,000 samples had been screened using this assay.
3.3 Outliers in the Virus Control and Cell Control Observations
Early in development of this assay outlier wells in replicate samples of individual dilutions confounded data analysis. To address concerns with outlier data and to improve plate acceptance criteria, statistical evaluations were used to identify outliers for exclusion from data analyses and to define the number of outliers that could be excluded while retaining the viability of the analysis.
The virus positive control consisted of total of 12 observations (six observations on each of two plates). The virus positive control was used to estimate the NT50, the threshold value used for detecting efficacy of a sample under specific dilution. The cell-only control also consisted of 12 observations, with six observations per plate; it was used to estimate NT100, another threshold value. These FRNA criteria determined whether sample readings for a given dilution were accepted or rejected.
The results of 15 experimental runs (30 plates with 2,280 observations in total) were analyzed. Box plots were produced from 180 virus control and 180 cell control observations (Figure 3 and Figure 4).
Placing control data side-by-side helped to visualize extreme results. For, example, in the virus control box plot results (Figure 3), the measurements of Plate “052020_10-20, Plate 1”, were concentrated close to 100%. Compared to other observations, these results may have indicated that this plate was “over-exposed” with the virus cells and the true measurement could not have been properly estimated. At the same time, extreme values around 0% were observed at plates “052020_17-33_17-34, Plate 2”, “052120_4-7_4-8, Plate 1”, and “052120_4-7_4-8, Plate 2”. The question was whether these observations should be discarded or corrected.
The histogram of the virus control observations (Figure 5) showed concentrations above 60% (mean = 66.8%). The tails of the normal distribution curve were outside of the fixed interval of 0– 100%. To estimate outliers, data were fitted to the beta distribution, which was a more appropriate choice for values in a finite interval. To estimate beta distribution parameters, the method of moments1 was used, which allowed the calculation of alpha and beta distributional parameters using the sample mean and standard deviation. To determine the boundary of acceptable values, 5% and 95% intervals of the beta (2.3, 1.1) were used. Increasing the acceptable region beyond the cutoffs would have increased the chance of accepting experimental errors and bias the sample statistics. Removing virus control observations below 24.7% and above 94.6% (outliers) shifted the sample mean from 66.8% to 67.8%, which improved the precision of detecting the proper dilution ratio (Figure 6).
The histogram of the cell control observations (Figure 7) showed data that were heavily skewed toward 0%. Thus, outliers only on the right-hand side were checked. An estimated 95% boundary of beta (0.3, 7.2)2 or 19.0% as a cutoff interval for control outliers (Figure 8).
When more than three values per plate (out of six) were outside of the acceptable region, it was recommended to discard results of the entire experimental run (both plates). Using a tailored algorithm to handle experimental results (Table 2 and Table 3), only one out of 15 runs was qualified as having faulty controls.
3.4 Detecting Outliers in the Sample Observations
Each level of the sample dilution was repeated four times (twice per plate). To detect potential experimental errors, the Dixon’s Q test was used as a statistical method to quickly detect gross errors in small samples [23]. To detect outliers, an algorithm was used to conduct Dixon’s Q test (Table 4). Here, the critical value of 0.829 was used for a 95% confidence level at N = 4, which is rather conservative. In this cohort of plates, less than 2% of the sample observations were outliers. A tailored algorithm was used for end-to-end sample dilution results processing (Table 5).
3.5 Assay Variability
In order to evaluate interassay variability, a high-titer product was tested 14 times with specific NT50 values calculated. This testing demonstrated an average NT50 of 277 with a standard deviation of 80 (Table 6).
3.5 Specificity
In order to determine the specificity of the FRNA relative to qualitative total IgG ELISAs, a small panel of randomly selected samples of pre-determined titers were tested in the FRNA and two commercially available ELISAs. These comparisons found good agreement between the FRNA NT50 and ELISA positivity (Table 7). However, there was variability in some samples that were weakly positive by FRNA NT50. This variability was likely driven, in part, by the relatively high lower limit of detection (1:40 dilution) used in the FRNA screening assay. If less dilute samples were tested (e.g., 1:10 or 1:20), the FRNA would probably identify borderline positive samples.
4. Conclusions
The FRNA is a specific and highly rigorous evaluation for the presence of neutralizing antibodies in a test sample. Unlike typical neutralization assays, this assay does not rely on the subjective determination of cell cytopathic effects nor the development of multicellular plaques or immunofoci using a low number of infectious particles per well. This assay quantifies individual cells that are infected with the initial addition of virus and does not require multiple rounds of viral replication. Extensive propagation of virus could confound results due to the presence of neutralization escape variants in a viral population and secondary infection of neighboring cells leading to the formation of plaques or foci. The assay described here is also highly quantifiable, with over 4,000 cells counted per individual sample dilution tested, compared to several hundred or fewer quantified plaques or foci in other assay types. While this assay is highly quantitative, it also relies heavily on high-content imaging instrumentation, which may not be available in many laboratory settings.
The inclusion of statistical evaluation of primary data provides additional rigor and is only possible due to the large number of quantifiable events recorded in the assay. The ability to “pass” or “fail” wells or test plates if data fall outside pre-established acceptance criteria provides confidence that determined titers are accurate. The use of 12-step dilution schemes and four-parameter logistical analysis to quantify a specific NT50, rather than reporting data as the reciprocal dilution value, provides a more precise understanding of neutralization capacity, particularly when evaluating monoclonal antibodies or nanobodies. The calculation of NT50 values also allows more accurate extrapolation of NT80 or NT90 values based on the calculated NT50 and the slope of the curve determined from test values.
The use of neutralization data has been critical to identifying potential convalescent plasma donors for hyper-immune intravenous immunoglobulin (IVIg) clinical trials. These trials are evaluating the potential therapeutic benefit of IVIg for treatment of SARS-CoV-2 infection in both hospitalized and ambulatory patients. Further, considerable effort by the World Health Organization (WHO) and others has focused on correlating data from a range of SARS-CoV-2-specific ELISAs to neutralization assays with the objective of understanding both the diagnostic and predictive value of these and other point-of-care diagnostic tools [24-28]. Due to requirements to handle wild-type SARS-CoV-2 in a BSL-3 facility, many laboratories have developed pseudotype-virus neutralization assays using lentivirus, vesicular stomatitis virus, and other platforms [29]. The advantage of the pseudotype systems is that they can be used at BSL-2 using reporter genes, such as luciferase or a fluorescent protein (e.g., NeonGreen) [10, 29, 30]. Disadvantages of such systems are that they generally only contain the SARS-CoV-2 spike protein and that the arrangement and organization of the virus spike on the pseudotyped-virus surface is unlikely to be representative of wild-type SARS-CoV-2. In many cases, such as testing plasma or a polyclonal antibody, the presentation of the spike protein is less of a concern given the array of antibodies. However, when testing monoclonal antibodies, a pseudotype-virus system could provide very different results relative to tests with wild-type virus, particularly if stoichiometric effects or cross-linking between spike proteins is a mechanism used in blocking virus attachment to the ACE2 receptor or in blocking spike trimer rearrangement as a part of the virus fusion process. Thus, any monoclonal antibody screened in a pseudotype virus assay should be validated in a live, wild-type virus assay.
Here, we describe a semi-high throughput highly quantitative neutralization assay for SARS-CoV-2 built around the Operetta high-content imaging system. While the Operetta is the preferred platform in our facility, similar equipment could be equally effective if appropriately validated. While there will always be variability in live-virus assays due to the nature of the biological system, using a rigorous statistical approach to inform acceptance of data can mitigate the potential negative effects of poor infection efficiency, pipetting errors, edge effects, and inconsistent staining.
Funding
This project has been funded in whole or in part with Federal funds from the National Institute of Allergy and Infectious Diseases (NIAID), National Institutes of Health (NIH), U.S. Department of Health and Human Services (DHHS), under Contract No. HHSN272201800013C. R.S.B., J.L., R.G., S.M., S.D., G.K., S.Y., S.G.-C., D.G., L.M., and M.R.H. performed this work as employees of Laulima Government Solutions, LLC while E.N.P. and Y.C. performed this work as an employee of Tunnell Government Services. V.V.L. provided statistical support pro bono. The content of this publication does not necessarily reflect the views or policies of the DHHS or of the institutions and companies with which the authors are affiliated.
Author Contributions
Conceptualization, E.P. and R.S.B.; methodology, E.P., J.L., R.G., D.G., S.G.-C., S.M., S.D., L.M., G.K., Y.C., S.Y., and V.V.L; formal analysis, E.P. and J.L.; resources, M.R.H.; data curation, R.G., J.L., and M.R.H.; writing—original draft preparation, R.S.B., E.P., J.L., and V.V.L.; writing—review and editing, M.R.H.; supervision, M.R.H.; project administration, M.R.H. All authors have read and agreed to the published version of the manuscript.
Data Availability
Data associated with this study are included in the manuscript. Additional primary data can be provided by request.
Conflicts of Interest
The authors declare no conflicts of interest.
Acknowledgments
The authors express our gratitude to Hua Wu, Ph.D. at SAB Biotherapeutics for providing SARS-CoV-specific serum that was used as a positive control in these studies. We thank the staff of NIAID and the Integrated Research Facility at Fort Detrick (IRF-Frederick) who contributed to this project. We would also like to thank Anya Crane (NIH NIAID Division of Clinical Research [DCR] IRF-Frederick, Frederick, MD, USA) for editing the manuscript.
Footnotes
↵1 Equations for alpha and beta using method of moments: is the sample mean and s2 is the sample variance (Owen, 2008). For cases in which or β is less than one, it is preferred to use an alternative maximum-likelihood estimation method of estimating these parameters. In those cases, we used the scipy.stats.beta.fit() command, a beta-fitting function in the statistical library within Python 3.6 2 [21,22].
↵2 The numbers were rounded to one decimal digit, which may have resulted in a slightly different cutoff value.