Abstract
Microplates are indispensable in large-scale biological experiments but the layout of samples and controls can have a large effect on results. Here we introduce an artificial intelligence based method for designing microplate layouts that reduces unwanted bias and limits the impact of batch effects, leading to more accurate and reliable experimental results. The method relies on constraint programming, and produces effective multiplate layouts for different experimental settings, while at the same time remaining flexible and modifiable to take into account particular laboratory settings. First we discuss the desired properties of effective microplate layouts, which we then implement as a constraint model. We show that our method produces layouts that lead to smaller errors in dose response experiments when estimating EC50/IC50 values, to the point of frequently obtaining smaller errors even when using fewer doses or replicates. We also show how effective layouts lead to more robust results in high-throughput screening experiments. Finally, we make our method easily accessible by providing a suite of tools, an online service (PLAID), a data format to enable automated construction of designs programmatically, and notebooks to evaluate and compare designs aiding decisions on the number of doses and replicates when planning microplate experiments.
Main
In the era of data-driven life science, the amounts of data produced are continuously expanding, and artificial intelligence techniques such as machine learning algorithms and constraint programming [1] are seeing adoption for many applications in order to convert the data into actionable insights [2, 3, 4, 5, 6]. While in many applications the primary focus has been to obtain as much data as possible, the importance of having data of high quality cannot be understated [7, 8, 9]. For large-scale biological experiments, many issues related to data quality pertaining to human operations can be effectively reduced or eliminated by using automated setups and robotised equipment [10]. However, several artefacts due to physical, biological, and temporal conditions still remain, and efforts generating large quantities of data can be fruitless if in the end conclusions cannot be drawn due to data-quality issues. A common approach to increase the confidence in the data is to perform multiple technical and biological replicates, but this is associated with higher costs and longer experiments, and often leads to a trade-off between the number of samples analysed and the number of replicates per sample. A better approach would be to improve the experimental design [11].
Microplates, or microwell plates, are standard components in many biological experiments. They are flat plates with multiple ‘wells’ used as small test tubes, organised in a 2:3 matrix. Common sizes are 24, 48, 96, 384 and 1536 wells, but smaller sizes also exist. Experiments carried out using microplates commonly exhibit plate effects [12], also known as positional effects, which are systematic variations across the geometry of the microplate due to factors such as well location, temperature and humidity being unequally distributed, and can affect the results to the point of rendering the experiment unusable. Using effective microplate layouts is one way to greatly improve the quality of the experimental design and, in turn, the quality of data obtained from such experiments [13]. Traditionally, microplate layouts have been designed manually using border layouts, that is, by placing controls in the outer-most wells, commonly in the left-most and rightmost columns, while samples are distributed following patterns that are easy to design and to pipette manually [14, 15]. Indeed many researchers still use border layouts as they help reduce human pipetting errors, allow for straight-forward plate visualisation by humans, for example in the form of heat maps [16], and can be easily designed using pen and paper [17]. Yet border layouts can only be used to effectively identify and adjust for a few plate effects [13, 18].
Considering the widespread availability of automation equipment [11, 13], including pipetting robots that can arrange controls and compounds anywhere on a plate, scientists have been advocating for the use of randomised plate layouts for many years [19]. Although there are many plate layout editors freely available, such as Brunn [20], FlowJo [21], Labfolder [22], PlateDesigner [23], and PlateEditor [24], some of which offer the option of randomised microplate layouts, they are not easily customisable, and generating new random layouts for each plate can be very challenging without advanced programming skills aligning liquid handlers and data analysis software. On top of that, even though randomised microplate layouts have been shown to perform well in practice and are currently considered the state-of-the-art [19], pure randomisation can still produce ineffective layouts, given that, for example, technical replicates may be randomly placed in adjacent wells which are then likely to be affected by the same plate effects. Consequently, plate designs that distribute both controls and samples in a statistically secured way are needed in order to reduce unwanted bias, as well as to detect and correct plate effects, leading to higher quality data being obtained from the assay. We refer to such designs as effective layouts.
In this manuscript we introduce an artificial-intelligence based model for designing effective microplate layouts that can easily be adapted for different experimental settings. In order to simplify usage, we developed a set of online tools, including a web-app for easily designing effective microplate layouts, together with Python notebooks for simulating different experimental settings and allow for planning and designing effective experiments.
Considerations when designing microplate layouts
Plate effects are frequent in microplate experiments [12], and can greatly affect experimental results. Common patterns of systematic experimental errors include:
(i) linear row effects; (ii) linear column effects; (iii) linear row and column effects; and (iv) bowl-shaped spatial effects [12], which can be seen in Figure 1 (left) and Supplementary Figure 1. Identifying and correcting plate effects is important in order to adjust the data so that the impact of the errors can be reduced or removed altogether. Various normalisation techniques have been developed to this end [16, 18], but an effective microplate layout is of particular importance for the normalisation to be effective [13, 16].
Examples of the distribution of 20 negative control in layouts for 384-well microplates. The colors indicate the intensity measured at each well. Top: data with strong systematic errors having a bowl-shaped relationship to well position. Bottom: data with strong systematic errors having a linear relationship to column number.
When it comes to detecting and correcting plate effects, the state-of-the-art recommendation for a layout is a complete randomisation of samples and controls [19]. In fact, most normalisation and correction methods rely on samples and controls being randomly distributed across the microplate [18]. Nevertheless, pure randomisation can still lead to, for example, large areas of the plate not having any controls, making it very difficult or even impossible to detect and correct errors in those areas [19, 25, 26]. For example, in Figure 1 (top row), we show a heat map of a microplate with strong systematic errors having a bowl-shaped relationship to well position, as well as the locations of 20 negative controls in 3 types of layouts: border, random, and Effective. Based on the intensity of the negative controls in a border layout, it is impossible to detect what kind of plate effect might be present on the plate. A random layout shows an obvious improvement as the controls are more distributed over the plate, but with an effective positioning of the negative controls obtained with an Effective layout, the strong bowl-shaped systematic errors become apparent even to a human eye.
The uneven distribution of controls is not the only problem with fully randomised plates. For example, in a dose-response experiment similar concentrations may be placed in adjacent wells, leading to a higher risk of them being similarly biased and affecting the regression accuracy negatively [19]. The same can be said about technical replicates in a screening experiment. Not only is it a problem that they will be similarly biased, it has also been shown that clusters of similar samples including similar doses of the same compound as well as technical replicates, can affect the results of adjacent wells [16]. There is, of course, the possibility of generating several random layouts and then evaluate them in order to select the best one [27], but that does not guarantee that effective plate layouts have been selected, regardless of how many layouts are generated.
Effective microplate layouts
In order to generate effective microplate layouts that are superior to random layouts, we first need to define their desired properties. The following properties are not meant to be exhaustive, and should be adapted for specific applications and experimental settings.
Distribution of control samples
In order to maximise the usefulness of positive and negative control samples, we would like to distribute the controls evenly among the wells of the microplate. For example, on each microplate, we could constrain the number of controls to be equally distributed among each of its four quadrants, that is, the difference in the number of controls between any two quadrants would be at most 1. This could be applied to either individual types of controls, the total number of controls, or both. The number of controls could also be evenly distributed between the upper and lower halves, as well as between the right-most and left-most halves of the microplate. Moreover, controls could also be evenly distributed across rows and columns, that is, the difference in the number of controls between any two rows, or any two columns, would be at most 1. As before, this could apply to both individual types of controls, the total number of controls, or both. The latter properties would be particularly useful when determining the existence of plate effects linked to row or column number. Furthermore, controls of the same type should, ideally, not be placed on adjacent wells. Whenever feasible, we would also want controls of any kind not to be placed in adjacent wells. This would contribute to both getting a better picture of the systematic effects across the whole microplate, as well as reducing clusters of similar samples, for instance, two or more positive controls located next to each other.
Distribution of samples
With the goal of mitigating grouping effects we can, for instance, enforce that the replicates of a sample are placed on different rows and columns, which would also contribute to reducing experimental bias by exposing them to different conditions across the plate. Similarly, we can consider specific kinds of experiment, for example a dose-response experiment [28], which aims to determine the relationship between the dose applied and the effect being observed. In this context we could enforce that for each compound, the difference in the number of individual doses between any two rows, or any two columns, is at most 1. Moreover, the doses of each compound could be divided between the upper and lower halves of the plate, as well as between the left and right halves of the plate. Furthermore, spreading doses this way helps to ensure that, in the case of losing some doses due to, for example, missing a row or a column due to pipetting problems, enough of the other doses will remain.
Edge effects
Also known as border effects, are discrepancies between the centre and the outer wells of a microplate primarily caused by evaporation, and that can greatly affect the results obtained from an experiment [29]. A common method to avoid edge effects is to avoid having samples in the outermost rows and columns, and instead fill them with medium [30]. Another method is to use specific edgeeffect correction methods that can be applied to particular types of assays [29]. Depending on the type of experiment and preferred way of dealing with the edge effect, one could enforce the number of outer rows and columns that should be left empty, if any.
Empty wells
If there are remaining empty wells after placing all samples (other than those in the edge), they should be distributed in a manner similar to controls. This way, empty wells can help avoid clusters of samples and controls.
Multi-plate experiments
In order to make a multi-plate experiment more robust, we could for example evenly distribute the controls across all plates in such a way that for each type of control, the difference in number between plates is at most 1. Across all plates, controls could also be balanced between the upper and lower halves of the plates, as well as the left-most and right-most halves of the plates. For example, in an experiment with two microplates, a control that has 3 replicates on each microplate, would have 2 replicates on the upper half and 1 replicate in the lower half of one plate, while on the other plate there will be only 1 replicate in the upper half and 2 replicates in the lower half. This could apply to either each type of control, the total number of controls, or both. Moreover, we could balance the controls per row or column across all plates, that is, the difference between the number of controls in any two rows or columns across all plates is at most 1. Given enough controls, this can help ensure that potential plate effects effect linked to any row or column will be detected, especially when the errors have been introduced consistently in all plates, for example by a faulty dispensing equipment. With the goal of reducing positional bias, we could also enforce that the replicates of a sample are placed on different rows and columns, even if they are on different plates.
Implementing constraints
After characterising effective plate layouts, we need a way to generate microplate layouts that actually fulfil the criteria. At first glance, one option would be to randomly generate microplate layouts until one that fits the criteria is generated. Although this sounds like an easy solution, writing the program for validating a layout might require advanced programming skills, and more importantly, randomly generating layouts offers no guarantee that a layout fitting the criteria will ever be found. In fact, if there is no plate layout fulfilling the criteria, this program would never finish. In order to guarantee that the program does finish, we could try solutions one by one in an orderly fashion, but there are simply too many possible ways of filling in the wells of a microplate to be able to find a solution by trial and error within reasonable time. We need to find an efficient way to generate effective microplate layouts that are guaranteed to fulfil all of the desirable properties we want, while also guaranteeing completeness, that is, if a solution exists it will be found in a finite amount of time. One way of doing this is to frame our characterisation of effective microplate layouts as a constraint satisfaction problem: we view each well of each plate as a variable whose value represents its content, and desirable properties of a layout as a constraint. Constraint programming is a subarea of artificial intelligence that offers a flexible framework for solving constraint satisfaction problems.
Effective layouts for dose-response assays
Dose-response experiments are commonly performed to determine the effect of e.g. compound-treated cells at different concentrations [28]. The result is typically a sigmoid curve, which is frequently summarised by determining the half maximal inhibitory concentration (IC50), or the half maximal effective concentration (EC50). In order to evaluate the impact of different types of microplate layouts in doseresponse experiments, we simulated more than 40000 microplates for dose response experiments with diverse numbers of doses, replicates, compounds of varying potency, as well as different types and strengths of plate effects. Examples of the resulting curves can be seen in Figures 2a and 3a.
Comparison between expected and obtained values for dose response curves with 8 doses, 1, 2, or 3 replicates, 20 negative controls on 384-well plate, and strong bowl-shaped plate effects. * indicates p < 10−41, ** indicates p < 10−12, *** indicates p < 10−43.
Comparison between expected and obtained values for dose response curves with 8 doses, 1, 2, or 3 replicates, 20 negative controls on 384-well plate, and strong plate effects with a linear relationship to column number on the right side of the plate. * indicates p < 10−4, ** indicates p < 10−12, *** indicates p < 10−43.
Higher-quality data
Figures 2d and 3d show the mean of the residuals calculated with respect to the dose-response curves used to generate the data. It is clear that, after error correction and normalisation, the data obtained using our effective layouts is of much higher quality, that is, much closer to their expected values, than the data obtained when using either random and border layouts.
Better approximation of curves
It is standard practice to discard doseresponse curves that are considered to have low quality, for example, curves where more than 20% of the variability is unexplained by the curve fit, that is, with R2 < 0.8 [31]. In general, our effective layouts lead to a higher percentage of high-quality curves. For example, in the case of experiments with 8 doses and 3 replicates, and strong plate effects with a linear relationship to column number on the right-half side of the plate, all curves generated using our effective layouts have an R2 0.8, while only 94% of the curves generated using random layouts and 70% of the curves generated using border layouts have a good curve fit with R2 0.8. Moreover, there is a significant difference between the various types of layouts in the absolute difference between the maximum value of the expected and obtained curves as can be seen in Figures 2e and 3e.
More accurate relative IC50/EC50
As can be seen in Figures 2b and 3b, there is a significant difference between using an effective layout compared to using either a random or a border layout, regardless of the number of replicates used (p < 10−4 for all pairwise comparisons, t-test). Similar results are obtained for other strengths of plate effects, as well as when using 6 or 12 doses (see Supplementary Figures 6 and 7).
More accurate absolute IC50/EC50
As can be seen in Figures 2c and 3c, there is a significant difference between using an effective layout and either a random or a border layout regardless of the number of doses and replicates used (p < 10−43 for all pairwise comparisons, t-test). Similar results are obtained for other plate-effect strengths, as well as when using 6 or 12 doses (see Supplementary Figures 3, 4, and 5). Also note that it is not always possible to estimate the absolute EC50/IC50. For example, in the case of experiments with 8 doses and 1 replicate, the absolute EC50/IC50 of almost 1% of the curves could not be estimated when using border layouts in the presence of strong bowl-shaped effects. This number grows to 13.4% when the negative controls are not included as data points.
Replicates vs. doses
Inline with the recommendations in [32], replicates do improve precision, but not enough to address systematic bias. In general, adding more doses had a higher impact in the estimations than adding more replicates, regardless of the layout. In particular, note that our effective layouts lead, in many cases, to more accurate results even with fewer replicates or fewer doses. For example, we obtained more accurate estimations of absolute EC50/IC50 for experiments with 8 doses and 2 replicates using our effective layouts than with 8 doses and 3 replicates using random layouts (see Figures 2c and 3c). Moreover, we also obtained more accurate estimations of absolute EC50/IC50 for experiments with 8 doses and 3 replicates using our effective layouts, compared to 12 doses and 3 replicates using random layouts (see Supplementary Figures 3, 4, and 5).
Evaluating effective layouts in high throughput screening assays
In a high throughput screening (HTS) assay, the goal is to extract biological significance from large amounts of data [13]. In order to evaluate the effect of different types of layouts in HTS experiments, we simulated more than a thousand 384-well microplates with 10 positive controls and either 10 or 20 negative controls, and various strengths of bowl-shaped plate effects.
Higher-quality data
We can see in Figure 4 that, after error correction and normalisation, the residuals obtained using effective microplate layouts are smaller than those obtained with other types of layouts. Effective microplate layouts lead to higher quality data, even in the case when some rows are completely lost, e.g. in the case of equipment malfunction or dispensing errors. Similar results are obtained for plates with only 10 negative controls and various strengths of bowlshaped plate effects (see Supplementary Figures 12a, 13a, and 14a).
Mean residuals for microplates with 10 positive controls and 20 negative controls on 384-well plates with strong bowl-shaped plate effects and varying numbers of rows lost. ** indicates p < 10−25, *** indicates p < 10−41.
Better data separation
As can be seen in Figures 5d-5f, tested samples with positive and negative effects have a better separation when using effective layouts, compared to both random and border layouts. Similar results are obtained for experiments with only 10 negative controls and varying intensities of plate effects (see Supplementary Figures 12, 13, and 14).
Comparison between expected and obtained values for screenings experiments using 10 positive and 20 negative controls on 384-well plate with strong bowl-shaped plate effects.
PLAID ecosystem
Plate scores
Low plate scores indicate low-quality results, but high plate scores do not necessarily guaranty good-quality results. As seen in Figures 5a and 5b, border layouts result in higher Z’ factor and SSMD scores before error correction normalisation (indicated as raw) for the experiments in Figure 5. After error correction and normalisation, microplates using border layouts still have scores comparable to those obtained using effective and random layouts, even though there is no separation between samples with positive and negative effects. This occurs partly because the both Z’ factor and SSMD scores only take into account positive and negative controls, which are in fact nicely separated for most of the plates using a border layout. Moreover, microplates with effective layouts had the lowest plate scores before error correction and normalisation, as well as the highest scores afterwards. Nevertheless, this is not always the case and we have observed results for which any of the types of layouts has the highest scores (Supplementary Figures 12a-b, 13a-b, and 14a-b). However, based on the observed smaller residuals (Supplementary Figures 9, 10, and 11) and visualisation of the results (Supplementary Figures 12, 13, and 14), we believe that our effective layouts lead to more robust and meaningful plate scores.
Discussion
We identified properties of effective microplate layouts and used them to build a constraint programming model capable of designing such layouts. We then showed that our effective microplate layouts improve experimental results, both in doseresponse and screening experiments. In every comparison, our effective microplate layouts performed significantly better, and in particular, we showed that effective microplate layouts produced more accurate results, in many cases even with fewer negative controls, replicates, or doses. These results aligns with the idea that higher-quality data is generally better than larger quantities of data, as well as with the guidelines proposed in [32] for experimental design. Being able to reduce the number of controls, replicates, or doses can have a significant impact in terms of time, costs, and number of samples evaluated in any type of experiment.
In the context of dose-response experiments, we also showed that adding more doses has a higher impact in the quality of the final results than adding more replicates. Moreover, in the context of HTS experiments, we showed that our effective layouts lead to higher-quality data and a better differentiation between samples with a positive effect (hits) and samples without any effect.
After observing the positive impact that our effective layouts have on the residuals and data-quality in both dose-response and HTS experiments, we believe that our effective layouts will also prove to be useful in other experimental settings.
Even though our error model is based on the literature [12], the intensity and type of plate effects observed might differ depending on factors like the type of experiment, laboratory facilities, temperature, among others. In the case that, for example, our error model is considered too strong, then layouts can be designed with fewer controls, further reducing costs, number of plates, time, etc. Our simulation experiments can be adapted to consider other strengths and types of plate effects, to be used during the design phase of an experiment.
We are aware that the prevalent use of border layouts is partly due to the fact that it is easier for humans to visualise the results directly on a plate layout format. Nevertheless, it is possible to, for example, visualise particular samples on a plate while hiding the rest (as in Figure 1), as well as to transform the results from one layout into another. For example, we provide one way to do so using spreadsheets as part of our supplementary material and our GitHub repository. Another reason is that plate randomisation is sometimes considered to be too challenging without the programming skills required to align liquid handlers and data analysis software [19]. We provide a web interface to ease the design step, which then generates a simple .CSV file with the plate layouts. We also provide a way to transform such file into the specific format needed for I.DOT pipetting robot using spreadsheets as part of our supplementary material and our GitHub repository.
Another common practice is to use a small set of layout templates. For example [13] presents a handful of good templates for distributing a fixed number and types of controls, but the templates do not include a way of placing all other sam-ples. Another draw back of using templates is that they restrict the experimental design. For example, it is not trivial to adapt them for a different amount of controls. In the case of a dose-response assay, a template cannot be easily modified to have a different number of doses or replicates. In contrast, it is possible to use our constraint model to design effective templates for any particular experimental design. We also believe that experimental design must keep up with technological advancements. As robotised equipment improves and new sizes of microplates are developed, layouts and templates should adapt.
Our implementation of the constraint model that creates effective layouts can be easily integrated into other existing workflows. At the same time, the constraint model may also be extended to be able to accommodate other specific types of constraints and experiments. For example, different laboratory settings can lead to different constraint models if a particular dispensing error is frequently observed. We also aimed to design a general model of effective layouts, but particular assays might benefit from specific constraints, as it is the case when spreading doses of the same compound in dose-response assays. We hope that the availability of the source code will promote such usage and speed up the implementation of new features.
Looking forward, our ultimate goal is to design an automated robotic laboratory system capable of iteratively designing experiments, executing them, evaluating them, and based on the results, repeating the full cycle again. In this context, our model for designing effective microplate layouts plays a key role in obtaining high-quality data from such automated system with minimum human intervention.
Methods
Constraint programming
Constraint programming (CP) [1] is a form of artificial intelligence used for modelling and solving combinatorial problems, which is currently successfully used in many real-world application areas such as scheduling [33, 34, 35], decision support [36], and packing [37].. Solving a combinatorial problem involves finding an assignment for a discrete, finite set of objects (decision variables) that satisfies a given set of conditions (constraints). The general idea behind constraint programming is that the user specifies the constraints that should hold among decision variables and a general-purpose constraint solver is used to find a solution. That is, the user specifies the problem without having to specify how to find a solution. For example, consider our microplate layout design problem. Each unknown in the problem, namely the content of each well on each plate, is a decision variable. Each decision variable Vi can take values in a given domain, denoted dom(Vi). In our microplate layout design problem, the domain of each decision variable is the set of possible substances to place on a well, i.e. a given compound at a certain concentration, a positive control, etc. Moreover, problem solutions are distinguished from non-solutions by constraints, which are the limitations to the values that the decision variables can take simultaneously. In this context, a constraint is, for example, a limitation that controls of the same type cannot be placed in contiguous wells.
In order to find a solution for a given problem, a constraint solver first removes infeasible values from the domains of the variables by applying inference methods, which is known in the literature as propagation. Then, the search for a feasible solution is performed in a branch-and-bound fashion: the left-most branch corresponds to a sub-problem that is created by assigning a value v dom(Vi) to a variable Vi. If the sub-problem turns out to be infeasible, a backtracking mechanism is used to try other sub-problems where the additional constraint dom(Vi) = v is added.
In general, constraint satisfaction problems are specified by data-independent models written in a modelling language such as AMPL [38], Essence [39], MiniZinc [40], or OPL [41].
Designing effective microplate layouts using constraint programming
We implemented a constraint model representing the microplate layout design problem in MiniZinc [40] and used Gecode [42] as the backend constraint solver. One of the many advantages of using MiniZinc is that only very minor modifications, if any, would be needed to use another constraint solver.
On top of including all the good properties of effective microplate layouts, we have chosen to include other constraints that are needed for practical matters. For example, we enforce that for each compound, all concentration levels of a given replica must appear on the same plate. Technical replicates of a compound can be chosen to appear on the same plate, on a different plate, or a mixture of both. We have also included the dimensions of the microplate as parameters in terms of number of rows and columns, allowing the use of any kind of plate size. Finally, it is also possible to specify how many rows and columns should be left empty on the border of every microplate in order to mitigate the edge effect.
Study design for dose-response experiments
We simulated multiple scenarios for dose response experiments according to [31]. The following scenarios were considered: all combinations of compounds having: (i) a sigmoid curve with slopes of 0.5, 1, 1.5, and 2; (ii) 6 concentrations with a dilution factor of 18, 8 concentrations with a dilution factor of 8, and 16 concentrations with a dilution factor of 4; and (iii) 1, 2, and 3 replicates per compound. Without loss of generality, for every compound the bottom of the curve was set to 0%, and the top of the curve was set to 100%. Fixing the top and bottom of the curve at these values makes the assumption that if a sufficient number of concentrations were to be used, a complete dose–response curve would be generated. To generate the sigmoid curves corresponding to each compound the only parameter remaining to be specified is the EC50/IC50. We generated curves with EC50/IC50 values ranging from 1 to 90 to simulate compounds having all kinds ± of potency. The highest concentration was arbitrarily set to 100 µM. For each test concentration, the replicates were generated by adding a random value within 1% to the value sampled from the curve in order to represent a very small error in measurement between wells having the same compound in the same concentration. Border layouts were designed by placing 20 negative controls in columns 2 and 23, and all other samples were placed horizontally from top to bottom. Random layouts were generated using the Python random package. We generated effective microplate layouts using our constraint programming model implemented in MiniZinc [40]. The Python functions and MiniZinc model used to generate the plates are included in the Supplement. The exact layouts tested are also included in the Supplement as NPY files. We then applied one plate effect to every plate having either: (i) a bowl-shape relationship to well position, (ii) a linear relationship to column number on the right-hand side of the plate, or (iii) a diagonal-shape relationship to well position. After applying plate effects, we correct the data using linear regression in the case of border layouts, and LOESS regression as implemented in [43] for the rest, and normalised the data as a percentage of the average of the negative controls. Finally, we use the data to estimate relative and absolute EC50/IC50. For each dose-response curve, we calculated the absolute value of the difference between the log10 of the true and the estimated EC50/IC50 values. Moreover, for every measurement, we calculated the residuals with respect to both the actual as well as the estimated curves.
Study design for high throughput screening experiments
We simulated a HTS experiment using 384-well microplates with 10 positive controls and either 10 or 20 negative controls. The remaining wells contained random compounds, two thirds with very low (negative) effects and one third with very high (positive) effects. We applied various strengths of bowl-shaped effects to every plate. After applying the plate effect, we calculated the raw Z’ factor and the raw SSMD of each plate. We then corrected the data using linear regression in the case of border layouts, and LOESS regression in the case of random and effective layouts, and normalised the data as a percentage of the average of the nearest negative controls. Finally, we used the data to calculate the Z’ factor and the SSMD of each plate after error correction and normalisation.
Data availability
The Python libraries and notebooks developed for the analysis, the experimental results, and the specific microplate layouts tested are available on GitHub at https://github.com/pharmbio/plaid.
Code availability
We have developed and made available a range of tools that can be used to design and evaluate microplate layouts under a wide range of conditions.
The constraint programming model
The constraint model, together with example files and scripts, is maintained at https://github.com/pharmbio/plaid. The model can be ran using the MiniZinc IDE, as well as in the command line. It is also possible to incorporate it into existing tools, for example, using the MiniZinc Python package.
The web interface
We implemented a web interface for our constraint model using React, and it is available inside a docker container at https://github.com/pharmbio/plaid-gui. It is possible to test it at https://plaid.devserver.pharmb.io/.
Analysis and visualisation notebooks
The analysis was done in Python notebooks, which are available on GitHub at https://github.com/pharmbio/plaid. The repository also contains a library of plate effects, error correction and normalisation methods, as well as some visualisation functionality. The notebooks can be used to evaluate experimental designs, for example, to explore the effect of varying the number of controls, doses, replicates, etc.
Author information
Contributions
MAFR and OS: Project conceptualisation; MAFR: Method development and implementation, tool development; MAFR, OS, JCP: Analysis and interpretations, manuscript preparation.
Ethics declarations
Competing interests
The authors declare no competing interests.
Acknowledgments
This project received funding from the Swedish Research Council (grants 2020-03731 and 2020-01865), FORMAS (grant 2018-00924), the Swedish Foundation for Strategic Research (grant BD15-0008SB16-0046), and the Swedish strategic research programme eSSENCE.
We thank Wesley Schaal for constructive feedback on manuscript, Markus Lucero and Travis Persson for contributions to the PLAID web interface, Polina Georgiev for constructive feedback on microplate layouts, and Gustav Björdal for constructive feedback on the constraint model.
Footnotes
jordi.carreras.puigvert{at}farmbio.uu.se
ola.spjuth{at}farmbio.uu.se
