A balanced measure shows superior performance of pseudobulk methods over mixed models and pseudoreplication approaches in single-cell RNA-sequencing analysis

Summary

Recently, Zimmerman et al.,¹ highlighted the importance of accounting for the dependence between cells from the same individual when conducting differential expression analysis on single-cell RNA-sequencing data. Their work proved the inadequacy of pseudoreplication approaches for such analysis – This was an important step forward that was conclusively proven by them. A hierarchical single-cell expression simulation approach (hierarchicell) was developed by Zimmerman et al.,¹ to generate non-differentially expressed genes upon which performance was evaluated using the type 1 error rate; the proportion of non-differentially expressed genes indicated as differentially expressed by a model. However, evaluating such models on their type 1 or type 2 error rate in isolation is insufficient to determine their true performance – for example, a method with low type 1 error may have a high type 2 error rate. Moreover, because no seed was set for the pseudo-random number generator used in hierarchicell, the different methods evaluated by Zimmerman et al. were done so on different simulated datasets. Here, we corrected these issues, reran the author’s analysis and found pseudobulk methods outperformed mixed models.