RT Journal Article
SR Electronic
T1 Dynamic allocation of computational resources for deep learning-enabled cellular image analysis with Kubernetes
JF bioRxiv
FD Cold Spring Harbor Laboratory
SP 505032
DO 10.1101/505032
A1 Dylan Bannon
A1 Erick Moen
A1 Morgan Schwartz
A1 Enrico Borba
A1 Sunny Cui
A1 Kevin Huang
A1 Isabella Camplisson
A1 Nora Koe
A1 Daniel Kyme
A1 Takamasa Kudo
A1 Brian Chang
A1 Edward Pao
A1 Erik Osterman
A1 William Graf
A1 David Van Valen
YR 2019
UL http://biorxiv.org/content/early/2019/09/15/505032.1.abstract
AB Deep learning is transforming the ability of life scientists to extract information from images. These techniques have better accuracy than conventional approaches and enable previously impossible analyses. As the capability of deep learning methods expands, they are increasingly being applied to large imaging datasets. The computational demands of deep learning present a significant barrier to large-scale image analysis. To meet this challenge, we have developed DeepCell 2.0, a platform for deploying deep learning models on large imaging datasets (>105-megapixel images) in the cloud. This software enables the turnkey deployment of a Kubernetes cluster on all commonly used operating systems. By using a microservice architecture, our platform matches computational operations with their hardware requirements to reduce operating costs. Further, it scales computational resources to meet demand, drastically reducing the time necessary for analysis of large datasets. A thorough analysis of costs demonstrates that cloud computing is economically competitive for this application. By treating hardware infrastructure as software, this work foreshadows a new generation of software packages for biology in which computational resources are a dynamically allocated resource.