We study why overparameterization—increasing model size well beyond the point of zero
training error—can hurt test error on minority groups despite improving average test error …

Overparameterized neural networks can be highly accurate on average on an iid test set yet
consistently fail on atypical groups of the data (eg, by learning spurious correlations that …

Language models are generally trained on data spanning a wide range of topics (eg, news,
reviews, fiction), but they might be applied to an a priori unknown target distribution (eg, …

AI is undergoing a paradigm shift with the rise of models (eg, BERT, DALL-E, GPT-3) that are
trained on broad data at scale and are adaptable to a wide range of downstream tasks. We …

Standard training via empirical risk minimization (ERM) can produce models that achieve low
error on average but high error on minority groups, especially in the presence of spurious …

Distribution shifts—where the training distribution differs from the test distribution—can
substantially degrade the accuracy of machine learning (ML) systems deployed in the wild. …

For machine learning systems to be reliable, we must understand their performance in
unseen, out-of-distribution environments. In this paper, we empirically show that out-of-distribution …

We introduce OpenFlamingo, a family of autoregressive vision-language models ranging
from 3B to 9B parameters. OpenFlamingo is an ongoing effort to produce an open-source …

… The project was initiated by Shiori Sagawa, Pang Wei Koh, and Percy Liang. Shiori Sagawa
and Pang Wei Koh led the project and coordinated the activities below. Tony Lee developed …

Abstract Models trained on one set of domains often suffer performance drops on unseen
domains, eg, when wildlife monitoring models are deployed in new camera locations. In this …