The interactions between antibodies, SARS-CoV-2 and immune cells contribute to the pathogenesis of COVID-19 and protective immunity. To understand the differences between antibody responses in mild versus severe cases of COVID-19, we analyzed the B cell responses in patients 1.5 months post SARS-CoV-2 infection. Severe and not mild infection correlated with high titers of IgG against Spike receptor binding domain (RBD) that were capable of viral inhibition. B cell receptor (BCR) sequencing revealed two VH genes, VH3-38 and VH3-53, that were enriched during severe infection. Of the 22 antibodies cloned from two severe donors, six exhibited potent neutralization against live SARS-CoV-2, and inhibited syncytia formation. Using peptide libraries, competition ELISA and RBD mutagenesis, we mapped the epitopes of the neutralizing antibodies (nAbs) to three different sites on the Spike. Finally, we used combinations of nAbs targeting different immune-sites to efficiently block SARS-CoV-2 infection. Analysis of 49 healthy BCR repertoires revealed that the nAbs germline VHJH precursors comprise up to 2.7% of all VHJHs. We demonstrate that severe COVID-19 is associated with unique BCR signatures and multi-clonal neutralizing responses that are relatively frequent in the population. Moreover, our data support the use of combination antibody therapy to prevent and treat COVID-19.