Redesigning the Eterna100 for the Vienna 2 folding engine

Abstract

The rational design of RNA is becoming important for rapidly developing technologies in medicine and biochemistry. Recent work has led to the development of several RNA secondary structure design algorithms and corresponding benchmarks to evaluate their performance. However, the performance of these algorithms is linked to the nature of the underlying algorithms for predicting secondary structure from sequences. Here, we show that an online community of RNA design experts is capable of modifying an existing RNA secondary structure design benchmark (Eterna100) with minimal alterations to address changes in the folding engine used (Vienna 1.8 updated to Vienna 2.4). We tested this new Eterna100-V2 benchmark with five RNA design algorithms, and found that neural network-based methods exhibited reduced performance in the folding engine they were evaluated on in their respective papers. We investigated this discrepancy, and determined that structural features, previously classified as difficult, may be dependent on parameters inherent to the RNA energy function itself. These findings suggest that for optimal performance, future algorithms should focus on finding strategies capable of solving RNA secondary structure design benchmarks independently of the free energy benchmark used. Eterna100-V1 and Eterna100-V2 benchmarks and example solutions are freely available at https://github.com/eternagame/eterna100-benchmarking.