2018Heymann Challenge

Citation

Heymann, J. B.; Marabini, R.; Kazemi, M.; Sorzano, C. O. S.; Holmdahl, M.; Mendez, J. H.; Stagg, S. M.; Jonic, S.; Palovcak, E.; Armache, J.-P.; Zhao, J.; Cheng, Y.; Pintilie, G.; Chiu, W.; Patwardhan, A. and Carazo, J.M. The first single particle analysis Map Challenge: A summary of the assessments. Journal of structural biology, 2018, 204, 291-300

Abstract

The recent successes of cryo-electron microscopy fostered great expectation of solving many new and previously recalcitrant biomolecular structures. However, it also brings with it the danger of compromising the validity of the outcomes if not done properly. The Map Challenge is a first step in assessing the state of the art and to shape future developments in data processing. The organizers presented seven cases for single particle reconstruction, and 27 members of the community responded with 66 submissions. Seven groups analyzed these submissions, resulting in several assessment reports, summarized here. We devised a range of analyses to evaluate the submitted maps, including visual impressions, Fourier shell correlation, pairwise similarity and interpretation through modeling. Unfortunately, we did not find strong trends. We ascribe this to the complexity of the challenge, dealing with multiple cases, software packages and processing approaches. This puts the user in the spotlight, where his/her choices becomes the determinant of map quality. The future focus should therefore be on promulgating best practices and encapsulating these in the software. Such practices include adherence to validation principles, most notably the processing of independent sets, proper resolution-limited alignment, appropriate masking and map sharpening. We consider the Map Challenge to be a highly valuable exercise that should be repeated frequently or on an ongoing basis.

Links

https://www.sciencedirect.com/science/article/pii/S1047847718302284