Performance Evaluation of MHC Class-I Binding Prediction Tools Based on an Experimentally Validated MHC–Peptide Binding Data Set

The question whether a protein can be processed and resulting peptides presented by major histocompatibility complex (MHC) is of high importance for immunotherapy design. MHC ligands can be predicted by in silico peptide-MHC class-I binding prediction algorithms. However, there are considerable differences in prediction performance, depending on the selected algorithm, MHC class-I type and peptide length. We evaluated the prediction performance of 13 algorithms based on binding affinity data of 8-11-mer peptides derived from the HPV16 E6 and E7 proteins to the most prevalent human leukocyte antigen (HLA) types. Peptides from high to low predicted binding likelihood were synthesized and their HLA binding experimentally verified by in vitro competitive binding assays. Based on the actual binding capacity of the peptides, the performance of prediction algorithms was analyzed by calculating receiver operating characteristics (ROC) and the area under the curve (AROC). No algorithm outperformed others, but different algorithms predicted best for particular HLA types and peptide lengths. Sensitivity, specificity and accuracy of decision thresholds were calculated. Commonly used decision thresholds yielded only 40% sensitivity. To increase sensitivity, optimal thresholds were calculated, validated and compared. In order to make maximal use of online available prediction algorithms, we developed MHCcombine, a web application that allows simultaneous querying and output combination of up to 13 prediction algorithms. Taken together, we here provide an evaluation of peptide-MHC class-I binding prediction tools and recommendations how to increase prediction sensitivity to extend the number of potential epitopes applicable as targets for immunotherapy.