Confidence and Reliability in Driving Simulator Based Expert Reviews of ADAS Controllability

Abstract Expert reviews are one of the possible methods for the assessment of controllability of advanced driver assistance systems (ADAS). This article presents the method of a study on the properties of expert reviews of controllability and reports the most important findings. For the study the criticality ratings and self assessed confidence ratings of a sample of 15 experts in controllability assessment were compared with the judgments of 49 naive test subjects in a driving simulator setting in 7 distinct scenarios of partially automated driving. The results of this study give insight on the retest reliability of the expert rating and provide findings on the self assessed confidence of judgment in naive and expert participants.