Reconsidering methods for automatic speech processing: The case of the Language ENvironment Analysis (LENA) system

Starting around a decade ago, the Language Environment Analysis (LENA) system for automatic speech processing began to be used widely by both researchers and clinicians in order to derive estimates of numbers of adult words or child vocalizations during day-long audio recordings in naturalistic environments. Widespread adoption was spurred in part by a number of studies of correlational evidence of LENA’s reliability. However, confidence in reliable and valid measurement of metrics of conversational interactions entails consideration of both true accuracy (cf. false positive and false negative rates) for frame-based classification, as well as variability in error rates in counts of communicative vocalizations. This talk reviews evidence indicating highly variable error rates across naturalistic recordings for both frame-based classification and counts of communicative vocalizations. Such findings prompt reconsideration of conditions under which LENA can be used with confidence of both high reliability and high validity for measuring communicative interactions.