Cosine similarity as a sample size-free measure to quantify phase clustering within a single neurophysiological signal

Abstract Background Phase clustering within a single neurophysiological signal plays a significant role in a wide array of cognitive functions. Inter-trial phase coherence (ITC) is commonly used to assess to what extent phases are clustered in a similar direction over samples. However, this measure is especially dependent on sample size. Although ITC was transformed into ITCz, namely, Rayleigh’s Z, to “correct” for the sample-size effect in previous research, the validity of this strategy has not been formally tested. New method This study introduced cosine similarity (CS) as an alternative solution, as this measure is an unbiased and consistent estimator for finite sample size and is considered less sensitive to the sample-size effect. Results In a series of studies using either artificial or real datasets, CS was robust against sample size variation even with small sample sizes. Moreover, several different aspects of examinations confirmed that CS could successfully detect phase-clustering differences between datasets with different sample sizes. Comparison with existing methods Existing measures suffer from sample-size effects. ITCz produced a mixed pattern of bias in assessing phase clustering according to sample size, whereas ITC overestimated the degree of phase clustering with small sample sizes. Conclusions The current study not only reveals the incompetence of the previous “correction” measure, ITCz, but also provides converging evidence showing that CS may serve as an optimal measure to quantify phase clustering.