Video clip assessment of a salivary gland ultrasound scoring system in Sjögren’s syndrome using consensual definitions: an OMERACT ultrasound working group reliability exercise

Objective To develop ultrasound (US) definitions and a US novel scoring system for major salivary gland (SG) lesions in patients with primary Sjogren’s syndrome (pSS) and to test their intrareader and inter-reader reliability using US video clips. Methods Twenty-five rheumatologists were subjected to a three-round, web-based Delphi process in order to agree on (1) definitions and scanning procedure of salivary gland ultrasonography (SGUS): parotid, submandibular and sublingual glands (PG, SMG and SLG); (2) definitions for the elementary SGUS lesions in patients with Sjogren’s syndrome; (3) scoring system for grading changes. The experts rated the statements on a 1–5 Likert scale. In the second step, SGUS video clips of patients with pSS and non-pSS sicca cases were collected containing various spectrums of disease severity followed by an intrareader and inter-reader reliability exercise. Each video clip was evaluated according to the agreed definitions. Results Consensual definitions were developed after three Delphi rounds. Among the three selected SGs, US assessment of PGs and SMGs was agreed on. Agreement was reached to score only greyscale lesions and to focus on anechoic/hypoechoic foci in a semiquantitative matter or, if not possible on a qualitatively (present/absent) evaluation of fatty or fibrous lesions. Intrareader reliability for detecting and scoring these lesions was excellent (Cohen’s kappa 0.81) and inter-reader reliability was good (Light’s kappa 0.66). Conclusion New definitions for developing a novel semiquantitative US score in patients with pSS were developed and tested on video clips. Inter-reader and intrareader reliabilities were good and excellent, respectively.