Exploring expert variability in defining pseudoparalysis: An international survey.

ABSTRACT Background There is currently disagreement among experts in the field of shoulder surgery when attempting to define the term ‘pseudoparalysis’. Multiple surgical techniques to address this condition have been investigated, however many studies have recruited heterogeneous patient populations and have used varying definitions for pseudoparalysis. This makes it difficult to compare outcomes between various techniques. To our knowledge, no previous study has surveyed international experts regarding the definition of pseudoparalysis using a questionnaire and video-based patient assessment. The purpose of this study was to evaluate the level of agreement among shoulder surgeons in defining and applying the term pseudoparalysis. We hypothesized that inter-rater agreement for classifying patients as pseudoparalytic would be poor. Methods Members of the American Shoulder and Elbow Surgeons, the European Society for Surgery of the Shoulder and Elbow and our national shoulder and elbow society were surveyed on two occasions using an electronic questionnaire. All surgeons were asked to identify their preferred definition for pseudoparalysis from one of four options. Surgeons then viewed video examinations of 10 patients and labeled them as pseudoparalytic or not. Inter- and intra-rater reliability were calculated as κ coefficients. Pearson chi-squared (χ2) was used to detect associations between preferred definition and demographic information. Results A total of 246 surgeons responded to at least one survey. Overall inter-rater agreement on classifying patients as pseudoparalytic based on video consultation was κ = 0.59 (95% CI, 0.58–0.60). The same verbal definition was selected by 56.1% of surgeons. Surgeons were not internally consistent in their choice of definition, with intra-rater reliability κ = 0.64 (95% CI, 0.48–0.81). Intra-rater reliability for classifying patients as pseudoparalytic was better, κ = 0.78 (95% CI, 0.72 – 0.83). An association was observed between how surgeons defined pseudoparalysis and their age (p=0.03) and shoulder caseload percentage (p=0.04). Conclusion Shoulder surgeons do not agree on how best to define pseudoparalysis of the shoulder. Inter-rater agreement based on video consultation was weak overall and improved with the elimination of an outlier video. Intra-rater agreement was less frequent when selecting a preferred definition compared to classifying patients as pseudoparalytic based on video examinations. Surgeons may rely less on explicit criteria and more on a conceptual framework when assigning a pseudoparalytic label. Care should be taken with use of the term pseudoparalysis in clinical outcome studies when there is clearly a lack of consensus among experts in defining this term.