Best Practices Guidelines for the Engagement of People With Lived Experience and Family Members in Mental Health and Substance Use Health Research: A Modified Delphi Consensus Study
Lisa D. HawkeWuraola Dada‐PhillipsHajar SeiyadJosh OrsonLucy GoldsmithSusan E. ConwayAdam W. JordanNatasha Y. SheikhanMelissa HiebertSean A. KiddKerry Kuluski
0
Citation
42
Reference
10
Related Paper
Abstract:
ABSTRACT Introduction People with lived experience of mental health and/or substance use conditions and their families (PWLE) are increasingly engaged in research, yet rigorous guidelines for engagement are lacking. This study aims to co‐design best practice guidelines to support the authentic, meaningful engagement of PWLE in mental health and/or substance use health research. Methods A multi‐panel modified Delphi study was conducted with 61 expert panelists (35 PWLE and family members, 26 researchers/research support staff from across Canada). Participants rated 56 recommendations for importance and clarity. Consensus was defined as ≥ 70% of participants rating items at 6 or 7 on a 7‐point Likert scale (‘very important’ or ‘essential’). Qualitative feedback was analysed using content analysis to identify new items and reviewed for improvements in item clarity. After each round, items not meeting the established threshold of importance were removed. Items with low clarity scores were reworded. A PWLE advisory panel was actively involved throughout the study's design, implementation, interpretation, and reporting, ensuring that the perspectives of people with lived experience were integrated throughout the research process. Results Three Delphi Rounds were conducted. In Round 1, importance ratings ranged from 51.7% to 96.7% of participants ranking the items above the established threshold (average 80.1%), with clarity ratings ranging from 39.3% to 86.9% (average 70.7%) and an average importance coefficient of variation (CV) of 0.16. Four items were deleted, two new items were added and fifty‐five items were revised. In Round 2, 60 (98.4%) participants responded. Importance ratings ranged from 57.6% to 96.7% (average 80.2%; average CV = 0.20). Clarity ratings ranged from 50.9% to 93.2% (average 77.9%). Five items were deleted and eleven revised. In Round 3, 60 (98.4%) participants provided importance ratings ranging from 66.7% to 98.3% (average 80.8%; average CV = 0.20), and clarity ratings ranging from 63.3% to 94.9% (average 81.1%). Three items were deleted and nine were revised. Forty‐four final best practices are proposed. Conclusion These co‐developed best practice guidelines offer recommendations for meaningful PWLE engagement in mental health and/or substance use health research. By following these guidelines, research teams can ensure that PWLE contributions are genuinely valued and effectively integrated, ultimately enhancing the quality and impact of the research and fostering authentic collaboration. Patient and public involvement People with lived experience were engaged throughout the project as key team members, from a patient‐oriented research perspective. They are also co‐authors on this manuscript.Keywords:
CLARITY
Delphi Method
Graduate students
Halo effect
Cite
Citations (25)
Likert scales traditionally used in student evaluations of teaching (SET) suffer from several shortcomings, including psychometric deficiencies or ambiguity problems in the interpretation of the results. Assessment instruments with Behavioral Anchored Rating Scales (BARS) offer an alternative to Likert-type questionnaires. This paper describes the construction of an appraisal tool with BARS generated with the participation of 974 students and 15 teachers. The resulting instrument eliminates ambiguity in the interpretation of results and gives objectivity to the evaluation due to the use of unequivocal behavioral examples in the final scale. However, BARS methodology presents the problem of losing behavioral information during scale construction. The BARS methodology presented by the authors introduces an additional step to the traditional procedure, which significantly reduces the loss of information during the scale construction. The authors conclude that the qualitative approach of the proposed instrument facilitates the application of the formative function of the evaluation.
Cite
Citations (19)
One of the important research tool is questionnaire. Decision makers and researchers across all academic and industry sectors conduct surveys and questionnaires to uncover answers to specific, significant questions. In fact, questionnaires and surveys can be an effective tools for data collection required for research and evaluation. In order to develop a survey/questionnaire, first the researcher should decide how to collect the required data. In this regard, scaling is the branch of measurement that involves the construction of an instrument. One of the most widely used scaling method is attitude scales to measure instruments and Likert scale is applied as one of the most fundamental and frequently used psychometric tools in sociology, psychology, information system, politics, economy and many more research. However, research methodology research have not particularly suggested the best rating scale to be chosen for a research. This study is going to provide an overview of the Likert scale and comparing rating scales of different lengths. Results will make researchers able to make decision on what number of Likert scale points use for their survey and questionnaire. Taken as a whole this study suggests using of seven-point rating scale and if there is a need to have respondent to be directed on one side, then six-point scale might be the most suitable.
Cite
Citations (337)
The presence of smartphones is one of the technological advancements that have contributed greatly to changes in social behavior. This led to the inception of the Fear of Missing Out (FoMO) smartphone device. The purpose of this study therefore was to determine the right instrument to measure the FoMO smartphone by analyzing response points obtained from respondents. The study sample consisted of three groups of test subjects, from large city, small town and villages with n values of 226, 248, and 55 respectively. The data in this study were obtained using the 5-point Likert scale politomy data from a Fear of Missing out scale instrument, distributed online. The research data were analyzed using the Rasch model by testing rating scale analysis through Threshold. The results showed that the rating scale answered choices turned into a 3-point Likert scale with those not right, less right, and very right.
Sample (material)
Polytomous Rasch model
Cite
Citations (0)
Cite
Citations (11)
Purpose: The purpose of this study is to identify problems in the current policies for the Korea sport industry and suggest ways to improving performance and the development of Korea sport industry by applying the Delphi method.BR Method: Two rounds of Delphi method were employed by using an email questionnaire in order to examine the current policies for the Korea sport industry such as providing sport industry information and trends, providing support for overseas expansion, and providing training of sport industry professionals. The data were collected from 40 experts including sport management professors, business professors, economics professors, researchers, professionals, and public officers in the Korea sport industry. The questionnaire in the first round consists of two parts including the brief description and performance of the current policies for the Korea sport industry and personal opinions from subjects with regard to the policies. In the second round, the subjects were asked to rate the degree how they agree with the identified problems and solutions from the first round of delphi method with 5 point Likert scale.BR Results: Based on the Delphi method, some problems were identified and several solutions were suggested for Korea sport industry development and strengthening the competitiveness of the Korea sport industry.BR Conclusion: Practical implications of findings and limitations of the study were discussed and suggested.
Delphi Method
Delphi
Sport management
Questionnaire
Cite
Citations (1)
Questionnaires comprising Likert rating scale items are probably the most commonly used instrument for data collection in education and educational research, yet there is much less guidance available concerning how to analyze and interpret such rating scales than there is on how to design them. Likert scale questionnaire data is most commonly analyzed using means, although this assumes that the Likert scale descriptors are equally spaced points on a continuum. Alternative ways to analyze rating scale data include percentage frequencies, medians and modes. If means are used, there are several different approaches to interpretation depending on how concrete, intuitive or unbiased the interpretations should be. This paper presents the options available in analyzing and interpreting Likert scale data and shows how the choice between options depends on the purposes of the research.
Cite
Citations (6)
Delphi Method
Delphi
Cite
Citations (319)
The statistical analysis of survey responses based on a categorical Likert scale, as those used in the Delphi technique, has been questioned because the methodology used is based on numerical data analysis.To develop criteria for defining the consensus achievement in Delphi studies conducted for the assessment of potentially inappropriate medications (PIM) in the elderly.It was constructed an index (Yq) which takes into account the agreement by pairs of evaluators, whose calculus equation is based on combinatorial analysis. Yq was applied in a pilot study with Delphi methodology for assessing the safety of 12 drugs through a Likert scale with five response categories.On the basis of analyzing the distance (d) in between each pair of categories from the scale, it was determined the associated weighting, w=1-d, to be applied in the calculus of Yq: proportion of agreements weighted. There were defined three criteria that need to be satisfied to obtained the consensus in each item (drug) of Delphi questionnaire : a) Number of evaluators ≥60% of the panel members, b)Yq≥0,800; c) frequency of the statistical mode ≥60%. On regards to the analysis, 8/12 drugs were evaluated as potentially inappropriate for elderly adults while the remaining 4 should be revalued in successive rounds due to not obtaining consensus.The index takes into account the real distances between the Likert scale categories and the developed criteria constitute a simple tool for the analysis of the Delphi questionnaires made for the valuation of PIM on older adults.
Delphi Method
Categorical variable
Delphi
Cite
Citations (2)
The purpose of this study is to identify problems in the current policies for the regional sport industry and suggest ways to strengthen the competitiveness of the promotion of the regional sport industry by applying Delphi method. Two rounds of Delphi method were employed by using an email questionnaire in order to examine the current policies for the regional sport industry such as sport tourism and sponomics. The data were collected from 40 experts including sport management professors, researchers, and public officers in the regional department of sport industry. The questionnaire in the first round consists of two parts including the brief description and performance of the current policies for the regional sport industry and personal opinions from subjects with regard to the policies. In the second round, the subjects were asked to rate the degree how they agree the identified problems and solutions from the first round of delphi method with 5 point Likert scale. Based on the Delphi method, some problems were identified and several solutions were suggested for strengthening the competitiveness of the regional sport industry. Practical implications of findings and limitations of the study were discussed and suggested.
Delphi Method
Promotion (chess)
Delphi
Sport management
Cite
Citations (1)