https://orcid.org/0000-0002-4769-1617
Figures
Abstract
Background
Patients with Duchenne muscular dystrophy (DMD) adopt compensatory movement patterns as muscles weaken. The Duchenne Video Assessment (DVA) measures patient ease of movement through identification of compensatory movement patterns. The DVA directs caregivers to video record patients performing specific movement tasks at home using a secure mobile application, and DVA-certified physical therapists (PTs) score the videos using scorecards with prespecified compensatory movement criteria. The goal of this study was to develop and refine the DVA scorecards.
Methods
To develop the initial scorecards, 4 PTs collaboratively created compensatory movement lists for each task, and researchers structured the lists into scorecards. A 2-round modified Delphi process was used to gather expert opinion on the understandability, comprehensiveness, and clinical meaningfulness of the compensatory movements on the scorecards. Eight PTs who had evaluated ≥50 patients with DMD and participated in ≥10 DMD clinical trials were recruited for the panel. In Round 1, panelists evaluated compensatory movement criteria understandability via questionnaire and tested the scorecards. In Round 2, panelists participated in an in-person meeting to discuss areas of disagreement from Round 1 and reach consensus (≥75% agreement) on all revisions to the scorecards.
Results
During the Round 1 revisions to the scorecards, there were 67 changes (44%) to the wording of 153 original compensatory movement criteria and 3 criteria were removed. During the Round 2 revisions to the scorecards, there were 47 changes (31%) to the wording of 150 compensatory movement criteria, 20 criteria were added, and 30 criteria were removed. The panel reached 100% agreement on all changes made to scorecards during Round 2.
Citation: Contesse MG, Lowes LP, White MK, Dalle Pazze L, McSherry C, Alfano LN, et al. (2022) Development of Duchenne Video Assessment scorecards to evaluate ease of movement among those with Duchenne muscular dystrophy. PLoS ONE 17(4): e0266845. https://doi.org/10.1371/journal.pone.0266845
Editor: Tai-Heng Chen, Kaohsuing Medical University Hospital, TAIWAN
Received: December 1, 2021; Accepted: March 29, 2022; Published: April 13, 2022
Copyright: © 2022 Contesse et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: This research was funded by a grant from Charley's Fund (https://charleysfund.org/), a non-profit organization. Charley's Fund was involved in the design of the study and preparation of the manuscript.
Competing interests: I have read the journal’s policy and the authors of the manuscript have the following competing interests: Marielle Contesse, Christine McSherry, and Mindy Leffler are all employed by Casimir LLC. Additionally, Christine McSherry and Mindy Leffler are 50/50 owners of Casimir LLC. Casimir LLC developed the Duchenne Video Assessment and uses it commercially. Their employment at Casimir did not interfere with the full and objective presentation of the research.
Introduction
Duchenne muscular dystrophy (DMD) is a rare, genetic disease characterized by progressive muscle degeneration and weakness that affects approximately 15.9 to 19.5 of every 100,000 live male births [1]. Genetic mutations in the dystrophin gene result in degeneration of muscle fibers involving inflammation and fibrosis, accompanied by reduced muscle regeneration. Progressive muscle weakness begins in early childhood and eventually leads to the loss of functional movement [2]. Death is often due to heart or respiratory failure [3, 4]. While there are a number of potential therapeutics being tested, there is still no cure for DMD [5–7]. Sensitive outcome measures that can be used in clinical trials for the development of potential therapeutics are critical.
People with DMD compensate for muscle weakness by changing their movement patterns [8–10]. Increased muscle weakness leads to a decline in ease of movement [11–13]. Many clinician-rated functional assessments used in DMD clinical trials are limited to differentiating between an inability to perform a test item, able to complete with compensations, or able to complete without compensations [14, 15]. They do not delineate between different severity levels of compensated movement for each test item. Measuring changes in the number or severity of compensations required to complete a movement task may allow clinical trials to detect incremental yet still clinically meaningful functional changes in a shorter duration of time. Clinicians, caregivers, and individuals with DMD consider needing assistance, using alternative techniques, and using more effort to complete a movement task as meaningful worsening in physical function [16]. When measuring such small increments of change, it will be important to ensure that each movement compensation acquired or lost represents a clinically meaningful amount of change in function.
The Duchenne Video Assessment (DVA) is a novel, home-based clinical outcome assessment that measures ease of movement through identification of compensatory movement patterns. Rather than instructing best performance (e.g. “get up from the floor using as little support as possible and as fast as you can” [14] or “walk as far as possible for 6 minutes” [17]) in a clinic setting, the DVA evaluates typical performance in the home environment. The DVA directs caregivers to video record patients doing specific movement tasks at home using a secure mobile application. Caregivers are provided with a training manual and videos to standardize the set-up, lighting, clothing, surfaces, time of day, and instructions during video recording of movement tasks. The DVA movement tasks were selected through qualitative interviews with caregivers and clinicians to be relevant to patients with DMD in each disease stage and able to detect changes in function [18, 19]. The movement tasks are either foundational tasks for daily life (e.g. walking, climbing stairs) or activities of daily living (e.g. putting a t-shirt on, eating). Cognitive interviews with caregivers confirmed that the training materials and mobile application are easy to understand and use [18]. When the DVA is used in a research study, study staff monitor data collection, and caregivers are asked to re-record any videos that do not meet quality standards. The DVA videos are scored by DVA-certified physical therapists using scorecards (example provided in Fig 1) with prespecified compensatory movement criteria. The goal of this study was to develop and refine, using a modified Delphi approach, the DVA scorecards.
Within each compensatory movement criterion, raters select the sub-criterion that indicates the highest level of severity demonstrated in the video. For example, a compensatory criterion when standing up from sitting on the floor is: (D) “Pushing hands/arms off thighs for leverage”. The sub-criteria are: (0) no use of hands on thighs for leverage, (1) pushes off thighs with one hand/arm, and (2) pushes off thighs with two hands/arms. Task severity points are summed for an overall severity score. Not all tasks have the same number of possible points, so a severity percentage is calculated by dividing the severity score by the maximum severity score. Higher severity percentage indicates more compensated movement; the inability to perform a task is assigned the maximum severity score.
Methods
Source material
In a longitudinal study (Duchenne Video Project) of male participants with and without DMD, caregivers collected video data of participants performing DVA movement tasks for outcome measure development and testing. Detailed information about the Duchenne Video Project study population and methods is provided in another manuscript [20].
Phase 1: Initial compensatory movement identification
In the first phase of scorecard development, four physical therapists from the Lowes Lab at Nationwide Children’s Hospital with extensive experience evaluating people with DMD contributed to the initial identification of compensatory movements for 15 movement tasks. They watched videos of approximately 25 Duchenne Video Project participants of varying abilities performing each movement task and independently created lists of the compensatory movements they observed for each task. They proceeded to discuss the compensations they observed and collaboratively developed preliminary lists of compensatory movements for each task.
Phase 2: Scorecard formation
In the second phase of scorecard development, researchers who specialize in outcome measure development (M.C. and M.L.) structured the preliminary lists of compensatory movements into scorecards. Using the videos from the Duchenne Video Project, they observed the movement patterns of those without DMD and those of varying stages of DMD for each task and identified relationships between compensations and spectrums of severity. For example, using two hands for assistance when climbing the stairs is more severe than using one hand.
Phase 3: Modified Delphi process
Study design and panel selection.
In the third phase of scorecard development, a 2-round modified Delphi process [21, 22] (Fig 2) was used to gather expert opinion on the scorecards from physical therapists who have evaluated a large volume of patients with DMD. The Delphi method [23] has often been used in health research to determine expert consensus for clinical problems and traditionally consists of iterative rounds of questionnaires with no direct interaction between panelists. The classic Delphi method has been criticized for not benefiting from the exchange of information between panelists that could identify reasons for disagreement [24]. This study used a modified Delphi process. The first round of the modified Delphi panel elicited qualitative feedback from panelists, and the identification of common themes allowed for fewer Delphi panel rounds than the classic method [25]. A second in-person round allowed panelists to explain disagreement and collaboratively revise the scorecards. The modified Delphi process benefited from both independent panelist evaluation of the scorecards in the first round and collaborative revision of the scorecards in the second round.
Eight panelists were recruited, consistent with the recommended modified Delphi method sample size [21], to be large enough to allow for diversity of experiences evaluating patients with DMD and small enough to allow all panelists to participate meaningfully in the group discussion. Physical therapists who had evaluated ≥50 people with DMD in clinic and participated in ≥10 DMD clinical trials were recruited from a list of United States physical therapists provided by a key opinion leader (L.L.). Only one of the physical therapists from Phase 1 participated in Phase 3. All participating physical therapists signed written agreements to provide consultant services for the purposes of the Delphi panel, and they were provided with an honorarium for their time.
Of the eight physical therapist panelists, three (38%) practiced physical therapy for ≥20 years, six (75%) provided physical therapy to ≥200 patients with DMD, and three (38%) participated in ≥15 DMD clinical trials (Table 1). All eight panelists participated in both Round 1 and Round 2 of the Delphi panel.
Modified Delphi process: Round 1.
The intention in Round 1 was to revise language in the scorecards to improve understandability to physical therapists. For this round, panelists received the following via email: a DVA overview, preliminary questionnaire, and rating exercises. The DVA overview described how caregivers record videos, how physical therapists score videos, and how the preliminary scorecards were developed. The preliminary questionnaire included questions about physical therapy experience as well as an evaluation of the clarity of each DVA scorecard. For each scorecard, panelists indicated whether each compensatory movement criterion was easy to understand, understandable but needed modifications, or not understandable. If a compensatory movement criterion was not understandable or needed modification, the panelists provided a suggested change to improve clarity, which was included in the qualitative analysis.
For the rating exercises, the panelists watched videos of a different set of 4 participants from the Duchenne Video Project performing each movement task. Based on their clinical impressions, panelists ranked the 4 participants from strongest to weakest for the movement task. Next, panelists used the scorecard to evaluate each of the 4 participants. They were then asked whether the scorecard severity percentages reflected where the participants fell on the spectrum of strongest to weakest and to explain how the scorecards did or did not match up with their clinical impression of participant disease severity. Additionally, panelists were asked to report any problems they experienced while scoring the videos with the scorecards. The responses were included in the qualitative analysis.
The primary analyst (M.C.) compiled all qualitative suggestions for each criterion and coded suggestions as:
- Apply suggestion during Round 1: used when clear changes to movement compensation wording were suggested.
- Discuss suggestion during Round 2: used when unclear changes to movement compensation wording were suggested, a question was posed by the panelist about the movement compensation, or a suggestion conflicted with the suggestion of another panelist.
- Add detail to rater training materials to address suggestion: used when a panelist indicated that more background information was needed to understand a movement compensation.
The secondary analyst (M.L) reviewed all coded qualitative suggestions, and any coding disagreement between the two analysts was discussed and resolved.
The specific compensatory movement criteria selected for each participant’s video during the rating exercises were entered into an Excel spreadsheet, and all scoring discrepancies between panelists (e.g. one panelist selected sub-criterion B2 and seven panelists selected B1 for a participant’s video) were coded as “discrepant” by the primary analyst. The secondary analyst reviewed and confirmed the spreadsheet with criteria coded as “discrepant”. All participant videos with panelist scoring discrepancies were added to the Round 2 discussion slide deck for panel review and revisions to improve scoring reliability.
All qualitative suggestions coded as “apply suggestion during Round 1” were used to revise the scorecards prior to Round 2. A slide deck with each discussion item was created for Round 2, and a discussion worksheet was created to track the panel votes for each suggested modification to the scorecards.
Modified Delphi process: Round 2.
The goals of the Round 2 meeting were to discuss all areas of disagreement from Round 1 and reach consensus (≥75% agreement) on all revisions to the scorecards. The primary and secondary analysts facilitated an 8-hour in-person meeting with the entire panel in Orlando, Florida on April 14, 2019. Ground rules were set that encouraged panelists to share their views openly during the discussion.
Panelists received copies of the original (Version 1) and revised (Version 2) scorecards at the beginning of the meeting. The facilitators presented the panel with fully anonymized feedback that required discussion from Round 1. The discussion during Round 2 focused on revising compensatory movement description language, adding compensations that were missing, and removing compensations that were not clinically meaningful. A vote was held about each decision during the in-person discussion, and the number of panelists who agreed with the change and the number of panelists who voted were recorded for each decision. Panelists were asked to keep in mind that the acquisition or loss of each compensatory movement on the scorecard should represent a clinically meaningful degree of change in function. At the end of the discussion for each scorecard, the panelists were asked if the scorecard was missing any compensatory movement criteria. The primary and secondary analysts revised the scorecards based on the changes made during Round 2 that received panel consensus.
Results
Modified Delphi process: Round 1
Based on panelist questionnaire responses, there were 67 changes (44%) to the wording of 153 compensatory movement sub-criteria in Version 1 (Table 2). As an example of the types of changes that were made to the scorecards, the specific revisions made to the Stand Up from Sitting on Floor scorecard are provided in Table 3. For that scorecard, the majority of panelists found all Version 1 sub-criteria except criterion A1 (“kneeling (one-legged)”) easy to understand, and A1 was revised based on the suggested wording of 4 (50%) of the panelists to become “half-kneeling (one-legged)”. All suggested wording changes for that scorecard were incorporated into the Round 1 revision, even if only a single panelist suggested a wording change.
Three compensatory movement sub-criteria from the Jump Forward scorecard were collapsed into related sub-criteria during Round 1:
- Criterion A: The knee flexion in the squat position before the feet leave the ground was broken into two severity levels in Version 1: (1) limited knee bend and (2) no knee bend. Those two levels were collapsed into one criterion in Version 2: “Limited or no knee flexion”.
- Criterion G: The knee flexion in the landing squat was broken into two severity levels in Version 1: (1) limited knee bend and (2) no knee bend. Those two levels were collapsed into one criterion in Version 2: “limited or no knee flexion on landing squat”.
- Criterion H: The bend at the waist after landing was broken into two severity levels in Version 1: (1) slight bend at the waist and (2) no bend at the waist. Those two levels were collapsed into one criterion in Version 2: “slight or no bend at the waist”.
Half of panelists reported needing modifications for Criterion A to define the degree of knee bend, 3 (38%) reported needing modifications for Criterion G to define the degree of knee bend, and 5 (63%) reported needing modifications for Criterion H to define degree of bend at waist and clarify language. In addition, there were scoring discrepancies for all 3 criteria during the rating exercises, with some panelists selecting one level of severity and others selecting the other for the same participant. Since the difference between the two severity levels for each of the three criteria were difficult to define and observe, they were collapsed into one level to improve scoring reliability and discussed during Round 2.
Modified Delphi process: Round 2
During the in-person discussion, the scorecards for all movement tasks, except Run, were discussed. Due to time limitations, the meeting ended before the panel discussion of Run, but a subset of the Delphi panel (n = 3) with availability met after the in-person discussion to discuss the Run task. For all scorecards, 100% of the panelists agreed on all changes made to the scorecards during Round 2. A universal change was made to the structure of all scorecards to add a level 0 sub-criterion to all compensatory movement criteria so that physical therapists could indicate when a compensation was not present.
There were 47 changes (31%) made to the wording of 150 compensatory movement sub-criteria to improve understandability. As an example of the types of wording changes made during Round 2, the Stand Up from Sitting on Floor (Table 3) criterion B (“walking hands along floor”) from Version 2 was changed to “walking hands towards the body along the floor before lifting torso”. The words “towards the body” and “before lifting torso” were added to clarify the exact movement compensation that raters should identify and avoid confusion about how and when the hands walk along the floor.
Twenty compensatory movement sub-criteria were added. The panelists identified when compensatory movements were missing from the scorecards. As an example of an added sub-criterion, a compensation in the Climb 5 Stairs scorecard involved a spectrum of severity for body positioning on the stairs that was missing a higher level of severity that panelists had observed in practice. The highest level of severity in Version 2 was “turning torso to face wall or railing to side step”, and the panelists voted unanimously to add the compensation “uses rail to bear weight” as a missing higher level of severity that occurs as muscles weaken.
Thirty compensatory movement sub-criteria were removed. When evaluating each compensatory movement on the scorecards, panelists were asked to ensure that each movement included was a clinically meaningful compensation. Of the 30 sub-criteria that were removed, 13 were removed to ensure the clinical meaningfulness of each movement (detailed in S1 Table and representative example provided in Table 4), 6 were replaced with related compensations, 3 were integrated into other criteria, and 8 were removed from Shift Weight in Bed and Arms Off and On Armrests since the part of the task they applied to was removed. For the Shift Weight in Bed task, the participant was originally instructed to sit up in bed, lean forward, and then lean to each side. During Round 2, the panelists unanimously decided that leaning to each side is not as functionally important to the task and that it should be removed, which resulted in the removal of the 5 criteria associated with leaning to each side. For the Arms Off and On Armrests task, the scorecard originally included evaluation of both taking the arm off the armrest and putting it back on. During Round 2, the panelists unanimously recommended removal of the 3 criteria associated with the gravity-assisted motion of lowering the arm off the armrest.
Discussion
This study developed and refined, using a modified Delphi approach, the DVA scorecards. The modified Delphi process was critically important to the revision of the scorecards to ensure that the compensatory movement criteria are understandable, clinically meaningful, and comprehensive. Understandability of the compensatory movement wording is essential for high inter-rater and intra-rater scoring reliability, which was tested in another study [20]. By delineating between different severity levels of compensated movement, the DVA provides an opportunity to detect incremental functional changes that occur in a shorter duration of time. Those incremental functional changes are only valuable to measure if they are clinically meaningful.
The DVA scorecard development utilized physical therapists, with expertise in the evaluation of movement patterns used by patients with DMD, as reporters of clinical meaningfulness and comprehensiveness of the selected compensatory movement criteria. Patients may not be aware of their compensatory movements since the movements develop in response to progressive muscle weakness and contracture [26, 27], and caregivers may not be able to articulate the specific compensatory movement patterns. Physical therapists understand the functional importance of each movement compensation as it relates to muscle weakness and contracture, and they have observed these patterns adopted by the breadth of patients they have evaluated.
The DVA is intended to assess clinical benefit in clinical trials by evaluating the impact of a potential therapeutic on participants’ daily function in the home environment. Existing functional assessments for DMD measure performance in a clinical setting, which may not be reflective of a patient’s daily ability as it can be influenced by patient effort in clinic, encouragement by medical staff or caregivers, and travel-related fatigue [2]. The DVA includes functional tasks that those with DMD perform in their daily lives and measures their typical performance in their home environment. The COVID-19 pandemic called attention to the need for clinical outcome assessments that can be administered remotely [28, 29] and accelerated a trend towards remote clinical trials [30]. Existing functional measures require frequent travel to clinical sites for evaluation by physical therapists, while the DVA mobile application allows caregivers to collect patients’ functional data securely in the home environment. Participation in clinical trials can pose physical, emotional, and logistical challenges for people with DMD and their families [31], and use of the DVA at home removes the travel and financial burden placed on patients and families.
This study was not without limitations. First, only a subset of panelists discussed the Run movement task during Round 2 due to time limitations during the in-person discussion. While it is possible that the panelists who were not present during the Run discussion may have had differing views, the subset of panelists who were available agreed on the changes that were made to the scorecard. Second, this study only included panelists who work in the United States, and it is possible that the understandability of the compensatory movement descriptions may be different in other countries; however, since the DVA is scored using central raters, it is possible for physical therapists who work in the United States to score videos collected from other countries.
Conclusions
Physical therapists with extensive experience evaluating patients with DMD confirmed that the compensatory movement criteria included in the DVA scorecards were understandable, comprehensive, and clinically meaningful. The use of the DVA in clinical trials has the potential to expand patient access to studies by eliminating the need to travel to clinical sites and including movement tasks that can be assigned to participants at any stage of their disease. Future research will evaluate whether the DVA is able to detect functional changes in a shorter duration than existing measures, which could reduce the time required of patients for clinical trial participation.
Supporting information
S1 Table. All instances of removal of compensatory movement criteria to ensure clinical meaningfulness during Round 2.
https://doi.org/10.1371/journal.pone.0266845.s001
(DOCX)
Acknowledgments
The authors are grateful to the Duchenne Video Project families who contributed videos for testing and training. The authors thank the physical therapists who made this study possible.
Conferences: A related poster was presented at the virtual 2020 Muscular Dystrophy Association Clinical & Scientific Conference and the virtual 2020 European Conference on Rare Diseases & Orphan Products (ECRD). The ECRD conference abstract was published in Orphanet Journal of Rare Diseases Supplement 1, Volume 15, P14.
References
- 1. Ryder S, Leadley RM, Armstrong N, Westwood M, de Kock S, Butt T, et al. The burden, epidemiology, costs and treatment for Duchenne muscular dystrophy: an evidence review. Orphanet Journal of Rare Diseases. 2017;12: 79. pmid:28446219
- 2.
U.S. Food & Drug Administration. Duchenne Muscular Dystrophy and Related Dystrophinopathies: Developing Drugs for Treatment. Silver Spring, MD; 2018 Feb.
- 3. Van Ruiten H, Bushby K, Guglieri M. State-Of-The-Art Advances in Duchenne Muscular Dystrophy. EMJ. 2017;2: 90–99.
- 4. Birnkrant DJ, Bushby K, Bann CM, Alman BA, Apkon SD, Blackwell A, et al. Diagnosis and management of Duchenne muscular dystrophy, part 2: respiratory, cardiac, bone health, and orthopaedic management. Lancet Neurol. 2018;17: 347–361. pmid:29395990
- 5. Korinthenberg R. A new era in the management of Duchenne muscular dystrophy. Developmental Medicine & Child Neurology. 2019;61: 292–297. pmid:30556126
- 6. Reinig AM, Mirzaei S, Berlau DJ. Advances in the Treatment of Duchenne Muscular Dystrophy: New and Emerging Pharmacotherapies. Pharmacotherapy: The Journal of Human Pharmacology and Drug Therapy. 2017;37: 492–499. pmid:28152217
- 7. Duan D, Goemans N, Takeda S, Mercuri E, Aartsma-Rus A. Duchenne muscular dystrophy. Nature Reviews Disease Primers. 2021;7: 1–19. pmid:33414454
- 8. Maciel FK de L, Santos ALY da S, Artilheiro MC, Sá C dos SC de. Motor Compensation Strategies for Reduced Upper Limb Function Among Individuals With Duchenne Muscular Dystrophy. Percept Mot Skills. 2021;128: 1059–1077. pmid:33726582
- 9. Peeters LHC, Kingma I, van Dieën JH, de Groot IJM. Don’t forget the trunk in Duchenne muscular dystrophy patients: more muscle weakness and compensation than expected. Journal of NeuroEngineering and Rehabilitation. 2019;16: 44. pmid:30917845
- 10. D’Angelo MG, Berti M, Piccinini L, Romei M, Guglieri M, Bonato S, et al. Gait pattern in Duchenne muscular dystrophy. Gait & Posture. 2009;29: 36–41. pmid:18656361
- 11. Hunter GR, Plaisance EP, Carter SJ, Fisher G. Why Intensity is not a Bad Word: Optimizing Health Status at any Age. Clin Nutr. 2018;37: 56–60. pmid:28214041
- 12. Hunter GR, McCarthy JP, Bamman MM. Effects of resistance training on older adults. Sports Med. 2004;34: 329–348. pmid:15107011
- 13. Landers KA, Hunter GR, Wetzstein CJ, Bamman MM, Weinsier RL. The interrelationship among muscle mass, strength, and the ability to perform physical tasks of daily living in younger and older women. J Gerontol A Biol Sci Med Sci. 2001;56: B443–448. pmid:11584029
- 14. Scott E, Eagle M, Mayhew A, Freeman J, Main M, Sheehan J, et al. Development of a functional assessment scale for ambulatory boys with Duchenne muscular dystrophy. Physiother Res Int. 2012;17: 101–109. pmid:21954141
- 15. Pane M, Coratti G, Brogna C, Mazzone ES, Mayhew A, Fanelli L, et al. Upper limb function in Duchenne muscular dystrophy: 24 month longitudinal data. PLoS One. 2018;13: e0199223. pmid:29924848
- 16. Staunton H, Trennery C, Arbuckle R, Guridi M, Zhuravleva E, Furlong P, et al. Development of a Clinical Global Impression of Change (CGI-C) and a Caregiver Global Impression of Change (CaGI-C) measure for ambulant individuals with Duchenne muscular dystrophy. Health and Quality of Life Outcomes. 2021;19: 184. pmid:34311756
- 17. McDonald CM, Henricson EK, Han JJ, Abresch RT, Nicorici A, Elfring GL, et al. The 6-minute walk test as a new outcome measure in Duchenne muscular dystrophy. Muscle Nerve. 2010;41: 500–510. pmid:19941337
- 18. White MK, Leffler M, Rychlec K, Jones C, McSherry C, Walker L, et al. Adapting traditional content validation methods to fit purpose: an example with a novel video assessment and training materials in Duchenne muscular dystrophy (DMD). Qual Life Res. 2019;28: 2979–2988. pmid:31302840
- 19.
Leffler MG, Contesse MG, Staunton H. Home-Based Video Assessment of the Quality of Movement of Patients with Duchenne: Interviews with Physical Therapists to Inform Task Selection. Abstract P.161 presented at: MDA Clinical and Scientific Conference; 2019 Apr 13; Orlando, Florida.
- 20. Contesse MG, Sapp ATL, Apkon SD, Lowes LP, Dalle Pazze L, Leffler MG. Reliability and construct validity of the Duchenne Video Assessment. Muscle Nerve. 2021;64: 180–189. pmid:34050939
- 21.
Fitch K, Bernstein SJ, Aguilar MD, Burnand B, LaCalle JR, Lazaro P, et al. The RAND/UCLA Appropriateness Method User’s Manual. Santa Monica, CA: RAND; 2001.
- 22. Eubank BH, Mohtadi NG, Lafave MR, Wiley JP, Bois AJ, Boorman RS, et al. Using the modified Delphi method to establish clinical consensus for the diagnosis and treatment of patients with rotator cuff pathology. BMC Medical Research Methodology. 2016;16: 56. pmid:27206853
- 23. Dalkey N, Helmer O. An Experimental Application of the DELPHI Method to the Use of Experts. Management Science. 1963;9: 458–467.
- 24. Murphy MK, Black NA, Lamping DL, McKee CM, Sanderson CF, Askham J, et al. Consensus development methods, and their use in clinical guideline development. Health Technol Assess. 1998;2: i–iv, 1–88.
- 25. Cantrill JA, Sibbald B, Buetow S. The Delphi and nominal group techniques in health services research. International Journal of Pharmacy Practice. 1996;4: 67–74. https://doi.org/10.1111/j.2042-7174.1996.tb00844.x
- 26. Goldberg EJ, Neptune RR. Compensatory strategies during normal walking in response to muscle weakness and increased hip joint stiffness. Gait Posture. 2007;25: 360–367. pmid:16720095
- 27. McGibbon CA. Toward a Better Understanding of Gait Changes With Age and Disablement: Neuromuscular Adaptation. Exercise and Sport Sciences Reviews. 2003;31: 102–108. pmid:12715975
- 28. Izmailova ES, Ellis R, Benko C. Remote Monitoring in Clinical Trials During the COVID‐19 Pandemic. Clin Transl Sci. 2020;13: 838–841. pmid:32526077
- 29. Goldsack JC, Izmailova ES, Menetski JP, Hoffmann SC, Groenen PMA, Wagner JA. Remote digital monitoring in clinical trials in the time of COVID-19. Nature Reviews Drug Discovery. 2020;19: 378–379. pmid:32409759
- 30. McDermott MM, Newman AB. Remote Research and Clinical Trial Integrity During and After the Coronavirus Pandemic. JAMA. 2021;325: 1935. pmid:33885728
- 31. Franson T, Kinnett K, Cripe TP. Unique Burdens of Pediatric Clinical Trials in Duchenne Muscular Dystrophy, April 20–21, 2017, Bethesda, Maryland, USA. Ther Innov Regul Sci. 2019;53: 154–163. pmid:29714576