"Another feature of clinical trials is the need for one or more launch meetings to, inter alia, train raters, provide information to study coordinators and leaders, and discuss the underlying methodology. Such launch meetings for training raters can be quite expensive, particularly if many such meetings are required.
"Current Methods of Rater Training
"Although many clinical trials depend on raters, previously known methods leave much to be desired. In psychiatric clinical trials, for example, ratings by human raters are often the primary outcome measures. Despite the critical role of human raters, large clinical trials typically offer only cursory rater training at a study launch meeting just prior to rater certification. Over the course of a 1-3 day launch meeting, the time allotment for training and rater certification is usually 2-4 hours. In addition, experience shows that human raters are frequently unaware how important it is to the success of a study that rater reliability be maintained.
"Often current rater training may be limited to reading though items on the rating scale(s). Some trials offer raters verbal and written conventions to help standardize the approach to common rater dilemmas (e.g., round up when rating falls between two anchor points, rate each item independently of contribution of concomitant drugs or general medical conditions). Trials generally do not provide raters with scripts for the primary outcome measures or other instruction for limiting the variation in a scale score due to the interview itself
"Problems in Rater Training
"Initial rater training, though necessary, is often of limited value. The interval between the launch meeting and local site enrollment of patients into the clinical trial is seldom less than three weeks and is often more than three months. Even when training manuals are provided, the raters often fail to consult them.
"Variations in scoring conventions from one trial to another may further dilute the benefits of initial rater training As a result, raters may deviate from the training instructions. It is not unusual to find, for example, raters in a study using the Young Mania Rating Scale (YMRS) (Young et al., 1978, Br. J. Psychiatric. 133: 429-435) applying scoring conventions that were taught in a previous training session for a completely different study that used another rating scale such as the Schedule for Affective Disorders and Schizophrenia, Current symptom version ('SADS-C') mania rating scale.
"Even the most skillful training at a launch meeting cannot train raters who do not attend the meeting. Some studies cope with this problem by staging multiple launch meetings, but this is expensive and does not address the need (which commonly arises) to hire additional raters after study start-up. Variations between the different launch meetings (which may be conducted by different personnel) may result in further rater variability.
"Problems in Rater Certification
"Rater certification refers to the process by which rater performance is documented to be within an acceptable range. Common practice for certification requires raters to score a rating scale based on viewing a videotaped patient interview. Certification is typically based on achieving agreement as determined by calculation of an intra-class correlation coefficient or more often by reference to an expert consensus score that serves as a 'gold standard.' Most clinical trials attempt to certify raters at the launch meeting itself, and require raters to meet a certification standard when tested on a single occasion. Since certification is typically carried out immediately following the training, the frequency at which raters achieve the targets of certification is likely to be much higher than might be expected with a delay between launch meeting and certification.
Most Popular Stories
- NSA Defends Global Cellphone Tracking Legality
- Top Websites for U.S. Hispanics
- Ad Counts Rise in 2013 for Hispanic Magazines
- Networks Vie for U.S. Hispanic TV Viewers
- Saab Gets Back into the Game; U.S. Auto Sales Soar
- Apple Activates Customer-Tracking iBeacon
- Dell Offers Undisclosed Number of Employee Buyouts
- 2013 Tech Gift Guide: iPad Mini Still Hot; Chromecast a Great Low-Cost Option
- A Biography of Jonathan Ive, Apple's Creative Chief
- Authorities Close to Deal with JPMorgan Chase over Madoff Response