Structured Abstract
Clinical Question: Can an intervention strategy whose research was done with one population be used effectively with a different population for the same purpose?
Method: Systematic Review and Meta-Analysis
Study Sources: Google Scholar, Academic Search Premier
Search Terms: initiation, script, script fading, language impairment, social communication, social interaction, conversation, and spontaneous
Number of Included Studies: 8
Number of Participants: 18
Primary Results:
1) Six of the eight studies reviewed were of acceptable quality.
2) Based on percent of non-overlapping data calculations, script training was found to be effective or very effective.
3) The body of script training research included six acceptable quality studies conducted by four sets of researchers across four geographical locations and with 18 participants.
Conclusions:
The research evidence indicates that script training interventions enhance the conversational independence of children with autism; however, there is no evidence to suggest it will be effective for a different population. Nonetheless, a thorough examination of the match between client characteristics and participant characteristics may help determine whether an intervention is appropriate for a client who has not been represented in the research literature. In addition, clinicians who apply interventions to populations that have not been included in research should rely on their professional judgment and clinical expertise to make reasonable implementation decisions and use progress monitoring results to inform subsequent clinical decisions.
Scenario
Heather is an elementary school speech-language pathologist (SLP) who works with Liam, a second-grade student with specific language impairment (SLI). Some of Liam’s Individualized Education Program (IEP) goals involve initiating conversation and maintaining conversational topic with peers without prompting from a clinician. He enjoys being near his friends, but rarely engages in conversation with them. To address Liam’s conversational independence, Heather provides support to Liam at times and in settings when his peers are likely to be nearby, such as at lunch, during art class, and at recess. Heather hoped that because she was supporting Liam in more natural contexts, she would eventually be able to withdraw her prompting and Liam would talk to his friends independently. Unfortunately, Heather has found it very difficult to fade her vocal prompting. Before Liam will initiate a conversation, Heather must still provide a vocal model, such as “What are you drawing?” or “I’m drawing a big dinosaur.” Liam responds to his peers’ questions, but after he has answered, he stops talking. Heather is concerned that her close physical proximity during social situations disrupts the natural conversation context and prevents the peers’ presence from cuing conversation. She also is concerned that her presence may make Liam’s friends uncomfortable. Heather would like Liam to initiate and maintain conversation with his friends more independently—without her vocal prompting and intrusive presence.
Heather described her challenges with Liam to Teresa, a special education teacher in her building. Sympathizing with her predicament, Teresa told Heather about script training, a response prompting strategy that does not require the close proximity of an adult. Teresa has used script training with many students who have autism who do not readily initiate or maintain conversations. Teresa loaned Heather the book Teaching Conversation to Children with Autism: Scripts and Script Fading (McClannahan & Krantz, 2005) to see if script training could work for Liam, too.
Script-training interventions are used almost exclusively to improve some aspect of social communication. For instance, scripts are used to promote bids for joint attention (MacDuff, Ledo, McClannahan, & Krantz, 2007), increase conversation exchanges (Brown, Krantz, McClannahan, & Poulson, 2007; Charlop-Christy & Kelso, 2003; Sarokoff et al., 2001), teach empathic statements (Argott, Townsend, Sturmey, & Poulson, 2008), and increase social initiations (Krantz & McClannahan, 1998; Stevenson et al., 2006). In most experimental applications of scripts, researchers have sought to improve communicative independence, such as initiations and statements prompted by common objects, familiar settings, and conversation partners. Because the procedures were designed to reduce children’s dependency on vocal prompts, the majority of studies employing variations of script training have included only participants with autism.
There are many ways scripts can be used to minimize the intrusiveness of a hovering clinician. Scripts can be embedded in picture-activity schedules (Krantz & McClannahan, 1998; Stevenson, Krantz, & McClannahan, 2000). On a page of a child’s activity schedule, the text “Come play with me” could be written beneath a peer’s picture. The child, prompted by the text, invites that peer to play by saying, “Come play with me.” Scripts can also be used during classroom activities, such as art projects (Woods & Poulson, 2006) or snack time (Sarokoff, Taylor, & Poulson, 2001). Variations of textual scripts include cue cards or automated auditory prompts called audio scripts. Cue cards are note cards with the child’s textual script that can be presented at appropriate times in a conversation to prompt specific comments or questions (Charlop-Christy & Kelso, 2003) or paired with events that are natural cues for conversation like someone entering the room (Matson & Francis, 1994; Matson, Sevin, Box, Francis & Sevin, 1993). Voice recorded audio scripts have been used for individuals with minimal reading skills (Stevenson et al., 2000) Audio and textual scripts promote independent initiation of conversation because they do not require the presence of another person to deliver the model (Green, 2001).
When the child is able to use the full script correctly, the clinician can fade it from back to front by repeatedly cutting or removing the last word (Krantz & McClannahan, 1993, 1998; Stevenson et al., 2000). The gradual fading of scripts helps to facilitate the continuation of conversation in the natural context after scripts have been removed. Eventually, the entire script can be removed and a child will independently initiate conversation with his or her friends.
Script training is designed specifically to improve conversational independence, but primarily of children with autism and not children with SLI. Because Liam exhibits social limitations similar to those characteristic of autism, Heather may consider it as a possible intervention strategy. Evidence-based practitioners recognize the importance of integrating research evidence with their clinical expertise and child-specific factors. Therefore, prior to implementing a script training intervention, Heather needs to find out how dependable the evidence is that supports this strategy, how meaningful the improvements have been, and how to design a script training intervention suitable for her client. The primary purpose of this brief is to examine the script training research for study quality and intervention effect. The secondary purpose is to consider the potential of generalizing a well-researched strategy outside the strategy’s research base.
Searching for Evidence
Inclusion Criteria
Google Scholar and the Academic Search Premier database were searched using the following search terms to locate studies: initiation, script, script fading, language impairment, social communication, social interaction, conversation, and spontaneous. To be included, all studies had to meet the following criteria: (1) the study was empirical, written in English, and published in a peerreviewed journal; (2) children with language disabilities (including SLI and autism) were included as participants; (3) the researchers investigated textual or auditory script prompts as the primary independent variable and these interventions were investigated separately from other packaged components; (4) one of the primary dependent variables measured some aspect of conversational independence, such as initiations or conversational responses prompted by naturally-occurring objects, statements, settings, or people, and these outcomes had to be reported separately from other dependent measures. Studies that included combined unprompted and prompted responses, untrained responses that were not necessarily unprompted, responses to initiations, and answers to questions were eliminated. Fourteen scripttraining intervention studies were located and eight of them met all four criteria. Those that were not reviewed did not meet criteria three and/or four. Although study design was not part of the selection criteria, all of these script training studies employed single-subject research designs. None of the participants in these studies had SLI diagnoses.
Identifying Current Best Evidence
After the studies were gathered and evaluated based on the inclusion criteria, a three-step process was used to identify the current best evidence. First, each article was compared to a set of quality indicators to determine if it was acceptable evidence. Second, for each acceptable study, the percent of non-overlapping data points (PND) method was used to estimate the intervention effect. Third, the acceptable studies were summarized and evaluated to determine the adequacy or overall strength of the current evidence, based on the number of studies and total number of participants.
Step 1: Measuring Methodological Quality
A set of indicators for evaluating the quality of single-subject research from The Use of Single Subject Research to Identify Evidence-Based Practice in Special Education (Horner, Carr, Halle, Odom, & Wolery, 2005) was applied to the included studies. Horner et al. described necessary features of acceptable single-subject research and outlined minimal quality indicators. In a table, they listed seven broad categories and 21 specific indicators for determining whether a study is of acceptable quality to use in an evidence-based practice review (see Horner et al., 2005 for details). The seven appraisal categories include: (1) participants and setting, (2) dependent variable, (3) independent variable, (4) baseline, (5) experimental control/internal validity, (6) external validity, and (7) social validity.
Step 2: Intervention Effect
The number of high quality research studies supporting a given intervention is one important aspect of the best available research evidence, and the size of effect is another critical aspect. A practitioner should consider interventions that have the potential for powerful and meaningful affect on target behaviors. In step 2, the size of effect in each study was estimated by using the percent non-overlapping data (PND) method. PND involves identifying the highest baseline data point and calculating the percent of intervention data points that are higher (Mastropieri & Scruggs, 1985-1986). It is important to note that several methods for estimating effect in singlesubject research are available and few are without limitations (Wolery, Busick, Reichow, & Barton, 2010). The PND method was selected for this review because PNDs can be calculated quickly with the data presented in graphs and it is a metric commonly reported in the assessment of treatment effects.
PND scores were derived for each data set (or each leg of a multiple baseline design) that represented a dependent variable of interest (initiations, unprompted responses, etc.). PND scores were calculated for intervention, generalization, and maintenance conditions and means were reported for each study (see Table 1). Based on Scruggs and Mastropieri’s (1998) recommendations, PND scores above 90 were interpreted as a very effective intervention; scores between 70 and 90 as an effective intervention; scores between 50 and 70 as questionable; and interventions with scores below 50 as ineffective. There were a total of 14 PND scores from all the comparisons in the six acceptable studies; eight scores were in the very effective range and the remaining six were in the effective range. The highest PND was 100% (there were three of these) and the lowest was 75%. Overall, the high PND scores indicate that script training interventions have produced strong and meaningful improvements on conversational independence of children with autism. In addition, these strong effects included test of both maintenance and generalization.
Step 3: Criteria for Determining the Adequacy of Evidence
In step 3, the set of acceptable studies were evaluated for the overall strength of the current body of literature. Horner et al. (2005) suggested that evidence supporting an intervention can be considered adequate if: a) experimental control is demonstrated across a minimum of five singlesubject studies that meet methodological standards; b) at least three researchers across three different geographical locations conducted the studies; c) the total number of studies included at least 20 different participants. The research evidence for script training was slightly less than the Horner et al. (2005) criteria because there were only 18 participants included in the group of acceptable studies. The script training research exceeded the standards on the other criteria; six studies were conducted by four sets of researchers and in four geographical locations.
Integrating the Evidence to Make a Decision
Even though the research evidence supports the use of script-training interventions, the evidence-based decision-making process is not complete. Heather must integrate this information with her own clinical expertise and Liam’s characteristics (Gillam & Gillam, 2006; Sackett, Rosenberg, Muir Gray, Haynes, & Richardson, 1996). Before she decides, Heather should consider her skill in designing and managing a script-training intervention, whether Liam displays the appropriate skills necessary to benefit from script training, and the feasibility of implementing script training during lunch, art class, and at recess.
Because all the script training research was conducted with children with autism, the appropriateness of script training for Liam, who does not have autism, is a key consideration. As evidence-based practice movement gains momentum, clinicians continue to be pressured to employ practices that have been shown to be effective through a sufficient number of high quality research studies. Clinicians are asked to select and apply what works to improve a specific behavior, with a specific population, and in a specific setting. In reality, there are few areas of research within education and treatment of children that are so thoroughly studied to be able to provide answers to such narrowly defined practical questions.
Regardless of the strength, breadth, and depth of research evidence, a degree of inference from research to practice is necessary and, in most clinical fields, valued. Because limited areas of research, such as script-training interventions with children with SLI, offer challenges during the selection of an intervention method, the clinician must use his/her clinical judgment to extrapolate from the research that is available. The clinician is ultimately responsible for selecting interventions with the best available evidence, matching the research parameters with the client’s characteristics and circumstance, and adapting the strategy to be suitable in the immediate context (Cook, Tankersley, & Harjusola-Webb, 2008). The role of professional judgment and clinical experience in evidence-based practice is epitomized in Heather’s responsibility to make a decision regarding the appropriateness of script training for Liam.
In this situation, the extent to which recommendations from the script training literature can be extended to different populations may be more importantly tied to target behaviors and child characteristics rather than to specific diagnoses. Heather reviews the researchers’ descriptions of participants and finds that their descriptions of participant behaviors match Liam fairly well. All the participants had similar issues of limited social initiations and topic maintenance with conversation partners. Textual scripts require reading skills, but some studies employed recorded audio scripts with pre-readers. Liam has adequate reading skills; therefore, he is a candidate for textual scripts. Another consideration is Liam’s tolerance of physical prompts. Initial teaching of script use would be done by manually prompting Liam to run his finger under the text while reading it aloud. If Liam does not do it already, Heather may need to manually orient Liam’s head toward his conversation partner. Children vary in responsiveness to manual guidance. Fortunately, Liam tolerates brief physical prompts from familiar adults. Lastly, successful script-users typically have adequate self-management skills. Liam independently manages his homework folder and will likely be able to manage a set of scripts successfully, too.
Heather considers how difficult script training procedures are to implement and what new skills she might need to acquire. Because scripts are response prompts, just like vocal prompts, they are implemented in a very similar fashion. Heather has extensive practice and skill delivering timely prompts. She can apply this skill when delivering scripts, the only new wrinkle is that the form of the prompt changes from spoken to written. The content of the scripts may require some thoughtful consideration of what Liam might need or want to say to his friends. In art class, for example, Liam might want to say, “Look what I’m drawing” or “What is your favorite animal?” At lunch, Liam may want to say, “I had fun today in music class. Did you like beating the drums, too?” Constructing scripts, however, would be as simple as writing them on a piece of paper or printing them from a computer. When Liam is comfortable using the full scripts independently, Heather can fade the scripts by reprinting them with the last word or two removed. Even if the situation changes abruptly, Heather can write new scripts for Liam on note cards, hand them to Liam, and stand back to monitor. Overall, scripts should be easy to construct, use, and adjust to Liam’s changing needs.
After concluding that she has the necessary skill to manage a script-training intervention, and Liam has the necessary skills to benefit from script training, Heather considered the setting. Heather would like to continue working with Liam in natural contexts when his friends are nearby. Initial teaching will require her to be present and active in the setting. After Liam has mastered using the scripts, Heather will withdraw from the conversation context. In some of the script-training intervention studies, initial training was conducted with adults prior to introducing scripts in more natural contexts. Heather may find it more efficient to provide some initial training to Liam in her clinic room before introducing scripts in the natural environments. However, when Liam understands that he needs to read the script aloud and orient to his conversation partner, the procedures can be transferred to lunch, art class, and recess. Standing a few feet behind Liam, Heather can wait until Liam needs to be directed to use a script before manually guiding him to point to an appropriate script. Because her goal is to reduce Liam’s dependency on her prompts, it is very important that Heather does not stand between Liam and his conversation partner or talk to Liam during his conversations with his friends. Using these strategies, Heather can minimize her intrusiveness and still provide effective support as needed.
As an evidence-based practitioner, Heather integrates all these sources of information and decides to try a script training intervention to increase Liam’s conversational independence. Heather recognizes that by trying out a script-training intervention with Liam, she is accepting a level of risk, making a number of assumptions necessary for generalizing from research to practice, and assuming the responsibility for the ultimate effectiveness of the intervention.
Implementing and Monitoring the Intervention
After a clinician chooses an intervention method, he or she is responsible for implementing it and monitoring its effect. Responsible implementation of evidence-based interventions should always incorporate frequent and regular progress monitoring. Even when interventions have been identified as evidence-based, there is no guarantee that they will work similarly in applied contexts. It is critical that clinicians are vigilant in monitoring the effect of interventions with a particular client and in the specific setting in which it is applied (Cook et al., 2008). Because of the risk Heather has assumed and assumptions she has made when selecting script training for Liam, careful monitoring is especially important. Heather needs to confirm that script training is, in fact, improving Liam’s conversational independence. To do this, Heather needs to conduct regular observations of Liam’s conversational independence during natural conversation contexts with his peers and record the number of times he initiates conversation with a friend or maintains the conversation after a friend asks a question within a specified time interval.
Before beginning the script training intervention, Heather conducted several observations during lunch, art class, and at recess. She continued to observe the same behaviors in the same settings, and for the same amount of time after she began the intervention. After several more observations, she reviewed her monitoring data to determine the effect of script training on Liam’s conversational independence. When the observation context is the same before and after the implementation of an intervention, the number of initiations and conversation maintenance questions/comments can be compared across conditions. If Heather’s monitoring reveals an increase in initiations following the onset of intervention, then script training procedures are likely working. In which case, Heather should continue the intervention and regular monitoring until Liam has reached his IEP goals. If, however, Heather does not observe an increase, then she has another decision to make. Using professional judgment, Heather may decide that she needs to give the intervention more time before the desired effect is observed. She may also examine whether she is implementing the script training procedures correctly and make adjustments to improve the consistency with which she delivers the intervention. As another option, Heather may consider increasing the intensity of intervention to help facilitate Liam’s conversational independence. Finally, if Heather’s observations indicate that script training is not likely to be effective for Liam, she should return to the research literature and search for a more suitable strategy.
