Evidence-based practice (EBP) refers to
clinical decisions as a result of the
careful integration of research evidence
and student needs (Dollaghan, 2007;
Dunst, Trivette, & Cutspec, 2002; Sackett, Rosenberg,
Muir Gray, Haynes, &
Richardson, 1996). Legal mandates
such as No Child Left Behind require
teachers to employ evidence-based
practices in their classrooms, yet teachers receive
little guidance regarding
how to determine which practices are
evidence-based and most appropriate
for their individual students (Lerman,
Vorndran, Addison, & Kuhn, 2004;
National Research Council, 2001).
Often practices are chosen based on
teachers’ expertise or on the availability of resources rather than on the
integration of research evidence and student-specific factors (Schroeder &
Schroeder, 1990). For this reason it is
critical that teachers learn how to
access research evidence and incorporate that with student variables to
make intervention decisions.
A seven-step decision-making
process adapted from Gillam and
Gillam (2006) was used to illustrate
how different intervention decisions
may result from the same research evidence (see Table 1). This process
begins by developing parameters to guide the search for research evidence.
An Evidence-Based Decision-Making Example
(external evidence). The second step includes conducting a search and determining which studies will be further evaluated. Once relevant research is located, each study is evaluated for quality and findings are summarized (step three). In steps four and five the internal evidence (i.e., student, family, teacher, and school factors) are considered as they relate to each case. Integrating the evidence and making a decision occurs in step six, and monitoring the outcome of the decision is step seven. This process will be illustrated using case studies of children with autism spectrum disorder (ASD).
Autism Spectrum Disorder (ASD)
An estimated one third to one half of all children who have ASD do not use functional speech to communicate (Light, Roberts, DiMarco, & Greiner, 1998; National Research Council, 2001). When children with ASD do not readily acquire speech, teaching them to use augmentative and alternative communication (AAC) is an appropriate option. The American SpeechLanguage-Hearing-Association (ASHA) defines AAC as the following:
A set of procedures and processes by which an individual’s communication skills can be maximized for functional and effective communication. This involves supplementing or replacing natural speech and/or writing with aided (e.g., picture communication symbols, line drawings, Blissymbols, and tangible objects) and/or unaided symbols (e.g., manual signs, gestures, and finger spelling). (ASHA, 2002, p. 98)
Aided systems incorporate the use of some type of device ranging from the simple (paper, pictures) to the complex (computers, voice-output devices), whereas unaided systems involve the individual’s body. There are advantages and disadvantages to both systems. For example, aided systems use pictures or words that are easily deciphered regardless of background knowledge. However, aided systems are sometimes cumbersome because of their technological complexity or the amount of materials necessary to maintain them. Unaided systems can be motorically difficult for some children to produce but are more readily accessible.
A commonly used aided AAC system is the Picture Exchange Communication System (PECS; Bondy & Frost, 2002; Charlop-Christy, Carpenter, Le, LeBlanc, & Kellet, 2002), and a popular unaided symbol system is sign language (M. Sundberg, 1993; M. Sundberg & Partington, 1998). PECS incorporates the use of a series of icons or pictures and sign language is a system of standardized gestures for use in communication. Teachers are often faced with having to make a decision about whether or not to use PECS or sign language with children with ASD. Because selecting an alternative communication system is a common and important task for teachers, it was chosen as a model. Three hypothetical children (Juan, Stephanie, and David) will be used as case examples to demonstrate how teachers can incorporate the use of a decision-making process to make evidence-based decisions about interventions for students with ASD and specifically how the internal evidence (e.g., student, family, teacher, and school factors) from each case shapes each decision.
Case 1: Juan Student and Family Factors
Juan is a 4-year-old boy with ASD who attends a preschool for children with disabilities. A full-time aide is assigned to assist him throughout the day. Juan receives 30 minutes of individual speech language intervention each day and 60 minutes of occupational therapy each week. He currently communicates by pulling, gesturing, hitting, or crying and he frequently has tantrums. His speech is severely limited, with occasional one-syllable vocal utterances. Although Juan’s motor ability is slightly impaired, he can imitate three to four fine motor movements. Due to difficulties obtaining valid results, his intellectual ability is unknown. Juan is unable to discriminate between multiple graphics and/or symbols and pictures and is unable to match objects to pictures. Juan’s parents defer decisions regarding his education to the school personnel and appear to be open to trying most intervention methods.
- Internet Resources for Evidence-Based Practice
- What Works Clearinghouse: http://ies.ed.gov/ncee/wwc/
- Institute for Education Sciences
- http://www.ed.gov/about/offices/list/ies/ncer/index.html
- National Center for Special Education Research
- http://ww.ed.gov/about/offices/list/ies/ncser/index.html
- American Speech-Language-Hearing Association
- http://www.asha.org/members/ebp
- Center for Evidence-Based Practice: Young Children With Challenging Behavior: http://challengingbehavior.fmhi.usf.edu/resources.html
- NICHCY (National Dissemination Center for Children With Disabilities) Research to Practice Database: http://research.nichcy.org/search.asp
- National Standards Project: http://www.nationalautismcenter.org
Teacher and School Factors
Juan’s school does not have any particular policy regarding alternative communication. His teacher and speech– language pathologist have experience using both PECS and sign language, and they are interested in determining whether an aided or unaided AAC system would be best for his specific circumstances.
Step 1: Develop Parameters to Guide the Search for Evidence. The purpose of this process is to solve a problem or answer a specific question. The following question was addressed: Which form of augmentative alternative communication is most effective in promoting language development in children with ASD who do not use functional speech to communicate? It is important to delineate the parameters for the search. Three major parts should be included: (a) child characteristics, (b) the intervention or practice, and (c) the outcome of interest. For the current example, (a) the learner can be described as a child with ASD, (b) the intervention or practice includes both PECS and sign language communication training, and (c) the desired outcome involves an improvement in communication.
Step 2: Search for Evidence. The second step is to search for research that includes the three components from step one. Teachers are recommended to first search the available databases that compile evidence-based practice information. See box, “Internet Resources for Evidence-Based Practice” for Web sites, many of which continually update and post new practices identified as evidence-based and some which provide teachers with guidelines for implementing those practices.
Because evidence-based practice databases are still young, it is possible that the intended practice has not yet been reviewed. If after a search of known databases yields nothing on practice of interest for the population of interest, teachers still have an obligation of finding whatever research is available. Teachers can do this by entering search terms into convenient electronic databases, such as Google Scholar at http://scholar.google.com or the Educational Resources Information Center (ERIC) database at http://www. eric.ed.gov/. In most cases it is not possible to perform an exhaustive search because of teachers’ limited time and resources. The goal is, by necessity, to gather a few studies that are a “good match” to the student, the practice of interest, and the desired outcome. A more comprehensive search for articles would be expected from a team of academic professionals whose task is to synthesize all that is known about a practice.
We searched for meta-analyses and systematic reviews first, because if found, the authors would have likely conducted thorough searches and reviewed several relevant studies. Meta-analyses and systematic reviews are considered a valuable source of evidence and can be similar to the research syntheses conducted by evidence-based practice organizations. An unpublished dissertation/meta-analysis (Wendt, 2006) and a research review (Millar, Light, & Schlosser, 2006) were found using the search terms augmentative and alternative communication, autism, and meta-analysis. In addition, an unpublished dissertation (Anderson, 2001) and a fairly recent study (Tincani, 2004) were found using the Google Scholar search engine using the search terms autism, sign language, and Picture Exchange Communication System. Several studies (Adkins & Axelrod, 2001; Hodges & Schwethelm, 1984; C. T. Sundberg & Sundberg, 1990; Wraikat, Sundberg, & Michael, 1991) considered as external evidence were cited in the reference section for Tincani. All full-text articles and unpublished dissertations were available without cost via links on Google Scholar.
After a quick review of the collection of articles and dissertations, we eliminated several. Hodges and Schwethelm (1984), C. T. Sundberg and Sundberg (1990), and Wraikat et al. (1991) were eliminated because the participants in each of these studies were either adults with developmental disabilities or children with mental retardation, which did not exactly fit our student characteristics. The systematic review conducted by Millar et al. (2006) was eliminated because they did not compare effectiveness of the different types of AAC systems. Instead they made conclusions about the effects of AAC systems (aided and unaided) on speech production. The remaining studies and dissertations specifically compared sign language and PECS and employed children with ASD as participants.
Step 3: Evaluate Each Study for Quality and Summarize Findings. Like the search for evidence in step two, the evaluation process probably cannot be extensive. Nonetheless, it is important to distinguish between poor and highquality studies because the quality of the study directly impacts the confidence in the results. It is too simplistic to assign a “high-quality” rating based exclusively on the study design (Dunst et. al., 2002). In general however, studies employing randomized control group designs and single-subject designs with multiple participants can better establish a causal relationship between the intervention and the observed outcome. Conversely, correlation studies and case studies are less capable of demonstrating a causal relationship. In addition to the design employed, there are a few general questions that teachers can ask to help determine whether a study is of high quality. For example: How strong is the relationship between the intervention or practice and the outcome? Is the effect replicated across multiple individuals? Can alternative explanations for the outcome be ruled out? To be considered high quality, studies need to show a causal relationship between the intervention and the outcome, demonstrate replication across several individuals, and provide sufficient experimental control to suggest that no other reasons for the outcome are feasible. Moreover, if an intervention and its relationship to the outcome have been demonstrated by more than one researcher and across multiple studies, more confidence can be given to that intervention’s effectiveness.
In step two of the evidence-based decision-making process, three studies and one meta-analysis comparing sign language and PECS for further evaluation were included. While reviewing the three individual studies, the quality questions were answered. Two out of the three studies were judged to have sufficient evidence of causality and replication, and adequately ruled out alternative explanations for the outcomes observed. In the excluded study the authors did not establish a causal relationship between the interventions and the outcome. Furthermore the intervention was applied to only one participant (Adkins & Axelrod, 2001).
In the meta-analysis, Wendt (2006) reviewed 44 studies related to AAC applications for individuals with ASD: Of those studies, 37 were retrieved from peer-reviewed journals and all but one of those studies employed singlesubject designs. One of the primary purposes of the meta-analysis was to compare various AAC modalities to determine their relative effectiveness in teaching functional requesting skills and speech production in individuals. who have ASD. Both PECS and sign language were identified as being highly effective in increasing functional requesting. The meta-analysis also revealed that the use of signs may be more effective than PECS in promoting speech production.
In Tincani (2004), two children with autism who did not use functional speech to communicate served as participants. A single-subject alternating treatment research design was used to demonstrate the effects of sign language and PECS training on the percentage of independent requests and the percentage of vocalizations. Tincani found sign language to be superior for one participant and PECS to be the better choice for the other participant. In addition to addressing the relative effectiveness of sign language and PECS, Tincani examined the relation of pre-existing skills (e.g., motor imitation) and success with each type of communication system. He implemented an imitation assessment to compare participant motor imitation abilities with the acquisition of requests. The participant with average imitation skills demonstrated more independent signs than PECS exchanges. In contrast, the participant with weaker motor imitation skills demonstrated more PECS exchanges than independent signs. It is interesting to note that both participants had a higher percentage of vocalizations when using signs.
Anderson (2001) investigated the differential effects of sign language and PECS on rates of acquisition, spontaneous use, eye-contact, generalization, and vocalizations. Six children with autism (ages 2–4) served as participants and were exposed to both training conditions. Similar to Tincani (2004), Anderson investigated child characteristics related to performance with each type of system. Results of this study include: (a) participants preferred one system over the other, (b) both types of systems produced adequate rates of acquisition, and (c) specific pretraining skills contributed to the posttraining performance. For example, rate of acquisition of PECS was associated with joint attention.
and rate of acquisition of sign language was associated with imitation and functional play. Additional benefits of PECS included “success with a broader range of children, faster rates of acquisition, and better generalization to novel items,” whereas benefits of sign language training involved “higher levels of initiation, eye-contact, and vocalization at post-treatment” (Anderson, p. x).
Summary of External Evidence
Wendt (2006) concluded that training in either PECS or sign language was highly effective, but that one advantage of sign language over PECS was increased vocalizations. Tincani (2004) and Anderson (2001) took the analysis a step further by assessing participants’ pretraining skills such as motor imita tion. They offer agreement that both sign language and PECS are effective interventions for use with children with ASD and that learner characteristics may determine which system is best. It is possible that motor and imitation skills are critical to the acquisition of sign language because the teaching procedures rely heavily on the fine motor imitation of signs. At this point, there is not enough evidence to make definitive conclusions about which type of system is most effective. However, considering the available evidence, it appears that sign language communication training may promote vocal language acquisition better than PECS training given adequate motor imitation ability.
Step 4: Consider Student and Family Factors. Upon reviewing Juan’s individual case, the student-specific factors that were considered to be most relevant included his ability to imitate three to four fine motor movements; his limited speech with some vocalizations; his difficulty with matching and discriminating between multiple graphics, symbols, and pictures. Relevant parent factors included their apparent willingness to try different intervention methods and their tendency to defer decisions regarding Juan’s education to school personnel.
Step 5: Consider Teacher and School Factors. Important teacher and school factors specific to Juan’s case include his placement in a specialized preschool class and access to a full-time aide. It is also important to note that Juan’s school does not have any particular policy regarding which type of AAC method to use, and his teacher and speech–language pathologist appear capable of supporting the use of both PECS and sign language.
Step 6: Integrate the Evidence. The external evidence indicates that sign language may be the superior choice when considering Juan’s current skills. Juan is able to imitate fine motor movements and emits some vocalizations. These may be positive indicators for success with sign language. Juan’s family does not have any particular opinion regarding the AAC system introduced to Juan, and his special education teacher and speech– language pathologist appear to be willing and capable of using either AAC system. It was therefore recommended that Juan learn sign language as an alternative mode of communication.
Step 7: Evaluate the Outcomes. Juan’s teachers implemented a sign language program and taught his parents how to encourage signs at home. After 1 month, Juan had acquired six signs—juice, candy, cracker, ball, movie, and mommy. As Juan makes requests, vocal approximations accompany each of the signs. Juan’s parents and the teaching staff are pleased with his progress and they continue sign language training and daily monitoring.
The next case study demonstrates how teachers could arrive at a completely different intervention decision using the same external evidence based on the integration of the student’s unique internal evidence.
Case 2: Stephanie Student and Family Factors
Stephanie is a 5-year-old girl diagnosed with ASD. She is currently attending a public school. For 60% of the day she attends a special resource class for children who have ASD and for 40% of the day she attends a general education kindergarten classroom with a parttime aide. In addition, Stephanie receives 90 minutes of speech and language intervention and 30 minutes of occupational therapy each week. She does not use speech to communicate and typically limits her communication to protesting (e.g., tantrums and hitting). Stephanie’s fine motor skills are moderately impaired and she does not imitate vocal or motor actions. Furthermore, she is unable to match objects to pictures. Based on the Stanford-Binet Intelligence Test, her intellectual abilities fall below average. Stephanie’s parents are very involved in her education and are apprehensive about sign language.
Teacher and School Factors
Her resource classroom teacher currently uses PECS for many of the students in the class and has never used sign language. The school district has a full-time AAC representative who typically encourages the use of PECS before advancing to a computer-assisted device.
Steps 1–3: External Evidence. The same practical question asked in Juan’s case is posed for Stephanie. Because the search parameters will be the same (i.e., children with ASD, comparing PECS and sign language, language development), the external evidence is also the same. The results of the first three steps for Juan yielded adequate support for both sign language and PECS for children with ASD. Sign language training is suggested for students with adequate motor imitation and in general promotes vocalizations more than PECS training.
Step 4: Consider Student and Family Factors. Stephanie also appears to have important student factors to consider. For example, she is unable to match objects to pictures or imitate speech and fine motor movements. Her limited speech and her method of communication appear to be very similar to Juan’s; however, her fine motor skills and intellectual abilities are moderately impaired. Potentially relevant parental factors include their apprehension regarding the use of sign language and their apparent desire to be highly involved in making decisions concerning Stephanie’s education.
Step 5: Consider Teacher and School Factors. Stephanie’s resource classroom teacher currently uses PECS for many of the students in the class and does not know any signs. Also, there appears to be school district support for the use of PECS.
Step 6: Integrate the Evidence. According to what is known about Stephanie’s pretraining skills, the evidence is inconclusive. Because she does not imitate (vocal or motor) and does not demonstrate strong matching skills, it is unknown which type of system will promote the best communication outcomes for Stephanie. Overall, the district, teacher, and family (internal evidence) are better prepared to support PECS. Considering all factors, PECS was deemed the more suitable AAC choice for Stephanie at this time.
Step 7: Evaluate the Outcomes. Stephanie’s kindergarten teacher, aide, and parents received training in PECS. They have implemented the training procedures for 2 months. Stephanie is able to consistently select the icon for music when it is in various positions and within an array of two distracter icons. She is working on icons for drink and eat, but has not demonstrated consistency at school. However, at home she is able to request drink and eat using the icons. Overall, her teachers and parents have seen a decrease in hitting and tantrums, and occasionally Stephanie vocalizes when she exchanges the music icon. The team is satisfied with the decision to use PECS and will continue to monitor her progress.
The last case study provides an additional example of how the outcome may be different depending on the student’s or family’s values and preferences. This case illustrates the potential for disagreement when working within multidisciplinary teams and with families, and it provides one way of managing the outcome in a professional and ethical manner.
Case 3: David Student and Family Factors
David is a 7-year-old boy diagnosed with ASD. He attends a private day school for children with ASD and receives 45 minutes of speech and language intervention a day and 30 minutes of occupational therapy a day. Despite the intensity of speech therapy, David has not acquired any vocalizations. His intellectual abilities are significantly impaired and he is unable to imitate any motor or vocal responses. Overall, David makes very few attempts to communicate with others and his social interactions are extremely limited. In his special education class, the only skills he has been able to acquire include following simple one-step directions and matching pictures. Because David’s cousin, who also has a diagnosis of ASD, had great success learning sign language, David’s parents requested that he be taught sign language.
Teacher and School Factors
David’s school district does not have any policy or programs in support of PECS or sign language. The speech language pathologist and occupational therapist believe that David is a candidate for an assistive technology device that would involve David touching a button that produces a voice output. In order for David to learn to use the device, it would be necessary to first teach him how to point to pictures of the things he wants.
Steps 1–3: External Evidence. As with Juan and Stephanie, the search parameters and summary of findings apply to David. Both PECS and sign language have adequate support with a few outcome benefits of using sign language (e.g., increased vocalizations). Adequate motor imitation skills appear to be prerequisite to successful acquisition of sign language.
Step 4: Consider Student and Family Factors. When considering David’s case, it is important to recognize that he has significantly impaired intellectual abilities and is unable to imitate any motor or vocal responses. His ability to follow simple one-step directions and to match pictures should be weighed. Other important student-related factors include his limited social engagement with others and his history of slow skill acquisition. David’s parents have requested that he be taught sign language—a significant family factor.
Step 5: Consider Teacher and School Factors. Specifically for David, it would be important to consider that the speech language pathologist and occupational therapist believe that he is a candidate for an assistive technology device, and that the school district does not appear to advocate the use of one type of system over another.
Step 6: Integrate the Evidence. Because David is unable to imitate motor movements or sounds and rarely engages others socially, it is less likely that he will be successful with sign language. The family would like the staff to teach him sign language yet the school personnel believe that a picture-based system would yield more functional communication for David. In a case such as this, when families and school personnel disagree on the path, school personnel should educate the family on the available research and outline the reasons to support their clinical opinion. Nevertheless, if the family maintains their position, the school personnel should, under most circumstances, comply with the family’s desires. Unless the family’s requests are inappropriate or potentially detrimental to the child, student– family factors generally outweigh teacher–school factors (Gillam & Gillam, 2006). Although David’s teachers and therapists agreed to teach sign language according to his parents’ wishes, they made one request. They asked that if after 3 months of teaching sign language sufficient progress was not made, they could reconsider the option of teaching PECS or adding PECS to his communication training.
Step 7: Evaluate the Outcome. At the 3-month marker, David required a great deal of manual prompting to form the signs for candy and cookie. No progress was noted in regards to social engagement or vocalizations. Because of David’s limited progress, the staff and family met to discuss the next step. Together they decided that adequate progress had not been made and to try PECS for a 3-month trial period. Daily progress monitoring will allow the team to evaluate the relative effectiveness of the selected communication training and to make continual decisions about what is most appropriate for David.
Conclusions
It is important to consider internal evidence along with existing external evidence when implementing intervention. However, once initial decisions are made, the process does not end. Evaluating the outcomes of an intervention is an ongoing process. Evidence-based teachers maintain regular progress monitoring in order to make decisions based on student performance and gauge the effectiveness of their interventions (Strain & Dunlap, 2006). It may be necessary to reenter the process when students gain skills or change classrooms. As new research becomes available and databases are refined, it will be critical for teachers to stay current with relevant research topics and continue to integrate research evidence with factors unique to individual students.
Teachers are engaged in EBP when they integrate findings from quality research studies with student–family and teacher–school factors to inform the decisions that they make about interventions with students (Dollaghan, 2007; Dunst et al., 2002; Sackett et al., 1996). We used a structured, evidencebased decision-making process to determine the most appropriate AAC system using three case examples. With each case, a different outcome resulted, all of which were based on the available external evidence and the relevant internal evidence. The purpose of this article was to provide teachers with an example of an evidence-based decision-making process that can be applied to a variety of practical questions. As we demonstrated using the seven steps outlined in this article, evidence-based decisions extend beyond the resources and preferences of teachers. It is critical to integrate best available research with the circumstances posed by students, families, and teachers.