Portraits of Programs Under Study
Over a four year period, SII researchers used program documents, field research, and survey data to examine the design, implementation, and instructional effectiveness of the three CSR programs examined in A Study of Instructional Improvement. Not surprisingly, these analyses showed that each of the CSR programs studied had different “designs”—that is, ideas about the kinds of instructional practices they wanted to implement in schools and the ways they wanted to organize schools to support the process of instructional change. More importantly, we have found that these designs had important consequences for the schools working with these programs. For example, we have found that schools working with the different programs varied in: (a) how they were organized to manage instruction; (b) the kinds of literacy instruction they provided to students; and (c) the patterns of student achievement growth that occurred at different grade levels. Here, we summarize our findings in these areas and provide a preview of results that will be discussed in more detail in the final three sections.
The Accelerated Schools Project
In previous studies (Rowan, Camburn and Barnes, 2004; Correnti and Rowan, 2007; Rowan and Miller, 2007), we have argued that at the time of our study, ASP was using a pattern of “cultural controls” to bring about instructional change in schools. We use that description because ASP’s approach to working with schools revolved around promoting a normative commitment among school leaders and faculty to the program’s abstract vision or ideal of “powerful learning” for all students. Indeed, from the onset of working with schools, ASP facilitators used the staff development process to emphasize the program’s commitment to this abstract construct, and to define powerful learning as constructivist in nature, with an emphasis on authentic, learner-centered, and interactive forms of instruction. However, ASP was not prescriptive in nature. For example, it did not target particular school subjects for improvement, nor did it provide teachers with a great deal of explicit guidance about curriculum objectives or teaching strategies. Instead, ASP facilitators helped schools use a systematic process of organizational development to design a unique path toward powerful learning and to adopt locally-appropriate forms of instructional practice consistent with this approach. In this sense, ASP had a design best labeled as “adaptive” in form.
This description, it is worth noting, suggests that ASP’s approach to producing instructional change is at least partially at odds with many of the design features that previous research on educational innovation has identified as promoting implementation success. For example, at the time we studied ASP, the program’s goals for change were generic in form—aiming at broad changes across the board rather than targeting specific areas of the curriculum for change. By contrast, prior research on design-based assistance suggests that successful programs often focus on changing specific, curriculum-embedded elements of instructional practice as opposed to more diffuse elements that cut across curricular areas or represent generic forms of teaching (Cohen and Hill, 2001; Desimone, Porter, Garet, Yoon, and Birman, 2002; Fennema et al., 1996). Moreover, the kinds of changes teachers were expected to make as a result of participating in ASP were not formally specified, and instead, each school (and each teacher within a school) was asked to “discover” the most appropriate means to producing powerful learning within his or her own particular context. Again, this differs from the stance taken by more successful programs, which tend to have clearly defined goals for change in particular curricular areas, that is, a clear specification of what features of curriculum and instruction will be changed, and of the steps to be taken to achieve these changes (McLaughlin and Marsh, 1978; Elmore and Burney, 1997; Nunnery, 1998). For these reasons, schools and teachers had a great deal of autonomy in the ASP system, with the result that there was little real focus on implementation fidelity, either from external program facilitators, or from internal leaders. In fact, in previous research, we have found that ASP schools had the lowest reported levels of instructional leadership of all the schools in our study sample (Camburn, Rowan, and Taylor, 2003). This too contrasts with previous research, which suggests that successful programs are those in which program designers, local program facilitators, and local administrative leaders demand fidelity to planned changes in instructional practice (Loucks, Cox, Miles, and Huberman, 1982; Huberman and Miles, 1984; Stringfield and Datnow, 1998).
In light of these patterns, it is not surprising that a particular pattern of instructional improvement emerged in ASP schools. First, as we report below, staff in ASP schools reported that school improvement plans were weakly (not highly) specified, that is, they saw plans as centered more on shared values and group investigation than on specific curricular goals and objectives. In this environment, teachers in ASP schools were trusted to make innovations in their classrooms as they saw fit, so long as these innovations were consistent with the normative ideal of powerful learning. The autonomy afforded by this system of instructional control promoted high levels of motivation for improvement among faculty. But, as we demonstrate below, the approach did not promote implementation of distinctive forms of literacy instruction in schools; instead, we will show that literacy instruction in ASP schools looked very much like literacy instruction in our sample of comparison schools. Moreover, because instructional practices were so similar, patterns of student achievement were also quite similar across ASP and comparison schools.
The America’s Choice program took a contrasting approach to instructional change at the time of our study, stimulating instructional improvement using what we have called in previous papers a pattern of “professional controls” (Rowan, Camburn and Barnes, 2004; Correnti and Rowan, 2007; Rowan and Miller, 2007). The AC program had its origins in the standards-based reform movement, and as a result, the program was built around some definite ideas about the curricular content and methods of teaching it wanted to occur inside classrooms, especially in the area of language arts. At the time of our study, for example, AC typically began its work in local schools by focusing on the school’s writing program (moving only later to changes in reading and mathematics programs). Moreover, AC typically provided teachers with a great deal of instructional guidance. For example, teachers in AC schools received a curriculum guide, were taught a set of recommended instructional routines for teaching writing (called “writers’ workshop”), and worked with locally-appointed AC coaches and facilitators to develop “core writing assignments” and clear scoring “rubrics” for judging students’ written work. Thus, in the area of writing instruction at least, AC was trying to implement a well-specified, standards-based curriculum grounded in professional consensus about what constitutes a desirable instructional program. AC also expected schools that adopted the program to create two new leadership positions—a design coach and a literacy coordinator. Design coaches were expected to help principals implement the program, while AC literacy coordinators were expected to work with classroom teachers. Previous research showed that levels of instructional leadership were highest in the AC schools in our study sample (Camburn, Rowan, and Taylor, 2003).
Our findings (discussed below) show that the “professional” control strategy used by AC had strong effects in the schools where they worked. First, AC schools tended to organize for instructional improvement in ways that were quite different from schools in the control group or in ASP schools. For example, given all the instructional guidance that AC teachers and leaders received, it is not surprising to find that staff in AC schools viewed their school improvement plans as clear and well-specified. Moreover, as a result of extensive coaching, AC schools tended to be characterized by strong instructional leadership. But this pattern of strong leadership and clear instructional guidance came at a price—with teachers in AC schools reporting lower levels of autonomy of practice than did teachers in comparison group schools. Still, as we shall see, the presence of strong instructional leadership—coupled with a well-specified instructional design—produced distinctive instructional practices in AC schools. As our findings demonstrate, AC teachers were much more likely than other teachers to emphasize writing instruction as part of their literacy program, and to place more emphasis on students’ production of extended written text. This approach to literacy instruction, in turn, seemed to produce accelerated growth in students’ literacy achievement at the upper elementary grades.
Success for All
Success for All illustrates yet a third model for promoting instructional change in schools: what we elsewhere have called “procedural controls” (Rowan, Camburn and Barnes, 2004; Correnti and Rowan, 2007; Rowan and Miller, 2007). Of the three programs under study, SFA gave schools the clearest and most highly-specified plan for instructional improvement by producing a set of highly-specified instructional routines for the teaching of reading. In particular, the SFA program was built around a clear and well-defined reading curriculum that provided teachers with a weekly lesson sequence, and each lesson in this sequence was designed around a “script” intended to guide teaching activities through a 90-minute reading period. In grades K-2, moreover, these scripts were accompanied by program-provided curricular materials for use throughout the school.
SFA schools also were more centrally managed than other schools in our study. For example, schools implementing SFA were expected to appoint a full-time literacy coordinator, and this staff member was given substantial responsibility for school-wide coordination of the reading program, including the task of constituting reading groups and making teaching assignments to these groups on a school-wide basis every eight weeks. In addition, instructional leaders in SFA schools and SFA linking agents were asked to supervise implementation of SFA instructional routines. In prior research, levels of instructional leadership were found to be as high in SFA schools as in AC schools, and much higher than levels of instructional leadership found in ASP schools (Camburn, Rowan, and Taylor, 2003).
Given SFA’s approach to promoting instructional change, it is not surprising that the SFA schools in our study were characterized not only by distinctive patterns of organizing for school improvement, but also by distinctive patterns of literacy instruction and student achievement. For example, as we demonstrate below, staff in SFA schools saw school improvement plans as highly specified and as focused squarely on a particular instructional target (reading). In addition, school leaders worked mainly to monitor instruction, reducing teacher autonomy and standardizing instruction. This emphasis on faithful implementation of instructional routines, we found, produced a distinctive pattern of teaching practices that was generally faithful to the program’s instructional design. In particular, more than any group of teachers in our sample, SFA teachers enacted what we have called a “skills-based” reading program focused on direct/explicit teaching of reading comprehension strategies and an emphasis on having students demonstrate their reading comprehension through simple, direct responses to oral questions and/or short written work. As we shall see, this instructional approach appeared to accelerate students’ reading achievement in the early elementary grades.
Camburn, E., Rowan, B., & Taylor, J. (2003). Distributed leadership in schools: The case of elementary schools adopting comprehensive school reform models. Educational Evaluation and Policy Analysis, 25(4), 347-343.
Cohen, D. K., & Hill, H. (2001). Learning Policy: When State Education Reform Works. New Haven, CT: Yale University Press.
Correnti, R. and Rowan, B. (2007). Opening up the black box: Literacy instruction in schools participating in three comprehensive school reform programs. American Educational Research Journal, 44, 298-338.
Desimone, L., & Porter, A., & Garet, M., & Yoon, K., & Birman, B. (2002). Effects of professional development on teachers’ instruction: Results from a three-year longitudinal study. Educational Evaluation and Policy Analysis, 24(2), 81-112.
Elmore, R & Burney, D (1997). Investing in Teacher Learning: Staff Development and Instructional Improvement in Community School District #2. New York: National Commission on Teaching & America’s Future & the Consortium for Policy Research in Education.
Fennema, E., & Carpenter, T., & Franke, M., & Levi, L., & Jacobs, V., & Empson, S. (1996). A longitudinal study of learning to use children’s thinking in mathematics instruction. Journal for research in Mathematics Education, v. 27, 403-434.
Huberman, A. & Miles, M. (1984). Innovation Up Close: How School Improvement Works. New York: Plenum Press.
Loucks, S., & Cox, P., & Miles, M., & Huberman, M. (1982). Portraits of the changes, the players and the contexts. A study of the dissemination efforts supporting school improvement. People, policies and practices: Examining the chain of school improvement, Vol. II. Network of Innovative Schools, Andover, MA. ERIC no. ED240714.
McLaughlin, M., and Marsh, D. (1978). Staff development and school change. Teachers College Record, v. 80(1), 69-94.
Nunnery, J. (1998). Reform ideology and the locus of development problem in educational restructuring. Education and Urban Society, v. 30(3), 277-295.
Rowan, B., Camburn, E., & Barnes, C. (2004). Benefiting from comprehensive school reform: A review of research on CSR implementation. In C. Cross (Ed.), Putting the pieces together: Lessons from comprehensive school reform research (pp. 1-52). Washington, D.C.: National Clearinghouse for Comprehensive School Reform.
Rowan, B. and Miller, R.J. (2007). Organizational strategies for promoting instructional change: Implementation dynamics in schools working with comprehensive school reform providers. American Educational Research Journal, 44, 252-297.
Stringfield, S., & Datnow, A. (1998). Scaling up school restructuring designs in urban schools. Education and Urban Society, v. 30(3), 269-276.