Researchers and educators hear a lot about the importance of experimental research, but experimental studies can seem like expensive efforts just to answer “Did it work?” especially if it the answer is “no.” Fortunately, because of the teachers who participated in one experimental study, we didn’t have to stop at no. We were able to go on and ask “why didn’t it work?” and “what can we study now?”
In 2006–2011, we evaluated a textbook-based professional development program in classroom assessment. The study featured an instrument designed to measure teachers’ classroom assessment practices by systematically collecting and rating samples of student work. Teachers sent in four examples of anonymous, assessed student work that included their feedback and a cover sheet that asked them to describe the assignment and how it was assessed. While teachers in the treatment group increased their assessment knowledge and their use of student-involved assessment, student mathematics scores did not increase relative to the control group. Therefore, the answer to “Did it work?” was “mostly no.”
However, during the study, we heard spontaneous feedback from participating teachers that they wanted to see others’ work samples. Some teachers mentioned that the assessment textbook presented few examples from mathematics, so although they felt they could apply the formative assessment techniques to language arts and social studies, it was more difficult with mathematics. They commented that peer review of mathematics assessments could be more effective and could sustain greater interest over time than studying a textbook. This feedback got us thinking about the next step—developing a new professional development program in formative assessment for middle school mathematics called AWSM (Assessment Work Sample Method). It is job-embedded, mathematics-specific, and features supportive peer review of authentic work samples. We’re just getting started with a pilot school now and are excited to see how the program develops.
We are always grateful to teachers who participate in our studies. Taking extra time to participate in data collection for research is only one of many ways that teachers demonstrate their commitment to the education profession. Researchers would know very little about education without the teachers everywhere who graciously allow us into their classrooms or fill out surveys and participate in interviews. They also help us figure out what’s next when we’re searching for answers after an study.
If you’re a teacher or school administrator, have you participated in a research study? What was your experience with it? What makes you decide to participate?
Written by Andrea Beesley, McREL senior director in research & evaluation.