Research to date has painted a mixed picture of the effectiveness of out-of-school-time (OST) programs (i.e., after-school and summer school programs). Recent syntheses of research on these programs have ranged from finding positive effects on student achievement to questioning the rigor of the research to finding no statistically significant impact on student achievement.
To get an overall picture of current research findings to date, McREL researchers analyzed all rigorous research conducted since 1984 on the impact of OST strategies in improving the reading and mathematics achievement of low-achieving or at-risk students. The research team identified 56 studies that met our criteria for inclusion by measuring student achievement in reading and/or mathematics and employing control/comparison groups.
McREL researchers quantitatively synthesized these studies using a meta-analytic method to estimate the overall expected impact of OST strategies on student achievement. Results were further analyzed for program characteristics that influence effect sizes, including timeframe (after school or summer school), grade level of students, focus of strategies (academic or academic plus social), duration of the OST program, and grouping of students (large or small groups or one-on-one tutoring).
Overall findings. The synthesis
resulted in positive effects of OST on student achievement that
were significantly greater than zero in both reading and math. Overall
effect sizes ranged from .06-.13 standard deviations for reading
and .09-.17 standard deviations for math, depending on the method
used for weighting sample sizes. These effect sizes can be statistically
converted into achievement gains of, respectively, up to 5 or 6
percentile
gains (see chart).
Although these effect sizes may appear to be small, it is important to note that they were found for programs that are of relatively short duration compared with the overall school day or calendar. In addition, these gains were achieved with students who most struggle to learn. Also, some programs had much larger effect sizes, as noted below.
Other influences. Researchers found that grade level was a statistically significant moderator of effect sizes for both reading and mathematics outcomes. For reading, the largest positive effect size (.26, a gain of 10 percentile points) occurred for students in the lower elementary grades (K-2). For mathematics, the largest positive effect size (.44, a gain of 17 percentile points) was for students in high school (9-12).
For reading outcomes, activity focus was not a statistically significant moderator of effect size. For math, strategies that were both academic and social had a slightly higher mean effect size than those that were mainly academic. For both reading and math, effect sizes were larger for OST programs that were more than 45 hours in duration. However, programs with the longest durations (210+ hours for reading and 100+ hours for mathematics) had the lowest effect sizes.
Overall, the largest positive effect size (.50,
a gain of 19 percentile points) occurred for the reading strategies
that used one-on-one tutoring. In summary, these findings suggest
that certain program features can result in even higher positive
effects of OST on student achievement.
