Category Archives: Research Insights

The next frontier of education: Implementation

In my latest “research says” column in Educational Leadership, I report that a new slew of “gold-standard” studies has unearthed (somewhat inadvertently) that in a lot of cases, educators really aren’t very good at the whole implementation thing. The studies, commissioned by the Institute for Education Sciences within the U.S. Department of Education, were carefully constructed with impressive sample sizes and rigorous statistical analyses. They found little or no effects for several popular education programs, such as Odyseey Math and Rick Stiggins’ Classroom Assessment for Student Learning.

Yet, almost without exception, the programs in question were so poorly implemented that it’s difficult to determine if they—or the poor implementation—were the reason for the weak results. In other words, the programs might have actually worked had they only been implemented with fidelity.’

This may be true of many education approaches and reforms, which ultimately get thrown on the trash heap because we believe they don’t work, when in reality, they may work just fine when they’re implemented well.

On the upside, we have seen a lot of improvements in education (for example, great teaching and curricula that challenge and engage students, to name just two) that can have a tremendous impact on student success … when done well. In fact, most of the big impact approaches aren’t new at all. For decades, we’ve known that teachers setting high expectations, being a “warm demander,” and intentionally matching instructional strategies to learning goals really do work. We just need to do these things correctly and stick to them.

Educators might take some solace in knowing that they’re not alone in struggling to do what everyone knows must be done. Businesses have the same trouble. Everyone in the airline industry knows Southwest Airline’s open secrets of success, such as their “all aboard” seating; yet few, if any, competitors have been able to effectively follow Southwest’s formula. Doing things right, of course, is a thorny challenge. Yet, it’s not impossible—in fact, we know quite a lot about the keys to good implementation.

So the good news is this: we don’t need to wait for silver bullets, or Superman, or some yet-to-be invented innovation to improve our schools. As we show in the video below, we simply need to do better what decades of research says matters most to change the odds for student success.

Bryan Goodwin is the author of a new book from ASCD, Simply Better: Doing What Matters Most to Change the Odds for Student Success.

Saying no to fads—in dieting and education reform

Simply Better: What Matters Most to Change the Odds for Student Success offers not a new “fad diet” for education, but rather the education reform equivalent of a “healthy lifestyle”—those things that decades of research says are most likely to have a big effect on student achievement.

Read More

What’s good for the body is good for the mind

First Lady Michelle Obama tours the country speaking of healthy eating habits, Dr. Oz answers your health questions on daytime TV, and the USDA recently updated the food pyramid. As obesity rates rise, healthy living is front page news. Then why are schools cutting physical education (PE) programs? That answer has also been front page news: budget cuts and falling academic scores. Schools need to do more with less, and cutting PE leaves more time and money for academics. In California alone, according to a policy brief released in May by the UCLA Center for Health Policy Research, 1.3 million teens in California do not participate in any school-based PE classes.

However, research shows that PE may be just what students need to perform better at school. Researchers Kathryn L. King, MD, and Carly J. Scahill, DO, from the Medical University of South Carolina Children’s Hospital implemented a program among 1st through 6th graders at low-performing schools in South Carolina that incorporated academic skills into physical activity. For instance, younger children used scooters to trace shapes on the ground, and older children climbed a rock wall outfitted with changing numbers to help them solve math problems. Students were engaged in this program for 40 minutes a day, five days a week. At the end of the year, test scores improved from 55 percent to 68.5 percent proficient.

John Medina, author of Brain Rules (2008), cites a similar study that examined the brain power of children before they began an exercise program. The children began jogging 30 minutes two or three times a week and, after 12 weeks, their cognitive performance had improved significantly. Perhaps just as important, when the exercise program was taken away, children’s scores plummeted back to pre-activity levels.

Because students are expected to learn more and more information at an increased rate, they need all the brain power they can create. Scores keep falling regardless of the programs and strategies schools implement—not unlike a “check engine” light that keeps appearing because, no matter how many times you take it to the shop, the mechanic isn’t fixing the actual problem. Maybe the mechanic is even making the problem worse.

Have you noticed the academic effects of cutting physical education in your school? Is more academic time a viable reason to cut ancillary programs?

New review of McREL’s The Future of Schooling

Dave Orphal, over at the Learning 2030 blog, offers this nice review of McREL’s latest book, The Future of Schooling.

In his review, Orphal praises the book for its timeliness. He notes, for example, that one of the critical uncertainties identified in the book—whether the outcomes of education will be standardized or differentiated—is currently playing out in the “movement to national common core standards” being countered by critiques from “Sir Ken Robinson and Daniel Pink who argue that standardization is exactly the wrong direction to go.”

Orphal also praises the book for its balanced view on these issues, noting that the authors take “great pains to not reveal where they stand in some of the hottest educational debates raging the country.” He adds, “Neither pro-Rhee nor pro-union; neither pro-testing nor pro-authentic assessment; neither pro-charter nor anti-charter, there is plenty in this book to anger every side of our overly partisan educational reform circles.”

Our intent is not to anger anyone. Rather, it’s to provoke thinking about what the future may hold, to move people out of their comfort zones so that they can begin to prepare themselves for what may lie ahead. As we write in the book, “Some of these potential futures may capitvate and energize you; others may dishearten and frigthen you. Some may do all of the above. That’s the point.”

Read Orphal’s entire review here.

Teacher evaluation wars: How North Carolina gave peace a chance

As a growing number of cash-strapped districts face staffing cuts, district leaders are pondering the potentially negative impact of “first in, first out” rules for layoffs. The concern, of course, as highlighted in a recent study by Dan Goldhaber at the Center for Education Data and Research is that letting teachers go based solely on seniority will likely result in some good teachers losing their jobs while less effective ones remain in the classroom. And as Marguerite Roza at the Center on Reinventing Public Education has determined, laying off teachers at the bottom of the pay scale requires larger job cuts to balance budgets. The impact on students of letting go the newest teachers instead of lowest-performing ones, according to Goldhaber, could be an estimated 2.5 to 3.5 months of learning per year.

So why don’t districts take teacher performance into account when making difficult reduction-in-force decisions?

One reason is collective bargaining rules—those hundred page documents that dictate all sorts of rules and procedures about hiring and firing teachers. Another is that many districts, even if they could dismiss ineffective teachers, often don’t know who they are.

For starters, as The New Teacher Project has noted, many teachers are not evaluated every year. On top of that, when teachers are evaluated, a sort of “grade inflation” exists with many current teacher performance evaluations. An examination of teacher evaluations in Colorado, found for example, that nearly 100 percent of teachers receive favorable ratings on their performance reviews.

To cut through the confusion, many reformers (as well as the federal Race to the Top program) have called for teacher evaluations to be based on actual student test results. Using just the right combination of data and statistics, the thinking goes, would allow us to create a “quarterback rating” of teachers, so we would let us know which ones are great, which ones need improvement, and which ones should be shown the door.

However, using this approach is fraught with all sorts of design challenges—for example, how do you measure the performance of an art teacher, a guidance counselor, or an eighth grade social studies teacher, when there are often no standardized exams in those subjects? Moreover, in New York City, Los Angeles, and elsewhere, serious concerns have emerged over the accuracy of the student data and whether teachers are even being tied to their own students.

In addition—and this is no small obstacle—calls for tying teacher evaluation to student achievement quickly become mired in contentious debate, with battle lines being drawn between teacher groups, leaders, and reformers.

It doesn’t have to be so difficult, though.

A few years ago, officials in the state of North Carolina took a different approach. From the start, they brought everyone together—teachers, school leaders, academics and researchers. They started with a premise on which everyone could agree: the challenges of the 21st century require everyone to rethink teaching and learning, and, as a result, teachers must develop new skills to prepare all children for the future. In other words, North Carolina did not base its reform efforts on a punitive notion of ridding the state of bad teachers, but rather, on a positive vision that by working together, teachers, administrators, and policymakers could transform education for the state’s children.

Working together, North Carolina defined new, rigorous standards based on research about good teaching. Then they designed an evaluation system that aligned to those standards. The result is a set of “stretch” goals for teachers with clear a roadmap for how to get there. Leaders of the states’ teachers association and school leaders agree that the system, while ambitious, is also reasonable and fair. Read more about the efforts here and here.

The North Carolina experience demonstrates that states and districts are likely to get farther, faster if they base conversations about teacher evaluation on three basic assumptions on which, I think, we can all agree:

  1. Every child deserves a great teacher.
  2. No one becomes a great teacher overnight; it takes practice, clear guidance, and coaching.
  3. And as with any profession, not everyone has what it takes to be a great teacher.

Bottom line: Teacher evaluations should be as much about developing teachers as they are about grading them. If we start from these premises, the North Carolina experience suggest that as John Lennon once sang, we can give peace a chance, and more important, give our teachers the support they need, and our kids the teachers they deserve.

Bryan Goodwin is McREL’s Vice President of Communications and Marketing.

Putting a little mystery in teaching

Want to hear a simple, surefire way to get kids interested in what you’re teaching?

First, think back to your childhood. For kids, the world can be a wonderful, mysterious place. That’s why, as any parent knows, children are naturally full of questions. Why is the sky blue? Why do I dream? Why do birds fly south for the winter? The list goes on and on.

As we grow up, we solve these mysteries and fill our heads with facts. Over time, we start to forget what made things so interesting to us in the first place. As teachers, it’s easy for us to take a Joe Friday “just-the-facts, ma’am” approach to teaching. As a result, we blow the suspense for children. We come right out and tell them the answers to the mystery, rather than building their interest by posing questions such as, “Have you ever seen a shooting star? What do you suppose that is?”

A few years ago, Robert Cialdini,  a psychologist at Arizona State University, wrote an article titled, “What’s the secret device for engaging student interest? Hint: The answer is in the title.” After sifting through dozens of dry science articles, Cialdini found that engaging science writers take a different approach: they pose a question, for example, “What are the rings of Saturn made of? Rock or ice?” Then they build suspense and mystery before finally resolving the mystery. The answer, in this case (spoiler alert!), is both.

Teachers, can, of course, do the same thing in their classrooms. Instead of coming right out and providing kids with the answers, they can build suspense in all kinds of subject areas, not just science. For example, in social studies, a teacher might offer this mystery: How could a rag tag army of volunteers (the American revolutionaries) defeat the world’s greatest superpower at the time (the British empire)? In math, a teacher might get kids wondering how to calculate the area of a circle. Gee … wouldn’t it be great if there were some kind of “magic” formula for that?

At two upcoming events—a lecture here in Colorado on Jan. 15 and a free, national, NASA-sponsored webcast on Jan. 20—McREL staff members will offer up some big space science mysteries (and their answers), helping teachers think about how to design their lessons around these mysteries.

So as you plan your next lesson, you might ask yourself, what’s the mystery here?

Bryan Goodwin is McREL’s Vice President of Communications and Marketing.

The rise of randomized trials… stay tuned

In his This Week in Education blog today, Alexander Russo wonders if the education community will finally “move off its duff” and begin conducting more scientifically based research.

His point is well taken: unfortunately, there’s still not a lot of top-quality, “gold standard” research out there to help educators answer important questions about which programs or approaches work and which don’t.

The good news, though, is that’s beginning to change. We are on the verge of seeing a deluge of new randomized control trials emerging in the coming months from the Institute of Education Sciences regional laboratory program (for which McREL adminsters the laboratory for the Central region of the U.S.).

To see a list of these studies (23 in all, three from the REL Central at McREL), go here.

And stay tuned.

“Superman”: You’ve heard the hype; have you read the research?

Even before its October 8 release, “Waiting for ‘Superman’” was everywhere—on the cover of Time, on “Oprah,” in news commentaries across the country. The film’s message—which presents charter schools as a (if not the) solution to our school systems’ ills and teacher unions as part of the problem—has resonated nationwide. But, as those who work in education know, reform is a tricky subject and not one to be answered in 111 minutes.

In a New York Review of Books critique of the film, Diane Ravitch made this point by referring to a charter school study by the Center for Research on Education Outcomes (CREDO), which indicated that “17 percent were superior to a matched traditional public school; 37 percent were worse than the public school; and the remaining 46 percent had academic gains no different from that of a similar public school” (p. 1).

So what did you think of “Waiting for ‘Superman’”? Do you have a charter school experience to share?

If you’d like to read up on the topic, here are links to some related articles and reports:

Charter Schools: Inputs and Impacts in Charter Schools: KIPP Lynn

The Harlem Children’s Zone, Promise Neighborhoods, and the Broader, Bolder Approach to Education

The Challenge of Leading Two Generations within the Teachers Union

The real lesson from NYC small schools

Smaller isn’t better; personalized is better

An MDRC study that came out in June reporting on the impact of New York City’s small schools of choice initiative has recently appeared in the spotlight again, thanks to a September 27 commentary in Education Week from Michelle Cahill of the Carnegie Corporation and Robert Hughes of New Visions, a public education network affiliate in New York City. It was also picked up in this morning’s Public Education Network newsletter.

The title of the Ed Week commentary, “Small Schools, Big Difference,” may raise some eyebrows, though, especially for those who remember the Gates Foundation’s $1 billion misadventure with small schools.

The disappointing results of this effort eventually prompted then-director of the Foundation’s education programs, Tom Vander Ark, to tell Education Week that, “I visited 100 great schools and made the observation that they were all small, autonomous, and assumed that was a path to school improvement. It turns out that giving a failing school autonomy is a bad idea.”

Yes, the small schools in New York City are showing promise—their students (the vast majority of whom are poor and minority) have a 6.8 percent higher graduation rate than a similar group of students in the city’s mostly large, comprehensive high schools.

All of that is good news and worthy of further examination and, probably, replication.

The headline given to Cahill and Hughes’ Ed Week commentary, however, is only partially correct. The authors of the MDRC report actually caution against concluding simply that small schools are better. They write,

Students enrolled in SSCs [small schools of choice] did not just attend schools that were small. SSC enrollees attended schools that were purposefully organized around smaller, personalized units of adults and students, where students had a better chance of being known and noticed, and teachers had a better chance of knowing enough about their charges to provide appropriate academic and socioemotional supports.

In other words, saying that smaller schools lead to higher achievement is sort of like saying wearing sneakers leads to weight loss. What’s more important is what you do in the schools (or your sneakers). (In fairness to Cahill and Hughes, their commentary is more nuanced than the headline given to it).

The real takeaway of the MDRC report is that creating learning environments where students know their teachers and pursue studies that interest them (most of the small schools are designed around career themes) is what has shown promise, not the size of their student population.

Indeed, the same could be said of the Gates’ small schools initiative; as David Marshak, a professor of education at Western Washington University, observed in a February 19, 2010 commentary in Education Week, many of the small schools funded by the Gates Foundation did show gains in student achievement; the key to their success was “a culture of personalized education.”

Incidentally, this finding mirrors a key conclusion of the McREL report, Changing the Odds for Student Success: What Matters Most, in which we observe that a key principle for curriculum design is to provide students with multiple, intrinsically motivating, pathways to college and career readiness.

Bryan Goodwin is McREL’s Vice President of Communications.