Wednesday, October 28, 2015

NAEP and MisNAEPery Roundup

No shortage of eduwonks have opinions on today's release of 2015 scores on the National Assessment of Educational Progress (NAEP), which showed that American students' proficiency in reading was basically flat and proficiency in math was slightly down.

Commentary can be divided into two camps. One group is understandably disappointed in national dips but sagely taking the long view.  Another group seizes the scores for political gain and, as Morgan Polikoff notes, engages in misNAEPery:
Some folks are out with screeds blaming the results on particular policies that they don’t like (test-based accountability, unions, charters, Arne Duncan, etc.). Regardless of what the actual results were, they’d have made these same points. So these folks should be ignored in favor of actual research.[1] In general, people offering misNAEPery can only be of two types: (1) people who don’t know any better, or (2) people who know better but are shameless/irresponsible. Generally I would say anyone affiliated with a university who is blaming these results on particular policies at this point is highly likely to be in the latter camp.
First, a roundup of the former group:

Arne Duncan in the Washington Post: "Big change never happens overnight. I’m confident that over the next decade, if we stay committed to this change, we will see historic improvements.”

Duncan also reminded the Wall St. Journal that "Massachusetts faced an initial drop in results after raising its standards two decades ago, but persevered and became a top performer."

Another top performer was DC:
Among cities, District of Columbia Public Schools saw some of the strongest gains in fourth-grade reading and math since 2013, building on a rise from 2011. Chancellor Kaya Henderson credited a more rigorous teacher-evaluation system, targeted professional development, a strong curriculum and a major investment in early childhood education. She said 90% of 4-year-olds and 72% of 3-year-olds now attend free preschool.
“The work we have done to radically improve the caliber of our teacher force and principal force is paying off,” she said.
More from Morgan Polikoff:
These results are quite disappointing and shouldn’t be sugar-coated. Especially in mathematics, where we’ve seen literally two decades of uninterrupted progress, it’s (frankly) shocking to see declines like this. We’ve become almost expectant of the slow-but-steady increase in NAEP scores, and this release should shake that complacency. That said, we should not forget the two decades of progress when thinking about this two-year dip (nor should we forget that we still have yawning opportunity and achievement gaps and vast swaths of the population unprepared for success in college or career).
Eric Lerum cautions in Education Post that we shouldn't get sucked into one-year shifts and, instead, look at the big picture:
Overall NAEP scores since 2003? They’re up.
And a report released just this week finds that while NAEP results have increased more than should be expected given demographic shifts during that same period, the results for similar students vary widely across states.
Finally, since we’re now solidly in the Common Core-era, it makes sense to examine the alignment between Common Core standards and the questions asked on the NAEP exam. Another new study tackles this issue and finds that although there is reasonable overlap when it comes to mathematics, the NAEP exam may be due for an update to better reflect what we expect kids to be learning in the classroom.
Aaron Pallas via Chalkbeat, an education professor at Columbia University, commented on New York State and New York City's NAEP scores, which didn't move in reading and trended slightly lower in math:
New York City’s scores are near or above the averages for other large urban school systems in all categories except for fourth-grade math, but lag well behind national averages. Experts say, though, that the lack of growth since 2013 isn’t especially meaningful.
“On challenging assessments like NAEP it’s really hard to move the needle,” said Aaron Pallas, a professor of sociology and education at Teachers College at Columbia University. “You’re not likely to see big changes over relatively brief periods of time.”
Matthew Chingos told  the Wall St. Journal,
Some researchers cautioned against seeing the declines as reflecting specific initiatives because so many factors can affect test results, including the economy, local school budgets and demographic shifts. For example, the percentage of test-takers who were poor and Hispanic rose. The results “are useful for keeping a pulse on student achievement,” said Matthew Chingos, a senior fellow at the Urban Institute. “The casual punditry of eyeballing the scores and saying this is because of Common Core, testing or teacher evaluations—that doesn’t make any sense.
And then there's the latter group that posits, implausibly, that stagnant scores point to deficits directly attributable to elevated expectations for students and teachers.

AFT President Randi Weingarten in the Washington Post:
Not only is there plenty of anecdotal evidence that our kids have suffered, these latest NAEP scores again show that the strategy of testing and sanctioning, coupled with austerity, does not work,” Weingarten said in a statement. 
And Weingarten in the Los Angeles Times:
The scores "should give pause to anyone who still wishes to … make competition [and] scapegoating teachers … the dominant education strategies," Weingarten said in a statement.
She called for a "reset on education policy" following President Obama's announcement this weekend that standardized testing should not take up more than 2% of class time.
Weingarten's remarks are echoed by Carol Burris, head of Diane Ravitch's Network on Public Education:
Today’s National Assessment of Educational Progress score flop should come as no surprise. You cannot implement terrible education policies and expect that achievement will increase.
And Diane Ravitch herself:
The federal tests called the National Assessment of Educational Progress released its every-other-year report card in reading and math, and the results were dismal. There would be many excuses offered, many rationales, but the bottom line: the NAEP scores are an embarrassment to the Obama administration (and the George W. Bush administration that preceded it).

You get the idea. Do we look to Massachusetts and D.C., which have implemented higher standards for several years and demonstrate increased student proficiency in reading and math? Or do we look to novice aspirants (i.e., just about everyone else) that have yet to move the needle? Do we acknowledge that American public schools have far to travel before we serve all students well or do we blame disappointing results on efforts to do exactly that?

Everyone wishes (well, maybe everyone except anti-reform spinmeisters) that the NAEP results were better. They're not. The lesson here isn't to abandon nascent reforms but to buckle down until all American children are effectively served by our public schools.


1 comment:

  1. From Diane Ravitch:

    "Rick Hess, senior education fellow at the conservative American Enterprise Institute, did not join the crowd of apologists and excuse-makers who downplayed the meaning of the 2015 NAEP scores. He said the scores were "dismal," "a train wreck," and "mass carnage."

    He noted that Secretary Duncan was in "damage control" mode.

    Hess writes:

    "Viewed against more than two decades of prior scores, these results can only be described as a train wreck. They were so disturbing mostly because we’ve gotten so used to steady improvement in NAEP scores. Never before had fourth-grade math scores declined. Eighth-grade reading scores hadn’t fallen since 1996. Fourth-grade reading scores haven’t dipped since 2003, or eighth-grade reading since 2005. In other words, the widespread carnage on display this year is wholly unprecedented. The Obama administration, which has bragged about the efficacy of its federally fueled school-reform agenda, immediately moved to aggressive damage control. Secretary of Education Arne Duncan explained that the declines should in no way raise questions about Obama-promoted education policies such as Common Core or the administration’s Race to the Top program. “Big change never happens overnight,” Duncan said. “I’m confident that over the next decade, if we stay committed to this change, we will see historic improvements.”

    Duncan’s insistence that it will take a while for Obama policies to bear fruit would be more compelling if he had not — just last week— already credited Obama policies such as the School Improvement Grant program with boosting the nation’s graduation rate. Or if, two years ago, he hadn’t credited administration policies for 2013’s NAEP gains. At the time, he said, “All eight states that had implemented the state-crafted Common Core State Standards at the time of the 2013 NAEP assessment showed improvement . . . and none of the eight states had a decline in scores.” He added, “Tennessee, D.C., and Hawaii have done some really tough, hard work, and it’s showing some pretty remarkable dividends” on the NAEP results."

    Other commentators blamed the recession of 2008, but Hess pointed out that the recession did not affect scores in 2009 or 2011.

    There are many reasons why scores go up or down, but whatever they may be, NAEP 2015 is a disaster for Duncan and Obama's years of boasting about their education agenda. You can't claim credit when scores go up but disown the scores when they go down.

    Having heard about the fierce urgency of reform for seven years, now we are told that it will take a decade to see the fruits of high-stakes testing and racing to the top. That doesn't sound like fierce urgency. It sounds like the worn-out status quo, making excuses for failure.

    My own hunch about the test-score stagnation is that teachers, principals, students, and schools were confused by the constant disruption that reformers prize. Children and schools need consistency and stability. That's precisely what reform could not deliver.

    Corporations can re-invent themselves, but schools must be a refuge from a world of uncertainty. To import that uncertainty into the school does not improve education. It turns the schools into heartless, soulless, cold institutions where teachers and principals come and go; staff members disappear. The school itself may close and vanish.

    Turmoil is not conducive to teaching and learning.

    Read more at: http://www.nationalreview.com/article/426222/reading-math-national-assessment-educational-progress-arne-duncan"

    Source: http://dianeravitch.net/2015/10/31/rick-hess-naep-scores-are-a-train-wreck/

    ReplyDelete