Credit: iStockphoto.com
Credit: iStockphoto.comAgainst the backdrop of enthusiasm regarding new reforms underway in California, from the Common Core to the Local Control Funding Formula, the just-released scores on the National Assessment of Educational Progress, or NAEP, brought a brush with reality.
Mirroring national results, scores in California on 4th-grade math dipped by 2 points and in 8th-grade math by 1 point compared with 2013, the last time the NAEP (pronounced nape) was administered. In reading, scores were flat in 4th-grade but dropped by 3 points in 8th-grade.
California continued to rank near the bottom compared with other states. In 4th-grade reading it ranked 49th, and ranked 43d in 8th-grade reading. In math, the state ranked near the bottom as well.
Media outlets published the scores with the usual depressing headlines.
“California’s decade of gains on this test just ended,” read one Los Angeles Times story.
“California test scores in the cellar,” read another in the San Jose Mercury News.
But what exactly do these scores tell us? It turns out that much depends on which scores one chooses to focus on, what time frame one looks at, and whether one looks at growth in scores rather than at scores at fixed points in time.
Those who don’t analyze tests for a living are likely to be confused.
For example, Terry Mazany, the chair of the National Assessment Governing Board, said that despite the dip in scores this year, scores are far higher than when the test was first administered in 1990. Mazany said the “big story” of this year’s tests was the narrowing of the test-score gap between large urban school districts and the rest of the nation.
Former California State Superintendent of Public Instruction Bill Honig said that the state is “still among the fastest-growing states since 2009 in 8th-grade scores.” California also leads the nation, along with Washington, D.C., in 8th-grade growth in reading scores, Honig said. Those scores have gone up by 6 points since 2009, compared with a national growth of 1 point. California, Honig added, is also among the four highest states in its growth in 8th-grade math scores.
Optimist reformers who promoted the Common Core standards could reasonably have hoped that gradual implementation of the new standards in California classrooms, and those in most other states in recent years, might have nudged NAEP scores up even slightly.
That is especially the case in light of a report issued last week by the NAEP Validity Studies Panel indicating that there is a “reasonable overlap” between the NAEP and what the Common Core expects of students, at least on math.
But because the Common Core standards are designed to progress cumulatively from grade to grade, it will take several years before students will experience their full impact. The gains should be greatest for those children who begin in kindergarten and have the benefit of instruction that builds on the standards on each of the preceding grades.
So should the fact that NAEP scores did not rise be a cause for concern?
Here, too, the advice was to keep anxiety levels in check. Mazany and others cautioned against placing too much emphasis on one year’s scores as a signal of a downward trend.
“As a country, we have made progress over time,” Mazany said. “It is likely that there will be some ups and downs.”
“We don’t know yet if these changes … are long-term,” said Peggy Carr, acting commissioner of the National Center for Education Statistics, which administers the NAEP.
However, Chris Minnich, executive director of the Council of Chief State Schools Officers, which represents state education heads in all 50 states, emphasized the lack of progress. This year’s scores, he said, “confirm that we have a long way to go with our kids across the country.”
At the same time, he said, tests are “an important data point,” but that there are “other data points that are also important,” such as graduation rates.
President Obama made a similar warning on a video he posted on Facebook cautioning against the overuse of tests, and saying they should not be the only measure of a student’s or a school’s progress. “Tests should be just one source of information, used alongside classroom work, surveys and other factors to give us an all-around look at how our students and schools are doing,” he said.
The scores on the subset of big-city school districts, including three in California (Los Angeles Unified, San Diego and Fresno) also provided a “glass half-full/half-empty” analysis. In Los Angeles Unified, for example, reading scores were flat, and math scores showed a slight decline. The district’s overall test scores put it in the bottom third of 21 big districts nationwide whose scores were reported separately.
Yet, as the Los Angeles Times reported, scores in 8th grade reading for low income students in the district have risen more than in any other district in the sample since 2003 — some 16 points. In 4th grade reading, Latino students’ scores grew faster than those in the majority of those districts. African American scores in math also rose at an impressive rate.
Michael Casserly, executive director of the Council of Great City Schools, pointing to the narrowing of the gap between large urban school districts and the rest of the nation when looked at since 2003, said that in general we should be “encouraged by the progress so many of the cities have made.”
Is it even reasonable to expect students to do better academically each year?
That is a question that Gov. Jerry Brown, arguably the biggest skeptic about the entire testing enterprise among all the nation’s governors, has asked.
“They are getting little children at the age of 5 infected with this idea that everything is measurable, and that they are accountable every day to improve,” he said in May. “I can tell you that the idea that you can improve every day for the rest of your life is not true. I just think there is a bit of a life cycle. Things go up and go down.”
That seemed to be a distinctly minority view on Wednesday. “It makes sense for kids to improve every year,” said Minnich, representing the Chief State Schools Officers. “It is reasonable to expect to see scores going up every year.”
Wait for that to happen next year – or not.
To get more reports like this one, click here to sign up for EdSource’s no-cost daily email on latest developments in education.
Comments (6)
Comments Policy
We welcome your comments. All comments are moderated for civility, relevance and other considerations. Click here for EdSource's Comments Policy.
Gary Ravani 8 years ago8 years ago
NAEP scores were rising at about 4 points per test series prior to implementation of NCLB and nationally, minority groups were closing scoring gaps on the NAEP. After NCLB they mostly flatlined, until this last series where they declined somewhat. So nearly a decade and a half of imposed "test based accountability" has been a failure, if you want to pay attention to the NAEP. I don't find it a mere coincidence that the NAEP scores … Read More
NAEP scores were rising at about 4 points per test series prior to implementation of NCLB and nationally, minority groups were closing scoring gaps on the NAEP. After NCLB they mostly flatlined, until this last series where they declined somewhat. So nearly a decade and a half of imposed “test based accountability” has been a failure, if you want to pay attention to the NAEP.
I don’t find it a mere coincidence that the NAEP scores became known in DC, Arne Duncan announces his stepping down from USDE, and Obama publicly repudiates “over-testing.” It was truly unfortunate that via Race to the Top and the NCLB waiver process that the Duncan led USDE doubled down on the failed test score mania that overcame common sense, but at least Obama seems to have learned from past mistakes. Better late then never. The new guy at USDE left behind him a trail of instructional chaos in New York but, with any luck, his relatively brief tenure won’t allow for much more damage to the education system.
Perhaps now the education system can get back to letting teachers control the “what and how” of classroom instruction using CCSS as a loose roadmap. There is discussion of aligning NAEP content with CCSS content, so when implementation is completed in a reasonable time frame, perhaps five years, the NAEP will have some meaning that should be paid attention to and comparisons to SBAC will make sense.
Replies
Doug McRae 8 years ago8 years ago
Gary -- Could you provide documentation for the "rising about 4 points per test series" before NCLB and then "flatlined" statement? I do not remember it that way, so I looked that the NAEP website. The scores before 2003 were on an irregular schedule and they changed NAEP to permit accommodations to increase the number of Students with Disabilities tested, so one has to be careful one uses apples to apples comparisons from test … Read More
Gary — Could you provide documentation for the “rising about 4 points per test series” before NCLB and then “flatlined” statement? I do not remember it that way, so I looked that the NAEP website. The scores before 2003 were on an irregular schedule and they changed NAEP to permit accommodations to increase the number of Students with Disabilities tested, so one has to be careful one uses apples to apples comparisons from test year to test year, but after 2003 it has been a regular every 2 year schedule for both Reading and Math. In any case, the data I saw showed greater gains after 2003 for Reading, but smaller gains after 2003 for Math, a mixed bag. For both tests I did not see gains near 4 points per series, and only for Reading grade 8 did I see a 0 point gain (flatline) and that was before 2003. In any case, the data I saw did not support your description.
Gary Ravani 8 years ago8 years ago
Doug: If you google "NAEP scores flatline" you will come up with about a dozen citations. Some beginning around 2006 after 5 years of NCLB when the lack of any kind of success of the standards, testing, accountability (sic) imposition on educational practice became clear. But, policy makers insisted, along with the pundits, editorialists, billionaires, and self-styled reformers, on doing the same thing over and over (for a decade!) and expecting different outcomes. Here are a couple: Independent … Read More
Doug:
If you google “NAEP scores flatline” you will come up with about a dozen citations. Some beginning around 2006 after 5 years of NCLB when the lack of any kind of success of the standards, testing, accountability (sic) imposition on educational practice became clear. But, policy makers insisted, along with the pundits, editorialists, billionaires, and self-styled reformers, on doing the same thing over and over (for a decade!) and expecting different outcomes.
Here are a couple:
Independent Test Results Show NCLB Fails
• Submitted by fairtest on October 28, 2015 – 11:42pm fairtest on national k-12 national news whats new
INDEPENDENT TEST RESULTS SHOW NCLB FAILS
[This one, from Fairtest, contains a link to all relevant original documents.]
Eighth Graders’ Flatline on NAEP U.S. History, Civics, and Geography Tests
By Jessica Brown on April 29, 2015 12:01 PM
[This one, from EdWeek, relates to subjects other than math and ELA, which were the focus of most test accountability measures. Did I mention that narrowing of the curriculum (and learning) can be seen as major faults of NCLB type reform? Well, yes, I guess I have.]
Doug McRae 8 years ago8 years ago
Hey, Gary — Your documentation is less than impressive. Google searches are notorious for fewer returns before NCLB (i.e., 2003) than after, FairTest is notorious for anti-assessment bias, and citing NAEP Social Studies is not on-point for your claim above [tho I would agree narrowing of curriculum is a concern, not due to “NAEP type reform” but rather for due to inappropriate overuse of test scores].
Tom 8 years ago8 years ago
Thanks Doug for pinning down Gary on his statements presented as facts, which are really thinly veiled political agendas – per the usual from him. His comments were in your wheelhouse of expertise and we appreciate your efforts to clear the record.
Doug McRae 8 years ago8 years ago
Long time educational testing guru Bob Linn from U Colorado provided very practical guidelines for interpreting large scale assessment program scores about a dozen years ago with a study that found that gains of 3-4 points for very large programs (like NAEP and California's statewide assessments) can be called meaningful gains, that positive gains can be expected over time, that loses of 3-4 points can be called meaningful losses. The NAEP results this year are … Read More
Long time educational testing guru Bob Linn from U Colorado provided very practical guidelines for interpreting large scale assessment program scores about a dozen years ago with a study that found that gains of 3-4 points for very large programs (like NAEP and California’s statewide assessments) can be called meaningful gains, that positive gains can be expected over time, that loses of 3-4 points can be called meaningful losses. The NAEP results this year are mostly minus, so it’s wishful thinking or twisted logic to suggest they are a cause for optimism, but they also do not reach a minus threshold to call for hand-wringing. Not even a red flag, but certainly not a full-speed-ahead green flag. How about settling for a pink flag, a cause for caution.