This spring, all California’s students in grades 3-8 and in grade 11 will take brand new tests. These end-of-year standardized tests – known in the field as Smarter Balanced assessments – will be administered as part of a new, more comprehensive state accountability system to measure student progress toward college and career readiness.
This system is based on the new Common Core State Standards for English language arts and mathematics adopted by California along with 42 other states.
Here is the tough part: For students and parents, the increased rigor of the new standards will likely mean that fewer students will score “proficient” on these new tests than on the ones they have been taking for the past 15 years.
But lower scores on the tests should not be an indictment of the new Common Core standards. Rather, they should underscore the hard work that will be needed to ensure our students become critical thinkers who will not need remedial courses in college before they can get started on college-level work. Let’s remember that test scores were also much lower when the previous standardized tests – the California Standards Tests – were first introduced.
The new tests, known as the Smarter Balanced assessments, will be significantly different from what our students are used to – and much more interesting than the tests students have been taking until now. Standardized bubble tests will be replaced by tests that allow students to show much more precisely what they know and are able to do.
Our students will be challenged with assessments that measure how deeply they are able to understand the concepts and, more importantly, use their knowledge to solve real-life problems.
The challenge does not stop there. Educators and district and state administrators have their fair share of challenges as well. If we are to use these assessments for student growth, and not just accountability, educators will need to design and implement systems to use the results to improve teaching and learning at every school.
The state’s push to drive more decision-making to a local district level will mean, among other things, that district administrators will have to design new ways of reporting on multiple measures based on the state’s priority areas. These include academic proficiency, student engagement, school climate and parental involvement. That means no simple single number will be used to describe the entirety of a school’s performance, as was the case with the Academic Performance Index, which is now undergoing revision.
Finally, state administrators will have to design and invest in data systems to collect student and school data from the more than 1,000 districts in our state. And alongside county offices of education, state administrators should provide tools and resources to support each school’s process of continuous improvement.
None of the challenges above will be simple to solve. With a lot of policy and systems yet to be developed, it will take time for students, teachers, principals, superintendents and state officials to adapt to this new world.
Nevertheless, real magic is happening in thousands of classrooms all over our state as a result of the new Common Core standards. These new standards, along with teachers prepared to implement them and the assessments to see whether students are benefiting from them, hold real promise for offering our kids an education that will prepare them for college, careers and life.
If we, as a state, fail to build public will for these changes, if we fail to address the challenges and investment needs at every level of the school system, if we shy away from more rigorous standards and assessments, we will lose the opportunity to offer a world-class education to all of our kids.
We simply cannot afford that. Our students are worth everyone working harder – and together – to provide them a world-class education.
•••
Gina D. Dalma is the senior program officer at the Silicon Valley Community Foundation, and leads the Silicon Valley Common Core Initiative.
The opinions expressed in this commentary represent solely those of the author. EdSource welcomes commentaries representing diverse points of view. If you would like to submit a commentary, please contact us.
To get more reports like this one, click here to sign up for EdSource’s no-cost daily email on latest developments in education.
Moloko Nasos 8 years ago8 years ago
The eternal theme.
Students and tests.
A contradiction that can not be resolved.
Thank you
John R. Walkup (@jwalkup) 8 years ago8 years ago
I am all for new standards development in my home state of California. Although I have never been fond of the Common Core State Standards (I consider them weaker than California's past standards), I try not to judge them according to personal politics. A few months ago I devised a new scheme toward creating new standards based on a three-dimensional modification of the Cognitive Rigor Matrix, which now uses Webb's Depth of Knowledge and both … Read More
I am all for new standards development in my home state of California. Although I have never been fond of the Common Core State Standards (I consider them weaker than California’s past standards), I try not to judge them according to personal politics.
A few months ago I devised a new scheme toward creating new standards based on a three-dimensional modification of the Cognitive Rigor Matrix, which now uses Webb’s Depth of Knowledge and both the Knowledge Dimension and Cognitive Process Dimension of Bloom’s Taxonomy.
For those interested in this new scheme: http://cognitiverigor.blogspot.com/2014/10/white-paper-driving-standards.html
Don 8 years ago8 years ago
One last word to clarify, Navigio. LCFF provides SC grants based on Free and Reduced student qualification, not on performance. If F and R qualified students perform well what is the logical basis upon which they should continue to receive extra funding compared to similar performing students? That they are poorer? This is not unlike receiving unemployment benefits after employment is secured. There's nothing in LCFF ( re: SC grant allocation ) … Read More
One last word to clarify, Navigio. LCFF provides SC grants based on Free and Reduced student qualification, not on performance. If F and R qualified students perform well what is the logical basis upon which they should continue to receive extra funding compared to similar performing students? That they are poorer? This is not unlike receiving unemployment benefits after employment is secured. There’s nothing in LCFF ( re: SC grant allocation ) to address what happens if it actually succeeds in doing what it intends.
Gary Ravani 8 years ago8 years ago
The Manufactured Crisis: Myths, Fraud, And The Attack On America's Public Schools – August 26, 1996 by David C. Berliner (Author), Bruce J. Biddle The Manufactured Crisis debunks the myths that test scores in America’s schools are falling, that illiteracy is rising, and that better funding has no benefit. It shares the good news about public education. Disputing conventional wisdom, this book ignited debate in Newsweek, The New York Times, and the entire teaching profession. Winner … Read More
The Manufactured Crisis: Myths, Fraud, And The Attack On America’s Public Schools – August 26, 1996
by David C. Berliner (Author), Bruce J. Biddle
The Manufactured Crisis debunks the myths that test scores in America’s schools are falling, that illiteracy is rising, and that better funding has no benefit. It shares the good news about public education. Disputing conventional wisdom, this book ignited debate in Newsweek, The New York Times, and the entire teaching profession. Winner of the American Educational Research Association book award, The Manufactured Crisis is the best source of facts and analysis for people who care about what’s really happening in our schools.
The Manufactured Crisis debunks the myths that test scores in America’s schools are falling, that illiteracy is rising, and that better funding has no benefit. It shares the good news about public education. Disputing conventional wisdom, this book ignited debate in Newsweek, The New York Times, and the entire teaching profession. Winner of the American Educational Research Association book award, The Manufactured Crisis is the best source of facts and analysis for people who care about what’s really happening in our schools.
Righting Wrongs
Perspectives on Education in America,” better known simply as “The Sandia Report.”
Excerpted from From HuffPost by Gerald Bracey
The draft of The Sandia report closed with “There are many problems in American public schools, but there is no system-wide crisis.”
Almost precisely 17 years ago, Lee Bray, the Sandia vice president who had overseen the analysis flew up to Denver to show me the 156 page report. I had written an Education Week commentary arguing that the SAT decline had been much smaller than people thought (in fact, I was only riffing off the College Board’s own panel which had produced On Further Examination
What had actually happened was that the Sandia group had gone to Washington and presented the report to department of energy and department of education staff and some Congressmen. At the end, David Kearns, former CEO of Xerox and then Deputy Secretary of Education said, “You bury this or I’ll bury you.” An article in Education Week said only that “administration officials, particularly Mr. Kearns, reacted angrily at the meeting.” The article also contained allegations of suppression and denials of such (“Report Questioning ‘Crisis’ in Education Triggers an Uproar,” October 9, 1991).
The engineers did get buried, being forbidden at one point to leave New Mexico to talk about their findings. “Dead wrong” was how Secretary of Energy James Watkins (Energy funds Sandia) described the report in the September 30, 1991 issue of the Albuquerque Journal. “It is a call for complacency when just the opposite is required,” he said. (It amazes me that each time someone points out that the educational sky is not actually falling, those who say it is lose all capacity for logic and accuse that the non-Chicken Littles of being messengers of complacency.
Five years after Lee Bray retired, I called him. He was not enthusiastic about reopening old wounds, but when I asked him directly if the report had been suppressed he said, “Yes, it was definitely suppressed.”
The Education Week article on The Sandia Report closed with the prediction that “Administration officials will use a lengthy review process to bury the report.” Indeed, it was never published. It appeared in print only when Jim Raths of the University of Delaware and then an editor at the Journal of Educational Research made it the entirety of the May/June 1993 issue of that small journal. Because nearly 20 years have passed, most people don’t know either of the report or the suppression that followed. But that act of suppression sits like Banquo’s ghost at the banquet table, seen, in this case, not only by Macbeth, but by to those of us who witnessed the murder.
Paul Muench 8 years ago8 years ago
I keep hearing that SB will have fewer multiple choice questions. I assume that is what the article derides as “bubble tests”. But when I’ve asked for data on the proportion of multiple choice questions on the SB tests it seems as if multiple choice questions are still a large part of SB (70-80%). I’d sure love for someone to refute those numbers.
Replies
Gary Ravani 8 years ago8 years ago
Paul: I (as well as John and Doug) have been at several SBE meetings where this issue was discussed. My impression is that SBAC is stepping way from many of the more performance based kinds of questions because: 1) they are time consuming to administer; and, 2) they are costly to score e.g., you either need teachers to do it out side of school or you hire "non-professionals" to take on the task. In either case … Read More
Paul:
I (as well as John and Doug) have been at several SBE meetings where this issue was discussed. My impression is that SBAC is stepping way from many of the more performance based kinds of questions because: 1) they are time consuming to administer; and, 2) they are costly to score e.g., you either need teachers to do it out side of school or you hire “non-professionals” to take on the task. In either case there are personnel costs; and; the ‘extra time” factor means teachers/schools might not get the test data back in a timely manner.
There are some real horror stories about testing companies hiring “non-professionals,” working in a “boiler room” type of environments, and pressed to process as many tests as possible per hour, based on simplistic rubrics, rather than giving them the kind of scrutiny actual classroom teachers might give them.
I can’t give you the % breakdown of bubble-in items v. performance based items. I don’t recall it being mentioned though perhaps John or Doug heard it.
There is a kind of presumption that even the bubble items are more sophisticated than traditional bubble items and, too, there is the “adaptive” function that is supposed to make it all more “accurate” at placing students on a performance scale of some kind.
This is still based on other presumptions that we need state based tests, that students need to be sorted and on placed on scales of varying types, and that teachers really need the data, though most teacher don’t believe that. And all of this is wrapped up in the bundle of “accountability,” though that makes little sense (and we’ve had almost two decades to establish that) because of the “tail wagging the dog” fallacy that is the conventional wisdom. As I’ve often stated ETS (and various other legitimate sources) assert that school related factors amount to about 1/3 of test measured achievement. So, you have the 1/3 school/tail being held “accountable” for wagging the 2/3rds family/community/state/federal “dog.” The fact that all of this pseudo-accountability hasn’t worked out so well makes way too much common sense to be acknowledged by policy makers. To not talk about accountability has become one of those “third rails” of politics that should not be tossed aside lightly, it should be thrown aside with great force. But, oh well.
Paul Muench 8 years ago8 years ago
I’m also remembering a discussion in this forum saying that the adaptive features of SB won’t be ready any time soon.
Doug McRae 8 years ago8 years ago
Paul — The adaptive features promised by Smarter Balanced haven’t been available to date (for example, the “field” test last spring was not adaptive, and the interim testing component is not adaptive this year), but the summative tests this spring for which the testing window opens next week will be computer-adaptive according to the latest information that I have seen . . . .
Doug McRae 8 years ago8 years ago
Paul, Gary -- Smarter Balanced has advertised their tests are designed to be 70 % multiple choice or machine scored, 30 % performance tasks or human scored. I haven't seen those figures broken down by grade by content area, but I'm sure there are differences from grade to grade and from ELA to Math. I do know, however, that the grade 3 ELA exercise that I did as part of the crowd sourcing standards-setting exercise … Read More
Paul, Gary — Smarter Balanced has advertised their tests are designed to be 70 % multiple choice or machine scored, 30 % performance tasks or human scored. I haven’t seen those figures broken down by grade by content area, but I’m sure there are differences from grade to grade and from ELA to Math. I do know, however, that the grade 3 ELA exercise that I did as part of the crowd sourcing standards-setting exercise last October had 85 % M/C or machine scored items and only 15 % P/T or human scored items, and it reflected the blueprint for the grade 3 ELA test.
Re Gary’s description of horror stories of testing companies hiring non-professionals vs the scrutiny that classroom teachers provide for scoring tests, Gary’s biased rhetoric doesn’t tell the whole story at all. Folks running testing programs like to minimize costs, and hiring highly qualified but not full-time teachers as volume scorers has been an efficient way to minimize costs for human scoring for large scale tests, yet generate reliable results. Full time teachers by contrast have to be paid regular wages, several times more costly on an hourly basis, and mostly do not have the time to do the human scoring task on a volume basis at the end of a school year. And while teachers value the professional development aspect of scoring, the PD aspect is accomplished mostly during the training part of the scoring process, and the repetitive part for hour after hour for days or weeks on end is not as valued by full-time teachers. Such is a more balanced description of the trade-offs for true teacher scoring vs the more efficient (and less costly) human scoring option offered by testing companies to their customers. Testing companies would be happy to offer true teacher scoring only to states, provided states pay the freight.
Finally, re Gary’s rhetoric against accountability tests, it does reflect the views of many teachers. However, end-of-year tests with strict security provisions are favored by 70-80 % of the general public as a way to hold the public education enterprise accountable for student achievement. That, as Paul Harvey used to say, is the rest of the story.
Gary Ravani 8 years ago8 years ago
Doug: Based on your statement below: "Folks running testing programs like to minimize costs, and hiring highly qualified but not full-time teachers as volume scorers has been an efficient way to minimize costs for human scoring for large scale tests, yet generate reliable results. Full time teachers by contrast have to be paid regular wages, several times more costly on an hourly basis, and mostly do not have the time to do the human scoring task on … Read More
Doug:
Based on your statement below:
“Folks running testing programs like to minimize costs, and hiring highly qualified but not full-time teachers as volume scorers has been an efficient way to minimize costs for human scoring for large scale tests, yet generate reliable results. Full time teachers by contrast have to be paid regular wages, several times more costly on an hourly basis, and mostly do not have the time to do the human scoring task on a volume basis at the end of a school year. And while teachers value the professional development aspect of scoring, the PD aspect is accomplished mostly during the training part of the scoring process, and the repetitive part for hour after hour for days or weeks on end is not as valued by full-time teachers. ”
I don’t believe you have disagreed with me at all here. The tendency to “minimize costs” is exactly what I was talking about. When you do that there is a complementary tendency to reduce quality at the same time. And being “statistically reliable” is something that can be manipulated. The point is, does the test have any “validity and reliability” to the professionals in the schools as being a good indicator of how much students are learning and and how the tests do, or do not, inform instruction. And there, they are sorely lacking.
And the public has been whipsawed for decades now about a manufactured “crisis in the schools” and just how much the test box scores in the newspaper actually mean. They don’t mean much. Mostly the numbers reflect the wealth and education levels of the students’ parents. We have gone through cycles of media generated hysteria about all kinds of things before, and this aimless exercise in “accountability based on test scores” is yet another of them. Based on the manufactured hysteria some parents are not getting their kids vaccinated or sending them to charter schools, not based on charter performance, but based on social cachet or phony claims based on even more phony “accountability.” And contrary to other media assertions, Ebola is not about to bring the “end times” upon us, and an Isis follower is unlikely to be hiding under your bed.
And my recollection is, CA once gave students “Golden Bell” awards based on assessments most teachers I know found valid and meaningful. I don’t recall there being a lot of trouble rounding up sufficient numbers to score the tests. But you are, right they could not be paid cut-rate wages for their efforts.
Don 8 years ago8 years ago
I don't see how the crisis in education in California can be termed "manufactured", Gary, when you consistently and rightly remind us of our last place (or second to last place) standing in per pupil funding. Assessments as overused and/or overrated can be debated, but can anyone credibly deny our disgraceful disinterest in prioritizing education is not in good part related to our underwhelming student achievement? And the idea that students test results … Read More
I don’t see how the crisis in education in California can be termed “manufactured”, Gary, when you consistently and rightly remind us of our last place (or second to last place) standing in per pupil funding. Assessments as overused and/or overrated can be debated, but can anyone credibly deny our disgraceful disinterest in prioritizing education is not in good part related to our underwhelming student achievement? And the idea that students test results only represent the wealth and education of the parents is a slap in the face to all hardworking students and teachers alike who work to defy the your expectations about the predictive power or demographics. If you don’t think students can overcome their socioeconomics, why quibble over the utility of standardized assessments?
el 8 years ago8 years ago
I don't think anyone here is saying that kids in low socioeconomic situations cannot learn, but we are suggesting that the way to improve the academic performance of kids who are socioeconomically challenged may not be more homework and firing every teacher they ever had or will have. I get the appeal, and I certainly think all kids should have great teachers and appropriately rigorous coursework, but I also think that a child who comes … Read More
I don’t think anyone here is saying that kids in low socioeconomic situations cannot learn, but we are suggesting that the way to improve the academic performance of kids who are socioeconomically challenged may not be more homework and firing every teacher they ever had or will have. I get the appeal, and I certainly think all kids should have great teachers and appropriately rigorous coursework, but I also think that a child who comes to school hungry and is worrying all day about the safety of his younger brother and who has no safe, quiet place to read and study is going to struggle no matter who teaches him or what the curriculum is. The answer is not to give up, but to work on addressing those other issues as best we can.
Schools can’t make neighborhoods safe and they can’t stop domestic violence or give families apartments. We ask them to solve some of these issues by having them help families access glasses and medical care and sometimes to the point of providing new clothes. Programs like the free breakfast program in another article today, an afterschool program that provides supervised, comfortable places to play and study, may be more beneficial to those kids than fretting about standards.
And I still wonder how much the ‘miracle of Finland’ is about all working people having 6 weeks of paid vacation and universal health care.
navigio 8 years ago8 years ago
Well, the manufactured portion of the crisis to which Gary was referring is the same one to which John Mockler (RIP) was referring with his term 'California schools suck industry'. This stance is generally based on a claim of low student achievement, which a portion of your comment captures perfectly: "can anyone credibly deny our disgraceful disinterest in prioritizing education is not in good part related to our underwhelming student achievement?" Though you go further than he … Read More
Well, the manufactured portion of the crisis to which Gary was referring is the same one to which John Mockler (RIP) was referring with his term ‘California schools suck industry’. This stance is generally based on a claim of low student achievement, which a portion of your comment captures perfectly:
“can anyone credibly deny our disgraceful disinterest in prioritizing education is not in good part related to our underwhelming student achievement?”
Though you go further than he in claiming that the low funding levels in CA are in fact a result of this perception of low achievement (I even agree with you on that); also implying that the recursive nature of this relationship cannot be broken without someday choosing to increase funding in spite of student achievement levels!
If in fact low achievement levels are a result of low funding levels, then the crisis of low achievement is clearly ‘manufactured’, especially when considering your quote above and comments like those from a former fed sec of education being quoted as saying he didn’t want to help traditional schools succeed, that he wanted to deny them so that they would fail and could then be replaced by charters and private schools.
But Gary’s concern is different and even more specific than this. It is instead about the nature of the tests; what they represent; and what aspect of institutional school quality can be gleaned from them, if any. I.e. Why is the connection made between student achievement and school ‘quality’? Ostensibly, in trying to ‘measure’ schools, we care about the quality of the things the schools can control: teachers, administrators, programs and their availability. So when someone says ‘schools suck’, the implication is it is those things being referenced. The problem with the current tests is that only in rare cases do they represent the ‘quality’ of those things (even the plaintiff’s goofballs in vergara admitted this). Instead, they much more likely represent something else:
http://goo.gl/qWlAZC
The only way the above graph could be possible under the assumption that the tests reflect any significant portion of school quality is if there were an extreme and systemic discrimination in how schools are staffed. While there clearly has been some evidence that this has happened, it was nowhere near an extent that would explain the chasms in that graph (and this should not be interpreted as a call to ignore those injustices when they occur).
This is not intended to be a slap in anybody’s face (except maybe for people who want to claim we can use current test results to ‘fix’ schools by destroying them). Instead it is a request to ‘be consistent’. If we truly want to measure the quality of those things schools control, then we should design a system that does that. The current one (and the coming one likely) does not.
If we don’t then we will be measuring something else (see graph). We should at least admit that.
Personally I think we should focus more on the things we know can improve the school environment (do we really not know this after all this time?). Instead of hoping that austerity-imposed market forces will magically result in a plethora of silver bullets that can then be distributed amongst the masses (as if that’s ever happened).
Don 8 years ago8 years ago
Navigio, not to belabor the frankly tiresome achievement debate, I'm sure we can reasonably agree that significant and well spent SC grants can make a difference in the achievement gap as long as funding is not cannibalized by step and column increases and they actually add to services for target students. That begs the question: if, based on statistical analysis, at least some significant portion of lower performing students reach proficient … Read More
Navigio, not to belabor the frankly tiresome achievement debate, I’m sure we can reasonably agree that significant and well spent SC grants can make a difference in the achievement gap as long as funding is not cannibalized by step and column increases and they actually add to services for target students. That begs the question: if, based on statistical analysis, at least some significant portion of lower performing students reach proficient and above as a result of these grants, why should the State continue to award these successful students extra funding in perpetuity under LCFF based upon their socioeconomic standing which would be been proven under this scenario to be not solely a function of Free and Reduced qualification? And if an insignificant portion attain proficiency will that provide the pause necessary to rethink the wisdom of aligning extra funding to socio-economic status? How ’bout them accountability beans?
navigio 8 years ago8 years ago
Not sure if this comment was intended for a different thread, but.. the quick answer is there is no 'in perpetuity' in LCFF. Such measures are responses to outcomes. If outcomes change, it's not unexpected that the responses would change. Regardless, as even you point out, LCFF s&c grants are not sufficient to adequately fund what low income students really need. Unfortunately, this will mean both a justification for extending LCFF as well as an … Read More
Not sure if this comment was intended for a different thread, but.. the quick answer is there is no ‘in perpetuity’ in LCFF. Such measures are responses to outcomes. If outcomes change, it’s not unexpected that the responses would change. Regardless, as even you point out, LCFF s&c grants are not sufficient to adequately fund what low income students really need. Unfortunately, this will mean both a justification for extending LCFF as well as an argument against it’s effectiveness (not unlike one of the disconnects we have now).
Gary Ravani 8 years ago8 years ago
Doug: And another thing! (LOL) Of course I have a bias. Are you asserting those in the testing industry don't? I would assert that most of us, being flawed human beings, are biased in some ways. Being unable to avoid biases, the next best thing is to know that you have them and try and examine from whence they came. Biases, like words, are our "windows on the world." I think S.I. Hayakawa, fine semanticist-lousy senator, said … Read More
Doug:
And another thing! (LOL)
Of course I have a bias. Are you asserting those in the testing industry don’t? I would assert that most of us, being flawed human beings, are biased in some ways.
Being unable to avoid biases, the next best thing is to know that you have them and try and examine from whence they came. Biases, like words, are our “windows on the world.” I think S.I. Hayakawa, fine semanticist-lousy senator, said something like that.
My “biases” on these subjects are framed by my 35 years in the classroom followed by, a mostly simultaneous, 25 years representing classroom teachers and their issues. This has caused me to poke around the research related to learning and school policy and note how infrequently those two areas seem to overlap. This is in large part because of that old bit of “conventional wisdom” about “the farther you are from the classroom the more impact your opinions have about what goes on in the classroom.” That’s for “official policies,” statutes, regulations, decisions on curriculum, testing, whatever. NCLB likely represent the epitome of that kind of “conventional wisdom” and we’ve all seen how all that worked out.
I have stated before on numerous occasions that I believe testing has its place, a limited place. That “place” has not been limited as of late and abuses have occurred. That needs to stop, and that is my bias.
TheMorrigan 8 years ago8 years ago
We all have a rest of the story, Doug. "end-of-year tests with strict security provision are favored by 70-80% of the general public as a way to hold the public education enterprise accountable for student achievement": While arguments of popularity and appeals to spite--with a sprinkling of the reification fallacy for the ignorant--may easily sway politicians and the irrational, they truly display a lack of serious lack of thought on the subject and reduce your … Read More
We all have a rest of the story, Doug.
“end-of-year tests with strict security provision are favored by 70-80% of the general public as a way to hold the public education enterprise accountable for student achievement”:
While arguments of popularity and appeals to spite–with a sprinkling of the reification fallacy for the ignorant–may easily sway politicians and the irrational, they truly display a lack of serious lack of thought on the subject and reduce your own rhetoric patina to a tarnished dullness.
Don 8 years ago8 years ago
Doug has always provided excellent information on the subject of assessment using fact-based analysis. He has gone out of his way to clarify the pros and cons of assessments as tools for better public education and has repeatedly highlighted the fact that quality assessment results are only as good as properly developed tests, proper implementation and administration of them, as well as sufficient instruction and all the resources that entails for proper alignment of … Read More
Doug has always provided excellent information on the subject of assessment using fact-based analysis. He has gone out of his way to clarify the pros and cons of assessments as tools for better public education and has repeatedly highlighted the fact that quality assessment results are only as good as properly developed tests, proper implementation and administration of them, as well as sufficient instruction and all the resources that entails for proper alignment of teaching and testing. He has been an outspoken critic of the inadequacies of SBAC and thank goodness for test experts like himself who are willing to speak out publicly to educate on the proper use and overuse of assessment. No doubt, many of the stances he’s taken have put him at odds with colleagues and I applaud his candor.
Then there are those with an agenda to whip up public dissent against what they consider to be the scourge of standardized tests. We find them often at work portraying the assessments and the industry as racist and anti-union. Politicians have used student assessments as a tool to evaluate teachers, a function for which they were not developed, and, as a result, unions have thrown down the gauntlet to denigrate the testing industry by attacking the validity of SBAC. It was Doug who spoke out on the proper utility of assessments.
Doug McRae 8 years ago8 years ago
Don — Thank you for your kind words . . . .
CarolineSF 8 years ago8 years ago
Reposting a comment from a few days ago as it's more relevant here than in the original spot. Critics are raising questions about whether having low-wage temps scoring the tests is a sound practice. This is from Diane Ravitch’s blog. Just a few days ago, someone tweeted this ad he found on Craig’sList in Indianapolis: “Test Evaluators Needed (to score K-12 standardized tests) (Indianapolis, IN) … Read More
Reposting a comment from a few days ago as it’s more relevant here than in the original spot.
Critics are raising questions about whether having low-wage temps scoring the tests is a sound practice.
This is from Diane Ravitch’s blog.
Just a few days ago, someone tweeted this ad he found on Craig’sList in Indianapolis:
“Test Evaluators Needed (to score K-12 standardized tests) (Indianapolis, IN)
“Compensation: $11.05/hr contract job
“If you have a Bachelor’s degree or higher, we need your help to evaluate student assessment tests. Come apply!
“For more information and to schedule an appointment visit our website http://www.kellyservices.us/ctb or call us at 1-877-535-5981
“Please be prepared to spend two hours going to through the application / orientation process. Please bring two forms of identification to complete and I-9 and bring proof of your degree.
“These are project based positions. Monday – Friday, 8:30am – 4:30pm
“Position Requirements:
-Must hold a completed Bachelor’s degree or higher
-Ability to sit and at a computer station for full work day
-Basic computer knowledge
-Knowledge of standard writing conventions and mechanics
-Availability to work Monday through Friday for the entire duration of a project
-Demonstrate flexibility while working on various projects”
Gary Ravani 8 years ago8 years ago
Glad you reposted that, Caroline. One of the hiring criteria: Knowledge of standard writing conventions and mechanics Note that they don't say "Knowledge of English language writing conventions and mechanics." Yes, some of the "horror stories" I talked about are of testing "subcontractors" (managing the test scoring boiler-rooms), not being too particular about the language skills of the scorers. Not everyone who has a BA received it in this country. Of course, controlling costs is primary objective. Some … Read More
Glad you reposted that, Caroline.
One of the hiring criteria: Knowledge of standard writing conventions and mechanics
Note that they don’t say “Knowledge of English language writing conventions and mechanics.”
Yes, some of the “horror stories” I talked about are of testing “subcontractors” (managing the test scoring boiler-rooms), not being too particular about the language skills of the scorers. Not everyone who has a BA received it in this country. Of course, controlling costs is primary objective. Some of us think it’s all about education while some are interested in sound business practices. But again, my biases are showing.
Doug McRae 8 years ago8 years ago
Gary -- The budgets set for human scoring projects involving large scale tests are set not by the testing companies, but rather by their customers. The testing companies are simply executing what the purchaser specifies, including the compensation level for scorers and the "quality" of the service requested. It isn't the testing companies business practices that are the issue. If a customer specifies they want only in-state full-time teachers to execute human scoring for a … Read More
Gary — The budgets set for human scoring projects involving large scale tests are set not by the testing companies, but rather by their customers. The testing companies are simply executing what the purchaser specifies, including the compensation level for scorers and the “quality” of the service requested. It isn’t the testing companies business practices that are the issue. If a customer specifies they want only in-state full-time teachers to execute human scoring for a statewide testing contract, then that is what the company that wins that contract will do — but, the state will have to pay the extra dollars for that specification. Very few states make that kind of specification, and those with familiarity with statewide testing contracts (both state employees and vendor folks) know that the quality of the scores produced are not significantly different . . . . but, that statement is not intended to reflect the quality of the experience for the hired scorers, or any professional development benefits that may accrue for the scorers. Fact is, some folks enjoy repetitive work and do well with it, others do not enjoy it and do poorly. The objective for the exercise for both the state and the vendor is achieving valid, reliable, fair scores, and that is the “bottom line” that both the state purchasing the service and the company supplying the service have.
Gary Ravani 8 years ago8 years ago
I don't disagree, Doug. The RFP does define the amounts that can be spent. It is then up to company to make that amount work best for them based on whatever "principles" the companies use to base those decisions on. As the criteria for hiring scorers posted by Caroline, indicate the qualifications for those scorers are "iffy." Which is another way of saying, the company is making it work for them. And again the establishment of … Read More
I don’t disagree, Doug. The RFP does define the amounts that can be spent. It is then up to company to make that amount work best for them based on whatever “principles” the companies use to base those decisions on. As the criteria for hiring scorers posted by Caroline, indicate the qualifications for those scorers are “iffy.” Which is another way of saying, the company is making it work for them.
And again the establishment of “validity and reliability” can be done quite creatively. The problems that led to lawsuits filed by the state of CA during the rollout of the last set of tests indicates that “validity and reliability” do not come from on high carved on stone tablets. Major errors by testing companies resulting in kids being retained in grade or not graduating across a number of states is no great endorsement of the whole process.
And, that some people are better suited for the “boiler room” environment and stunning receptiveness of scoring is certainly true. I wouldn’t want to do it. The same perspective can be used to suggest the whole scoring process leaves a lot to be desired.
And the final test for the tests is, are they useful educationally? Or, as the late and great John Mockler said, are they just “statistical porn?”
Mike McMahon 8 years ago8 years ago
Here is a rebuttal to this line of thinking from an editorial: The Trouble with the Common Core http://www.rethinkingschools.org/archive/27_04/edit274.shtml We have seen this show before. The entire country just finished a decade-long experiment in standards-based, test-driven school reform called No Child Left Behind. NCLB required states to adopt “rigorous” curriculum standards and test students annually to gauge progress towards reaching them. Under threat of losing federal funds, all 50 states adopted or revised their standards and … Read More
Here is a rebuttal to this line of thinking from an editorial: The Trouble with the Common Core http://www.rethinkingschools.org/archive/27_04/edit274.shtml
We have seen this show before. The entire country just finished a decade-long experiment in standards-based, test-driven school reform called No Child Left Behind. NCLB required states to adopt “rigorous” curriculum standards and test students annually to gauge progress towards reaching them. Under threat of losing federal funds, all 50 states adopted or revised their standards and began testing every student, every year in every grade from 3–8 and again in high school. (Before NCLB, only 19 states tested all kids every year, after NCLB all 50 did.)
By any measure, NCLB was a dismal failure in both raising academic performance and narrowing gaps in opportunity and outcomes. But by very publicly measuring the test results against benchmarks no real schools have ever met, NCLB did succeed in creating a narrative of failure that shaped a decade of attempts to “fix” schools while blaming those who work in them. By the time the first decade of NCLB was over, more than half the schools in the nation were on the lists of “failing schools” and the rest were poised to follow.
In reality, NCLB’s test scores reflected the inequality that exists all around our schools. The disaggregated scores put the spotlight on longstanding gaps in outcomes and opportunity among student subgroups. But NCLB used these gaps to label schools as failures without providing the resources or support needed to eliminate them.
The tests showed that millions of students were not meeting existing standards. Yet the conclusion drawn by sponsors of the Common Core was that the solution was “more challenging” ones. This conclusion is simply wrong. NCLB proved that the test and punish approach to education reform doesn’t work, not that we need a new, tougher version of it. Instead of targeting the inequalities of race, class, and educational opportunity reflected in the test scores, the Common Core project threatens to reproduce the narrative of public school failure that has led to a decade of bad policy in the name of reform.
The engine for this potential disaster, as it was for NCLB, will be the tests, in this case the “next generation” Common Core tests being developed by two federally funded, multi-state consortia at a cost of hundreds of millions of dollars. Although reasonable people, including many thoughtful educators we respect, have found things of value in the Common Core standards, there is no credible defense to be made of the high-stakes uses planned for these new tests.
The same heavy-handed, top-down policies that forced adoption of the standards require use of the Common Core tests to evaluate educators. This inaccurate and unreliable practice will distort the assessments before they’re even in place and make Common Core implementation part of the assault on the teaching profession instead of a renewal of it. The costs of the tests, which have multiple pieces throughout the year plus the computer platforms needed to administer and score them, will be enormous and will come at the expense of more important things. The plunging scores will be used as an excuse to close more public schools and open more privatized charters and voucher schools, especially in poor communities of color. If, as proposed, the Common Core’s “college and career ready” performance level becomes the standard for high school graduation, it will push more kids out of high school than it will prepare for college.
This is not just cynical speculation. It is a reasonable projection based on the history of the NCLB decade, the dismantling of public education in the nation’s urban centers, and the appalling growth of the inequality and concentrated poverty that remains the central problem in public education.
Replies
FloydThursby1941 8 years ago8 years ago
Is it poverty or the average American kid studying 5.6 hours a week vs. over 40 hours of screen time? Obama said “you’re never so poor you can turn on the TV at night but can’t do homework with your kid or teach your kid to read at a young age.” How do Asians do well while in poverty?
el 8 years ago8 years ago
So the answers we’re apparently prescribing to that problem are:
1. Fire the teachers because the kids don’t do enough worthwhile stuff outside the classroom.
2. Give kids more screen time inside the classroom.
FloydThursby1941 8 years ago8 years ago
El, when you dismiss my point as "fire the teachers" that is disingenuous. There would be the same number of teachers working. My problem is with a nearly ironclad guarantee of lifetime employment and what that does to the profession and children. If you guarantee tenure for life and LIFO and pay by seniority and only lay off in extreme circumstances, you get what we have now. Teachers are often unresponsive … Read More
El, when you dismiss my point as “fire the teachers” that is disingenuous. There would be the same number of teachers working. My problem is with a nearly ironclad guarantee of lifetime employment and what that does to the profession and children. If you guarantee tenure for life and LIFO and pay by seniority and only lay off in extreme circumstances, you get what we have now. Teachers are often unresponsive to parents. Teachers often don’t worry about taking a day off they don’t need. Now most teachers are good, but even good ones don’t strive as hard with this sort of a guarantee. Uncertainty drives people. LIFO undercuts innovation. In a system of LIFO, you don’t do the same things as in a system of merit where you can say, look at that great teacher who is 29, what are they doing? There’s no pressure to be the best, or do your best, no fear, etc. Lemons drift from school to school when they should be replaced by people who work harder.
Teaching should be like other professions. If you do great you can make more money fast, if you do terrible, you do something else.
Kids need to be #1. They aren’t now.
The net # of teachers would be the same. For every one fired, one would be hired. They’d just all work harder and be more incentivized.
el 8 years ago8 years ago
I think it’s worth mentioning that scores will go down for reasons not related to student understanding of the academic material – for example, just operating the computers and having to type answers are going to be challenging for some kids, which will add time and some stress to the process.
Certainly these computer skills can and will be taught… but I’ll note we haven’t removed any time from the curriculum nor added any minutes to do so.
Don 8 years ago8 years ago
"... Smarter Balanced assessments – will be administered as part of a new, more comprehensive state accountability system to measure student progress toward college and career readiness." Can someone please explain how some of the proposed and more comprehensive factors for a new API such as attendance rates, suspension rates, expulsion rates, dropout rates, school climate factors and other measures of schools are indicators of student progress towards college and career readiness? A school's general progress … Read More
“… Smarter Balanced assessments – will be administered as part of a new, more comprehensive state accountability system to measure student progress toward college and career readiness.”
Can someone please explain how some of the proposed and more comprehensive factors for a new API such as attendance rates, suspension rates, expulsion rates, dropout rates, school climate factors and other measures of schools are indicators of student progress towards college and career readiness? A school’s general progress on several telltale indicators that spell failure is worth tracking, but I fail to see how this gives us more than a vague measure of college and career readiness. Smarter Balanced assessments may one day provide an accurate measure of student content knowledge, however if those assessments are mixed with the other factors this will only blur the picture as it relates to a school’s overall readiness. What the graduating 12th grade class needs to know to move on to college or career is not measured by how many students dropped out.
Replies
Paul Muench 8 years ago8 years ago
My understanding from one of John's previous articles is that multiple measures means multiple indicators as well. The multiple measures will not be combined into a single indicator. So if one wants to look at only test scores that will still be possible. If one wants a broader view of a school that will be possible too. I think both approaches are possible today if you're willing to dig deep enough … Read More
My understanding from one of John’s previous articles is that multiple measures means multiple indicators as well. The multiple measures will not be combined into a single indicator. So if one wants to look at only test scores that will still be possible. If one wants a broader view of a school that will be possible too. I think both approaches are possible today if you’re willing to dig deep enough into the data on the web. So I think the major change will be how the data is presented, which so far seems likely to be a dashboard approach.
FloydThursby1941 8 years ago8 years ago
Can we see the pure testing score and the politically correct score with all these weird factors separately, so you can at least look up the top academic scores and the public can choose which number to consider highest?