Are you a public school struggling to make "Adequate Yearly Progress" in the No Child Left Behind era? Consider a move to Michigan, where the bar is lower.

That's not exactly how the Thomas B. Fordham Institute puts it in a report comparing school accountability systems nationwide, but it's close.

"The Accountability Illusion" says that it's easier to "make AYP" in Michigan than in most of the 27 other states it scrutinized. It's relatively harder in South Carolina and Massachusetts, but probably easier in Wisconsin and Arizona.

Why? Because even though NCLB's intent was to make sure that every third- through eighth-grader reaches proficiency in reading and math by the year 2014, the law leaves considerable detail up to individual states. "All children" is not defined the same way in each state, nor is "proficient." Plus, while every state has 2014 as an end goal, they determine individually how quickly their schools must get there.

The result is a different bar in every state, to the point that "making AYP" tells a parent more about state guidelines than about schools, the authors argue.

"We found demonstrably good schools that failed AYP far too often, and some pretty mediocre schools that slid by in some states," the report said.

The report calls NCLB at once too lenient and too strict — too lenient in letting states set their own rules, but too strict in setting unrealistic achievement goals, particularly for children with disabilities and those just learning English. This flaw both allows and encourages states to cut corners in order to avoid sanctions, the authors write.

NCLB has been up for reauthorization since 2007, and Chester E. Finn and Michael J. Petrilli, Fordham president and vice president, respectively, have long suggested that the country switch to a national assessment test, the same in every state. The results should be posted publicly "for every school in the country," they say, but states should have final authority over whether their schools measure up and what to do about laggards.

Coincidentally, two Michigan legislators have suggested essentially the same thing, but for different reasons, and so has the National Governors Association.

STATE-TO-STATE COMPARISONS

"The Accountability Illusion" asked whether a school that makes AYP in one state would make it in another state, and why or why not.

Analysts at the Kingsbury Center at Oregon's Northwest Evaluation Association did the study by selecting 36 elementary and middle schools from across the country and figuratively "moving" each school to 28 different states. The study used 2005 standardized test scores from each school, held against 2008 AYP guidelines in each state.

Results showed that schools that had no trouble meeting NCLB guidelines in their own state failed to meet the bar in other states, and vice versa.

In Michigan, 10 of the 18 elementary schools and four of the 18 middle schools would have shown adequate yearly progress. (Every school would have made AYP in Michigan except for subgroup performance, discussed below.) By comparison, only one of the elementary schools and none of the middle schools would have met Massachusetts or Nevada standards, while 17 elementaries and seven middle schools would have passed muster in Wisconsin.

One reason for Michigan's results is the relatively low "cut scores" it adopts on Michigan Educational Assessment Program tests, the report said. A cut score is the lowest score a student can attain and still be considered proficient. A second reason is that Michigan counts students as proficient if they don't actually reach the cut score, but fall within a range intended to account for measurement error.

The researchers made "cut score" comparisons by holding each state test against a common yardstick — the Measures of Academic Progress assessment developed by NWEA. Millions of children nationwide, including many in Michigan, have taken the MAP over the years, giving the NWEA the means to compare one child's performance against his or her peers across the country.

The comparison showed that a "passing" score on Michigan's third-grade standardized math test is equivalent to scoring at the 6th percentile on the MAP. By comparison, a "passing" score on the Massachusetts exam would be equivalent to scoring at the 68th percentile on the MAP exam. (That doesn't necessarily mean the Massachusetts test itself is harder, but that Massachusetts requires its students to score at a higher level before calling them "proficient.")

If the Measures of Academic Progress test were given to all third-graders, how well would students have to score in order to be considered “proficient” in their home state? In Michigan, students would be considered proficient in math if they scored at the 6th percentile compared to their peers nationwide. In Massachusetts, they would have to score at the 68th percentile on the MAP before being considered proficient. The Fordham Institute arrived at those numbers by correlating each state’s 2005-2006 “cut scores,” (the lowest score a student can earn and still be considered proficient), to the nationally-normed MAP assessment. Their findings are listed in more detail at http://edexcellence.net/accountability_illusion/AccountabilityIllusion_DataMap.html

The Michigan-to-Massachusetts comparison is an extreme example, but differences of 30 percentile points from state to state and subject to subject are not uncommon.

"Same kids, same academic performance, same schools — different states, different cut scores, different rules. And very different results," the report said.

CLOSING THE EDUCATION GAP

Cut scores aren't the only differences among states, according to "The Accountability Illusion." Closing the education gap between disadvantaged and other students — one of the key goals of NCLB — has become one of its murky areas.

In theory, the law requires schools to show that students with disabilities, low-income students, minority students and English-language learners are making as much progress as the student body at large.

In practice, that means that if a state requires 65 percent of the children in an elementary school to be proficient in math, then 65 percent of that school's English-language learners must be proficient as well, and 65 percent of African-American students, and so on. If one subgroup fails, the whole school fails.

The pressure to bring subgroups up to par, particularly English-language learners and students with disabilities, creates an incentive for states to tweak the system. One way they do that is by defining how many students constitute a "subgroup," the report says.

Michigan takes a relatively stern approach in this area by putting the minimum subgroup number at 30 students in most cases, the report says.

That means that a school that enrolls 30 or more students with disabilities must show that those students meet the same proficiency targets as the student body at large. A school with 25 students with disabilities does not have to meet that target, though those student scores would be included in district-wide tallies.

In North Dakota, individual schools have to track groups as small as 10 students; in California, the number is 50. If the subgroup number is too low, it probably reflects unfairly on schools that are doing a good job overall, the report said. But too high a number allows schools to hide their lack of progress with needy students.

Acknowledging this problem, President Barack Obama has said the country needs to develop better ways to test students with disabilities and English-language learners.

A BETTER IDEA?

Convoluted and uninformative as NCLB's "adequate yearly progress" has become, the authors say that abandoning it is politically unlikely.

Still, legislators could improve the law by encouraging different states to use the same assessment test, the authors write. Using a common test would give parents some much-needed transparency on how well their child is doing academically compared to other students nationwide.

Coincidentally, two Michigan legislators have proposed a similar idea, but for a different reason. So has the National Governors Association.

State Sen. Mickey Switalski, D-Roseville, and Sen. Ron Jelinek, R-Three Oaks, have suggested that Michigan begin using the same "off-the-shelf" standardized test that some other states do. Both members of the Senate K-12 Appropriations Committee, they estimated that Michigan could save $10 million by buying a test rather than writing its own, as is current practice.

The National Governors Association also favors national academic standards, essentially saying that the United States should decide as a country what it wants children to know and be able to do, particularly in comparison with other countries.

Critics argue, though, that even if all parties could agree on what to include in a national curriculum, such a move would give the federal government too large a role in educational matters. It also continues what some see on too large an emphasis on standardized tests in the nation's classrooms.

The Fordham Institute also suggests using growth models to assess schools rather than proficiency targets. Growth models measure how much students have learned in a given year, based on pre- and post-year testing, and give schools credit for how many students meet individual growth targets, rather than for how many students meet a standard proficiency level.

Michigan already uses a growth model in certain scenarios, giving schools credit for progress among students who are below the proficiency target, but whose test scores show they could catch up within three years.

#####

Lorie Shane is the managing editor of the Michigan Education Report, the Mackinac Center’s education policy journal. Permission to reprint in whole or in part is hereby granted, provided that Michigan Education Report is properly cited.