This blog post about our NewsTrust Baltimore local news experiment was written by NewsTrust Baltimore writer Andrew Hazlett and originally published on our Baltimore blog on June 20, 2011. It is republished here for the benefit of our national community.
In March 2011, Maryland Governor Martin O'Malley stated on his website that Maryland has "America's number one public school system." This claim, which was frequently used by O'Malley and other state leaders in campaign speeches, has been questioned by several experts and commentators, who cite evidence that Maryland public schools are behind other states on a number of key measurements.
Given this controversy, we invited the Baltimore community to join our first local Truthsquad and fact-check this statement on the NewsTrust Baltimore site in June 2011. To find out if O'Malley's claim was true, NewsTrust staff and members posted and discussed a wide range of education statistics and competing arguments over school policies and student achievement.
After reviewing the evidence and views of our community on this statement, NewsTrust Editors find O'Malley's claim to be "HALF-TRUE."
Here are some of the key facts and insights that led us to this finding.
We're No. 1! … Or are we?
O'Malley's statement that Maryland has "America's number one public school system" is based in part on research by the newspaper Education Week, which has given Maryland a No. 1 ranking for the past three years. These favorable findings were frequently cited by other state officials, such as Maryland schools superintendent Nancy Grasmick, who has been widely praised for helping improve the state's public schools over the past 20 years.
Maryland schools also received a No. 1 ranking from the College Board for the state with the highest percentage of graduates who have been successful on Advanced Placement tests. And last year, "Maryland was one of only a dozen states to be awarded a $250 million competitive federal grant, known as "Race to the Top," as noted in this Baltimore Sun story.
By these measures, the state's schools would appear to be among the nation's best. But when those reports were released and when O'Malley made this No. 1 claim, several experts and commentators questioned the findings. In recent years, Maryland public school graduates have been deemed unprepared for college. State-by-state comparisons of performance on standardized tests place several states ahead of Maryland in student achievement. And most observers we spoke to agree that Baltimore City's public schools face serious challenges, which seem to contradict this claim.
Evaluating O'Malley's claim
Given this mixed evidence, we asked NewsTrust staff and community members to review and post links to relevant news stories and factual evidence related to this claim. Participants were then invited to weigh in with their reactions on a special Truthsquad page, from Monday, June 6 to Sunday, June 19, 2011.
Over the past two weeks, we collected a dozen links to determine the accuracy of Gov. O'Malley's statement. We found a wide range of news stories and opinions, think-tank reports, government statistics, and other evidence and commentary supporting or opposing this claim. Members of the NewsTrust community weighed in with their views and observations, as well. Of 42 respondents at the time this post was written, a plurality of 17 voters found the statement false, but there was no majority view. Fourteen voted that the statement is "true" and nine were "not sure."
Let's review the evidence we gathered and discussed during our collective quest.
The primary basis for O'Malley's claim appears to be this report produced by Education Week. Their annual "Quality Counts" report is based on test scores, spending figures, and aggregated statistics from 50 distinct indicators. The data are grouped into broad categories: chance of success, K-12 achievement, school finance, and transitions and alignment. This report has given Maryland the highest grade for three years running and, as education reporters for The Sun have pointed out, "Gov. Martin O'Malley and other state leaders mentioned [the Education Week rankings] frequently in campaign speeches."
The Education Week analysis of states is about the best system that we have for ranking the states. The long-term consistency of Dr. Nancy Grasmick has provided Maryland with the opportunity to enact reforms and see them through, particularly on such issues as student assessments and holding schools (and systems) accountable for achievement and teacher performance. Setting clear, consistent standards is really one of those things that makes the state stand out.
Still, Grasmick acknowledged some ambiguities in the survey results. At a Washington, D.C., event after the report's release (video available here), Grasmick welcomed the positive attention to Maryland schools, but she shared that her team had "drilled down" into all of the report's 50 indicators and found that "we're not consistently strong in all of those indicators."
There are also important questions about the criteria and findings of the Quality Counts report. In her comments on our Truthsquad page, Sun columnist Marta Mossburg questioned the formula used by Education Week: "A number one ranking should reflect student knowledge, not money spent and other inputs."
Though the Education Week report does reflect attempts to measure student knowledge, it certainly gives considerable weight to "inputs" that may not translate into positive outcomes for students.
At the time the report was released, a story in the The Baltimore Sun noted some skeptical voices, including from a representative for the Thomas B. Fordham Institute. The Sun article pointed out that "Maryland falls behind in the gap between the achievement of its poor children and those who aren't poor, ranking 37th in the country in the National Assessment of Education Progress."
In their observations during this Truthsquad, a number of NewsTrust Baltimore community members echoed those concerns.
Community member Khalilah Harris, who heads the Baltimore Freedom Academy, a Maryland charter school, wrote:
The state of Baltimore City schools, Prince George's County schools, schools in NW Baltimore County, etc. are evidence of this claim being questionable. I need evidence of a measure that pairs MD scores with states who use similar tests, rates family engagement, student success post high school, massive reduction of achievement gap on a school by school basis, and a large percentage of its most decrepit school buildings in good condition to agree with this. Being #1 with two of the wealthiest counties in the country is so very easy. Further, there is little to no evidence of efforts or reform that reduce poverty at a rate that would impact the necessary family investment in eliminating low expectations for children.
In reviewing the Education Week study, community manager Gin Ferrara, a former Baltimore media educator, said:
While Maryland ranks 4th in the Achievement Index, the 3rd measurement in that category, achievement gaps between rich and poor, is much lower than the other top states. Maryland's 15.9 ranks us around 37th for providing opportunities for all our citizens.
Several other commenters raised the issue of statewide figures masking inequality and inconsistent results in local districts. To support his finding that O'Malley's statement was "false," community member Chip Molter wrote:
Providing effective education to young people inside Baltimore City is not an easy issue by any means. It is intertwined with so many other issues facing the city and its inhabitants. However, as long as Baltimore City and Anne Arundel school systems occupy a second tier status within the state, it is difficult for the residents of those school districts to cheer along with the Governor as he congratulates himself for the fortune of the rest of the state's educational success.
And NewsTrust Baltimore member Christopher Siple wrote on the Truthsquad page:
A lot of the rural schools and Baltimore City (another school system that rocketed to #1 under O'Malley's leadership if you were to ask him) aren't doing so hot, while Howard and Montgomery Counties are some of the richest in the union, so it isn't terribly shocking that these schools tend to be of a higher quality. Maryland is so heterogeneous in its quality of life and education that it's only political sophistry to claim its #1 status in the entire United States.
Indeed, Census Bureau figures make clear that Maryland is one of the wealthiest states in the nation. But these statewide figures can hide the fact that very large islands of persistent poverty co-exist with the state's wealthier districts.
In addition to the Education Week rankings, several other measures would also place Maryland at the top of the education pyramid. The Maryland State Department of Education has celebrated students' high scores on Advanced Placement tests in news releases like this. Though some have doubted the push to enroll students in AP courses and Maryland's efforts specifically, many students in the state seem to be finding success through these opportunities.
On the other hand, there is some evidence that many Maryland students are graduating from high school without the tools they need to succeed in college. In a Baltimore Sun opinion column about Nancy Grasmick's legacy, Marta Mossburg points to remedial education statistics and anecdotes to suggest that there is a "swelling tide of students who graduate from state public high schools without basic reading or math skills."
Last year, a think tank advocating for a more rigorous education compared Maryland's standards with those of other states and the Common Core standards that have recently been adopted in many states. The Thomas B. Fordham Institute gave relatively low marks to Maryland's standards in English language arts (grade "C") and math (grade "D") and said:
The Maryland [English and Language Arts] standards are a mixed bag. Standards are generally well organized, and many are clear and specific. Others, however, fail to clarify expectations or omit essential content that students should master as part of a rigorous, K–12 curriculum. ... Maryland's [math] standards are poorly organized and difficult to interpret without additional explanation, which is only occasionally provided.
In June 2010, Maryland adopted the Common Core standards. Still, the Education Week ranking would have been based on the pre-existing standards that were critiqued by the Fordham study.
Another source to consider in evaluating the governor's claim are data collected by the U.S. Department of Education. According to information we reviewed, Maryland students have done well on reading and math assessments, but they have not scored higher than students in several other states.
The Condition of Education 2011 report is a comprehensive study worth exploring in depth. For our purposes, we can look at some recent statistics on achievement by eighth-graders. The percentage of students scoring at or above "proficient" in reading was 36 percent in Maryland, but 43 percent in Connecticut and Massachusetts. Minnesota, Montana, New Hampshire, New Jersey, Pennsylvania and Vermont also had a higher percentage of proficient readers than Maryland.
The results in math were similar. Forty percent of eighth-graders in Maryland were rated at or above proficient in math, but other states had a greater proportion, notably Massachusetts (52 percent).
Another Department of Education report, Indicators of School Crime and Safety, had some disturbing figures about Maryland schools. In 2007–2008, 8.4 percent of Maryland public school teachers reported that they were physically attacked by a student from school during the previous 12 months. This was the highest percentage of any state in the nation.
Though the Education Department's vast stores of information should be read and used with care, we found enough data to suggest a more complex reality than O'Malley's claim would indicate.
NewsTrust contributing editor Kristin Gorski, an educator herself, reflected the ambivalence of many commenters:
What makes a school system #1? Governor O'Malley's claim got me thinking. While the Ed Week report indeed gives Maryland public school system its top score, a B+, followers Massachusetts and New York received Bs (all other systems received Cs and below). All school systems have individual districts and schools that are sorely in need of improvement – focusing on broad statements like who is "the best" compared to "the worst" doesn't inform.
NewsTrust member Bob Herrschaft also questioned the basis for making claims like these:
Political rhetoric can't readily be verified when using terms like "number one" (i.e. best). How do we define "number one"? Even two Phds in education are likely to have a completely different definition. Do we go by the number of graduates that go onto higher educational institutions, some of which are sham factories of propaganda, or do we look at the approach to the individual student's capacity to enhance his or her aptitude for learning?
NewsTrust Baltimore editors acknowledge that Gov. O'Malley's claim can be confirmed by credible independent sources based on certain measurements, but we also found enough reliable evidence to contradict that statement based on equally important measurements, leading to our finding that this claim is only half-true.
More important, we question the value of making such sweeping statements. Dubious claims and overstatements are an inevitable part of our political background noise, but there are costs. A governor should proudly share good news about the state's schools, and many who have celebrated Grasmick's tenure as state schools superintendent are fully cognizant of the major challenges still faced by Baltimore City schools in particular. But a proclamation that Maryland's schools are already "number one" can seem dispiriting to those who are engaged in an uphill struggle to bring educational opportunity to all young people in the state.
This short investigation of Maryland's school rankings has been a rewarding experience for our team. This was our first local fact-checking experiment, and we are bolstered by its results. This short comment from the governor's office gave us an opportunity to delve into complex issues around educational achievement and accountability as a community. We enjoyed this opportunity to learn from each other, through shared links and thoughtful observations from Truthsquad contributors.
We'd like to thank all the participants in this Truthsquad. Together, we explored a complex subject that is a major concern of Maryland's citizens. Your contributions helped expose some pressing issues and open up a valuable discussion, and we hope it will continue. We invite you to post your comments about our findings on our Truthsquad page -- or email us at editors-at-newstrust-dot-net.
Special thanks to Craig Newmark, NewsTrust advisor and founder of Craigslist.org, for promoting this local Truthsquad in his blog post, which stated: "It's up to us to do the factchecking that we see little of, in TV or newspapers." We wholeheartedly agree, and we appreciate Craig's support of pro-am initiatives like ours.
For more information about separating fact from fiction, check out our "Crap Detection 101" guide by Howard Rheingold, as well as his video version; the Factcheck.org and Politifact websites; and the book "Blur," by Bill Kovach and Tom Rosenstiel.
More about Truthsquad
To learn more about the Truthsquad initiative, visit our project overview page. Truthsquad aims to strengthen the field of fact-checking by combining the best practices of crowdsourcing and social media with the expertise and knowledge of experienced journalists. This new experiment empowers citizens and journalists to collaboratively fact-check controversial claims from politicians, newsmakers and members of the media. Participants are invited to post questionable claims online, research factual evidence supporting or opposing these claims, and verify their accuracy as a community, with professional oversight.
NewsTrust created and tested Truthsquad in 2010, with funding from Omidyar Network and with the help of partners at the Poynter Institute, as well as advisors such as Brooks Jackson, director of FactCheck.org. The first pilots were well-received by online participants, partners and advisors, as well as by third-party observers, such as GigaOm. To learn more, read our pilot reports on PBS MediaShift, as well as on the national NewsTrust blog. NewsTrust has since hosted a variety of Truthsquads with other partners, including MediaBugs.org and RegretTheError.com, and with advice from Craigslist founder Craig Newmark.
NewsTrust has now formed a strategic partnership with the Center for Public Integrity to develop a daily service on Truthsquad.com, which we hope to launch in fall 2011. The goal is to create a one-stop destination for fact-checked information -- featuring its own findings, as well as promoting the work of other trusted research organizations, such as FactCheck.org and PolitiFact.
Stay tuned for more announcements about this initiative in coming weeks. To get our free newsletters, we invite you to sign up as a NewsTrust member, if you haven't already. This will also enable you to participate in more Truthsquads like this one.
-- By Andrew Hazlett, on behalf of the NewsTrust editors