NewsTrust 2010 Education Report

In 2010, the NewsTrust team developed a range of new educational tools to help students tell apart fact from fiction in the news. Here are some highlights of this year's educational programs, in partnership with ten different universities and high schools.

 

Border News - A student-powered news site
This fall, we hosted our most ambitious educational pilot yet: Border News was our first nationwide, interscholastic collaboration at the college level. Convened by professors Sally Lehrman and Venise Wagner, this project brought together 235 students from five colleges to find good (and bad) journalism about immigration. 

The participating colleges and their faculty hosts were:

• Santa Clara University [hosted by Sally Lehrman]
• San Francisco State University [hosted by Venise Wagner and Katynka Martinez]
• University of Alabama [hosted by George Daniels]
• University of Nevada, Reno [hosted by Donica Mensing]
• Columbia College [hosted by Teresa Puente]

We invited all students to find good journalism on immigration, using our new group curation tools. Their student-powered news site, Border News, featured their most trusted stories throughout this experiment. For thirteen weeks, they collaborated with each other and with the NewsTrust community in a special Immigration News Hunt, from September 13th to December 1st, 2010. Together, we searched for the best news coverage of immigration and generated 1,386 reviews of 340 stories on this topic, of which 1,086 reviews (78%) came from the students, with coaching from their professors. (see photos from our SFSU kick-off meeting with Venise Wagner's class)

On Border News, the students curated an informative feed of immigration stories on NewsTrust. We also experimented with other social tools for curating and sharing the news. The professors asked their students to publicize their activity on Twitter under the hashtag "#bordernews." This enabled an open conversation about immigration issues on social media, with NewsTrust stories as the background. It also enabled these journalism students to practice their Twitter skills. Catchy headlines, smart keyword use, and short-linking are essential skills for journalists today, and Border News students were able to practice them, thanks to deep Twitter integration on NewsTrust.

Border News was the first public pilot to use our new NewsTrust Groups application. On the Border News group page, you can see a current listing of all the stories posted, reviewed, and starred by the students and faculty members, as well as their comments. The stories on this page display a special blue Trust-O-Meter showing the Group Rating, meaning the average NewsTrust rating given by members of the group, rather than the community at large. Underneath the Group Rating, the site-wide rating is shown in green numerals for comparison. This allows you to see how the ratings from Border News students compare to those of the NewsTrust community in general. (See also our full listing of the students' most trusted stories)

The journalism professors who partnered with us on this project thought Border News was a success. SCU professor Sally Lehrman, who came up with the idea for this experiment, shared these reflections:

"I was delighted when San Francisco State and the other universities agreed to jump on board with SCU. Now NewsTrust gives students the opportunity to interact thoughtfully with immigration news in real time and with others in their age group around the country... I've noticed greater awareness of the perspectives embedded in the news and the code words that help shape these."

Journalism professor Donica Mensing at UNR, a long-time NewsTrust friend and partner, felt that Border News "was a great success. The students were far more engaged by focusing primarily on one topic than in previous years when I left the topic open."

Most encouraging of all were the responses from the students themselves. Here are just a few of them:

"I really like how [NewsTrust] is very organized, especially when it comes to search for a news story. By being a member, I get the opportunity to read the most recent top stories and add my own opinion (it help me a lot when it came to find articles on immigration) and rate it. It would be awesome to see a similar version of NewsTrust in spanish, but overall great site. Keep up the wonderful work!"
- Marianna Ballesteros, SFSU

"I like the user-interface and the scoring system of rank and merit on the site. I like the idea that if I contribute enough to the site, I will watch my rating increase. I like that the stories themselves are being reviewed (if only at times) by fellow journalists, which make for a system in self-regulation. Essentially, I like that journalists are patrolling their own, ensuring that the ideals of objectivity and peer-review are upheld."
- Christian Bertolaccini, UNR

"The ability to view ratings, reviews, and other people's opinions on a story... helped me to decide if the story was trustworthy. I think this collaborative gathering of news, to put together a source of "good journalism" is a great idea and very useful tool."
- Isabella Sleister, UNR

"I joined NewsTrust for a journalism class that I am in. I look forward to contributing; I wouldn't have heard of this site if it weren't for this class and I think that it's a great idea and I'm glad to be a part of it."
- Che Bajandas, SFSU

 

Reinventing the News at Northeastern
NewsTrust can help teach news literacy, civics and journalism in a variety of ways, and more schools are now starting to use our tools on their own, without requiring our help. This semester, while we were facilitating the Border News project, long-time NewsTrust advisor Dan Kennedy's "Reinventing The News" course at Northeastern University and the JRN 215 class at Eastern Michigan University both used NewsTrust in their classrooms.

Check out Dan Kennedy's round-up blog post, in which he links to some great comments and criticisms of NewsTrust from his students. Earlier this year, Dan invited another class of students to review stories on NewsTrust and write about their experience. See our earlier blog post. Northeastern student Hannah Martin offered this eloquent description of what she learned using NewsTrust:

"What I liked about the reviewing experience was it forced me to really analyze my news on its journalistic value, which, as bad as it sounds, is often something that slips my mind." ... "I browse the headlines of nyt.com, read what looks important, and accept it as fact, rarely stopping to count sources or assess context. The process of reviewing though, forced me to think through all the elements of each piece, and consider what, as a journalist, should ultimately be there."
- Hannah Martin, Northeastern

 

Studying Journalism at EMU
At the same time, other journalism projects like the News Writing and Reporting class at Eastern Michigan University have started using NewsTrust on their own, to help students develop their news literacy skills. For a student perspective, be sure to read Kaylee Lentz's wonderful blog post entitled "What I've learned from NewsTrust." Here's a highlight:

"I feel so much more aware of my fellow inhabitants just from browsing [NewsTrust]’s news section. So, after having such an eye-opening experience, I’ll probably keep this website close to me and reference it often."
- Kaylee Lentz, EMU

 

Our First High School Pilots
In April 2010, we hosted our first high school pilots with the International High School in San Francisco and the Science Leadership Academy in Philadelphia, with funding from the Ayrshire Foundation and Omidyar Network. This spring, 120 high school students participated in our Global Economy News Hunt with Global Voices and Link TV (see photos from IHS). This pilot demonstrated that NewsTrust can be effectively extended for secondary education. According to their teachers, students became more engaged as citizens and more savvy about the news; they also learned a great deal about the global economy and surrounding civic issues. Read more about this high school pilot in our full report (PDF).

Here are some comments from high school teachers who used NewsTrust in their classrooms:

"Students were quite inspired. Many had the opportunity to really think about the conventions and current state of journalism for the first time. The instrument itself as a piece of IT was impressive and represents a positive aspect of how journalism is evolving in this digital age."
- Andrew Brown, International High School of San Francisco

"The relevance of the content, interactive interface and socially networked learning all impressed me about the NewsTrust framework for classroom use. Most of the 'liked least' issues were addressed throughout the spring with adjustments and enhancements to the service."
- Diana Laufenberg, Science Leadership Academy of Philadelphia

 

Bad Journalism News Hunt
In March 2010, we ran a one-week Bad Journalism News Hunt with journalism students from Stanford University, under the guidance of professor and NewsTrust board member Howard Rheingold. This project produced some excellent work, as well as some valid and thoughtful criticisms of NewsTrust and the process of crowd-sourced news curation. Stanford student Susana Montes-Delgado shared these observations:

"NewsTrust, apart from helping readers discern between good and crappy information, is a great tool to test your journalism literacy. By using a review form, you evaluate stories based on how factual, balanced and contextual they are. But most importantly, the system allows you to be more critical about the information you consume on the Web."
- Susana Montes-Delgado, Stanford

See our report to learn more; there were many valuable insights from this News Hunt.

This was our second hunt for bad journalism with college students; our first, with Santa Clara University in Fall 2009 (see blog post), paved the way for this year's Border News collaboration.

 

Feedback from Educators
This summer, we surveyed our partners and advisors to get their feedback about these educational applications of NewsTrust.

Here are our key observations from this survey:

• most teachers we surveyed found NewsTrust useful for their school (90%)
• over three-quarters were planning to use NewsTrust again next year (77%)
• teachers told us their students became more engaged in the news as a result

Teachers pointed to these key benefits when asked what their students could learn best from NewsTrust:

• to think critically about what they read (90%)
• to recognize good journalism (63%)
• to recognize bad journalism (58%)

Going forward, they were planning to use NewsTrust for these applications:

• for a single class activity (as outlined in our teacher guides) (65%)
• as an ongoing research tool (for learning about any topic you like) (53%)
• for a weeklong News Hunt (or news comparison) (47%)

They also thought that NewsTrust would be most useful for media studies, civics / government, social studies and journalism classes.

Because this particular survey was limited to educators who were already familiar with our service, their feedback was well informed, and their insights were invaluable for planning our next steps in education.

 

 

Personal Observations
Prior to joining NewsTrust, I coordinated civic engagement programs in a mid-sized public school district, and NewsTrust's potential as a civics and media literacy teaching tool was one of the key features that drew me in to this job. To complement our report above, I would like to share my personal observations about NewsTrust's potential as an educational tool.

Our spring 2010 educational pilots were some of my first NewsTrust projects as a contributing editor for NewsTrust. After taking the reins as managing editor this fall, I had the privilege of conducting the Border News project and experimenting with NewsTrust's versatile set of tools for teaching news literacy and media studies.

Based on my experience this year, it appears that we've built some useful tools for students and educators alike. NewsTrust review tools and story listings already enabled deeper engagement with the news, and now, with the new Groups application, I think NewsTrust is coming of age as a resource for students.

NewsTrust Groups provide an ideal platform for any number of ways classrooms could engage with the news. A group can be a student-curated news site, allowing students to run their own virtual newsroom. It can also be an online news discussion section moderated by an instructor. We can create groups for entire schools, or for individual sections or classes, or even for small-group work within a class. It provides teachers the ability to evaluate their students' work, and it allows students to respond to each other. For schools concerned about confidentiality, we even have a variety of privacy options, so that NewsTrust Groups can be visible to the web, to NewsTrust members only, or restricted to members of the group.

For an ideal example of what students and teachers can do with NewsTrust, look no further than Border News.

We'd like to thank the students who worked with us this year for doing an amazing job, and their instructors for bringing these unprecedented and innovative projects into their classrooms. We learned a lot in the process and hope to develop a full set of educational tools over time, to help the next generation of citizens become more savvy about they hear in the news.

If you're an educator interested in using NewsTrust with your students, be sure to read our teacher guides, student guides, activities and other educational resources on our site. If you have any questions, please send us a message at schools-at-newstrust-dot-net. We would love to work with you to help you use NewsTrust in your classroom.

We look forward to hosting more educational activities as part of our upcoming NewsTrust Baltimore pilot -- and we hope all these experiments will lead to some exciting new projects in the future.

-- by Jon Mitchell, Managing Editor

Support our work
If you think our educational work is useful, please consider a donation, so we can keep providing services like these to the public in 2011. NewsTrust is not a commercial site, it is free community service dedicated to helping people find quality news and information online. Read more about our nonprofit organization, which is devoted to promoting good journalism, news literacy and civic engagement.

Thank you!

NewsTrust Baltimore - A Local News Experiment

NewsTrust Baltimore is an online experiment to improve the way people get their local news, now under development at NewsTrust, with the generous support of the Open Society Foundations and OSI Baltimore.

Our goal for this news literacy project is to help Baltimore residents get better access to good journalism about their community -- and become more discerning news consumers in the process (especially university and high school students).

To that end, we will organize a two-month pilot in Baltimore due to launch next February 2011, based on a local version of the NewsTrust curation platform. Our special website will provide an online "story bank" where Baltimore residents can find good journalism about their area -- and share the best reporting with friends and neighbors.

During this pilot, a small team of professional editors will lead an online community of citizens, teachers, students, journalists and local experts to evaluate and promote the best local news coverage in their area. Participants will use the NewsTrust review tools to rate news stories based on journalistic qualities such as facts, fairness and context -- and learn to tell apart good journalism from misinformation.

Collectively, we will recommend news and opinions from a wide range of sources: mainstream and independent, commercial and public, national and hyper-local, ethnic or niche blogs -- in print, broadcast or online media. We will also track each source's expertise by topic.

For this project, we will partner with several local universities and high schools -- as well as some of Baltimore's newspapers, radio and TV stations, bloggers, ethnic and nonprofit organizations. Our local news feeds will be featured on their websites as well, using our widgets.

This news hub and civic engagement network will give citizens a practical way to share good journalism on important issues such as politics, crime, education, justice, poverty, race and youth development. This will help raise public awareness about these issues, and identify informative news sources for each topic we cover.

In the process, we hope to make citizens more aware of their local news ecosystem, help them discover new sources of credible information, and start a discussion on how to improve the way Baltimore residents get their news.

We will post more information on our project page in coming weeks.

 

Support our work

NewsTrust is a nonprofit organization dedicated to helping people find quality news and information online. Read more about our initiative on our About page. If you are able, please consider a donation today, so we can keep providing more services like these to the public. Thank you!

Crowdsourced Fact-Checking: What we learned from Truthsquad

[This entry was originally posted on the PBS MediaShift blog]

In June, Senator U.S. Senator Orrin Hatch made the statement that "87 million Americans will be forced out of their coverage" by President Obama's health care plan.

It was quite a claim. But was it true?

That's a common, and important, question -- and it can often be hard to quickly nail down the real facts in the information-overloaded world we live in. Professional fact-checking organizations like PolitiFact and FactCheck.org have taken up the charge to verify the claims of politicians, pundits and newsmakers, and they provide a great service to the public. But I believe there's also a role for the average person in the fact-checking process. By actively researching and verifying what we hear in the news, we can become more informed citizens, and more discriminating news consumers. These are essential skills for all of us to develop.

With that in mind, we at NewsTrust, a non-profit social news network, have been working on Truthsquad, a community fact-checking service that helps people verify the news online, with professional guidance.

Our first pilot for Truthsquad took place in August 2010, with the help of our partners at the Poynter Institute, our advisors at FactCheck.org and our funders at Omidyar Network. That pilot was well received by our community, partners and advisors, as noted in our first report, and by third-party observers such as GigaOm. We've since hosted a variety of weekly Truthsquads, and are starting a second pilot with MediaBugs.org and RegretTheError.com to identify and correct errors in the news media. (Disclosure: MediaShift managing editor Craig Silverman runs RegretTheError.com.)

Our first test project was by our standards a success; more importantly, it revealed several important lessons about the best ways to manage crowdsourced fact-checking, and about why people participate in this activity. Here are our key takeaways from this first pilot, which I'll elaborate on below:

• A game-like experience makes fact-checking more engaging.
• A professional-amateur (pro-am) collaboration delivers reliable results and a civil conversation.
• Crowd contributions are limited, requiring editorial oversight and better rewards.
• Community fact-checking fills a gap between traditional journalism and social media.

What is Truthsquad?

Truthsquad.com features controversial quotes from politicians or pundits and asks the community whether they think they are true or false. Community members are welcome to make a first guess, then check our answers and research links to see if they are correct. They can change their answer anytime, as they come across new facts.

To help participants find the right answer, we invite them to review and/or post links to factual evidence that supports or opposes each statement. A professional journalist leads each "truthsquad" to guide participants in this interactive quest. This "squad leader" then writes an in-depth verdict based on our collaborative research. That verdict is emailed to all participants, with request for comments. (It can be revised as needed.)

Finding #1: Game-Like Experience Makes Fact-Checking Engaging

We noted a significant increase in community participation for Truthsquad compared to other NewsTrust services we tested this year. Some data from our first pilot:

• This pilot generated twice as much participation as other 2010 pilots.
• Users gave ten times more answers per quote than reviews per story on our site.
• Over half of the participants read linked stories, and a third answered a Truthsquad quote.
• One in six participants reviewed the stories linked as factual evidence.

We think this high level of engagement is partly due to the game-like quality of our user experience, which starts by inviting people to guess whether a statement is true or false -- an easy task that anyone can do in under a minute.

After their first guess, people are more likely to participate as researchers, because their curiosity has been piqued and they want to know the answer. As a result, participants often take the time to review linked stories and post more evidence on their own. Without realizing it, they are becoming fact-checkers.

Finding #2: Pro-Am Collaboration Delivers Reliable Results

We decided early on that professionals needed to guide this collaborative investigation. We wanted to avoid some of the pitfalls of pure crowdsourcing initiatives, which can turn into mob scenes -- particularly around politically charged issues. At the start of this experiment, we asked experienced journalists at FactCheck.org and the Poynter Institute to coach us and our community and help write and edit some of our first verdicts.

We think the pro-am approach paid off in a number of ways:

• Amateurs learned valuable fact-checking skills by interacting with professionals.
• A few community members posted links that were critical to reaching our verdicts.
• Answers from our community generally matched final verdicts from our editors.
• We came to the same conclusions as FactCheck.org in side-by-side blind tests.
• Comments from participants were generally civil and focused on facts.

The results of our first pilot led our advisor Brooks Jackson, director at FactCheck.org, to comment, "So far I would say the experiment is off to a solid start. The verdicts of the Truthsquad editors seem to me to be reasonable and based on good research."

This collaboration between journalists and citizens made us all more productive. The professionals shared helpful information-gathering tips, and the citizens extended that expertise on a larger scale, with multiple checks and balances between our community and our editors. Our editors spearheaded this investigation, but the community made important contributions through comments and links to factual evidence (some of which were invaluable). On a couple occasions, we even revised our verdicts based on new evidence from our community. This focus on facts also helped set the tone for our conversations, which were generally civil and informative.

Finding #3: Crowd Contributions Are Limited, Requiring Better Rewards

Despite high levels of participation, we didn't get as many useful links and reviews from our community as we had hoped. Our editorial team did much of the hard work to research factual evidence. (Two-thirds of story reviews and most links were posted by our staff.) Each quote represented up to two days of work from our editors, from start to finish. So this project turned out to be more labor-intensive than we thought, and a daily fact-checking service will require a dedicated editorial team to guarantee reliable results.

Managing our community and responding thoughtfully to their posts also takes additional time, and is an important part of this process. In future releases, we would like to provide more coaching and educational services, as well as better rewards for our contributors.

Training citizens to separate fact from spin is perhaps the greatest benefit of our initiative, but keeping them engaged will require ingenuity and tender loving care on our part.

"It seems based on this pilot that citizens can learn fact-checking skills quite easily," said Kelly McBride of the Poynter Institute. "The challenge is to motivate them to do this occasionally."

To address this issue, future versions of Truthsquad could reward members who take the time to fact-check the news in order to get them to do it more often. We would like to give them extra points for reading, reviewing or posting stories, as well as special badges, redeemable credits and/or prizes. We can also feature high scores on leaderboards, and give monthly awards to the most deserving contributors.

Finding #4: Community Fact-Checking Fills a Need

Every survey we have done in recent years has consistently shown fact-checking as a top priority for our community, and this was confirmed by the results of this pilot.

Here are some key observations from our recent survey about NewsTrust's 2010 pilots:

• A majority of survey respondents (61 percent) found Truthsquad useful or very useful.
• Half of survey respondents wanted a new quote every day -- or several quotes a day.
• Half of survey respondents said they could fact-check quotes several times per week.
• One in seven survey respondents were prepared to donate for this service.

We think the generally favorable response to Truthsquad is due to two factors: a growing demand for fact-checking services, combined with a desire to contribute to this civic process. Fact-checking is still the best way to verify the accuracy of what people hear in the news, and it is perceived as an effective remedy to expose politicians or pundits who propagate misinformation.

At the same time, the explosion of social media makes people more likely to participate in investigations like these. They want this civil watchdog network, and expect to have a voice in it.

Next steps

Based on the lessons from this experiment, we would like to offer Truthsquad on an ongoing basis, with a goal to fact-check one quote a day, year-round -- as well as to feature the work of other trusted research organizations on Truthsquad.com.

We also want to let members post their own quotes for fact-checking and reward them for their contributions through both a game-like interface and more educational benefits. We have an opportunity to track the expertise of participants based on their answers, which could allow us to measure their progress with core news literacy skills, as well as their overall understanding of important public issues and the overall impact of our service.

Over time, we hope to provide more training and certification services, to build lasting research skills that could help students and adults alike play a more active role in our growing information economy. If this appeals to you, we invite you to sign up here and join our experiment.

As for that Orrin Hatch quote? In the end, 163 participants helped us fact-check his statement about health care. Our final verdict was that the Senator's claim was false. That finding was based on factual evidence provided by one of our NewsTrust members, who dug up the right set of regulations, and pointed out they had been misstated by Hatch. Our editor's verdict was confirmed by similar findings from FactCheck.org, and also matched our community's consensus: 138 participants answered that this statement was false (versus 11 who thought it was true).

More importantly, we as a community learned how to separate fact from fiction -- and became more engaged as citizens.

-- by Fabrice Florin