This blog post about our NewsTrust Baltimore local news experiment was written by NewsTrust Baltimore community manager Gin Ferrara and originally published on our Baltimore blog on June 29, 2011. It is republished here for the benefit of our national community.
In April and May 2011, NewsTrust Baltimore staff invited members, partners and supporters to take a short online survey about our service. We asked them to share their perspectives on the usefulness and impact of our site and to make suggestions for improvements and new features. Here are our findings about this survey.
Overview
We collected both quantitative and qualitative data in this brief survey, using tools from the website Survey Monkey. Individuals were asked a combination of multiple-choice and short-answer questions. There were 12 questions total, which could be completed in about 5 to 10 minutes, on average. Questions addressed user satisfaction with the site, frequency of use, areas of strength and weakness, and potential new features. Links to the survey were distributed to all members of NewsTrust Baltimore via email, and emails were also sent to community partners. Public links to the survey were posted on social media, our blog and on the homepage of the site.
The online survey took place from April 22 through May 6, 2011. In total, 192 people began the survey, and 135 completed it. A smaller, normalized sample of 87 respondents was used for analysis purposes, to feature more diverse responses; this community sample was intended to be more representative of our community, with fewer college student responses. It also excluded NewsTrust staff responses, as well as duplicate or incomplete responses.
The survey gave respondents the option to select a user group that best represented them: 50 respondents self-identified as college students, seven were educators, seven were journalists, 14 were unaffiliated members, and the remaining nine were visitors. This breakdown is similar to NewsTrust Baltimore’s overall statistics, with one exception: We experienced a very high response rate from college students, many of whom were encouraged by their teachers and likely driven by NewsTrust incentives, such as our student certification and awards, to complete the survey. The percentage of student members on the NewsTrust Baltimore site is about 40 percent of total members (versus up to 60 percent of total survey respondents). We used this sample for much of the analysis in this report, as well as for the charts.
Key findings
The majority of respondents (about 60 percent) found NewsTrust Baltimore to be personally useful or very useful. Many survey participants thought the project was unique and a valuable complement to existing news sources, as well as a way to identify trustworthy sources for local news. Respondents also told us that they were introduced to several new media outlets via NewsTrust Baltimore, and they said they felt comfortable commenting in what they considered to be a respectful online community.
Based on the pilot stats we collected, we know that NewsTrust Baltimore had many unique visitors (11,215 visitors in its first three months), but fewer people signed up as members (514) and reviewed stories (329). Most survey respondents (85 percent) said they visited the site weekly, and roughly half (52 percent) reviewed stories at least once a week.
When asked which features of NewsTrust Baltimore were most interesting to them, 59 percent of survey respondents said they were interested in finding good local journalism all in one place and 57 percent were interested in discovering local news sources that they hadn’t heard about.
“I like that I pay attention to a wider variety of new stories and outlets because of NewsTrust,” member Kate Bladow wrote in a survey response.
Diana Soliwon, the former editor of Owing Mills Patch, commented that the local site was “very helpful for someone trying to figure out where to get their information in the greater Baltimore area.”
Responses to a question about possible new services shows that respondents’ interests were divided somewhat evenly among activities for college students (49 percent), a suggestion box for new story ideas (47 percent), and a field guide for local news sources (44 percent).
Areas of improvement recommended by many respondents included making the site simpler to use, adding more diverse and entertaining stories, and increasing community dialogue opportunities.
Findings by group
NewsTrust Baltimore serves many groups of people with different backgrounds, interests and approaches to the site. We have filtered the survey findings into groups to examine their different perspectives on NewsTrust Baltimore.
College students
College students were one of our most active groups of members. Of the students in the community sample, 50 percent of them said they visited the site more than once a week, and 48 percent visited the site weekly or monthly. Students were also our largest pool of reviewers, with 30 percent of respondents saying they reviewed stories every few days and 62 percent reviewing stories weekly or monthly.
Students told us that the program was beneficial to their education and future goals. Many of our student participants were studying journalism and made the connection between NewsTrust’s services and their own careers. Fifty-four percent of respondents liked discovering news sources they hadn’t heard about, and 52 percent liked rating the work of other journalists.
Devin Hamberger said in the survey, “I think it is a great way for students to expose themselves to good journalism that not only helps them be critical consumers, but also helps their own writing skills.”
Micah Mohlmann was one of several students who felt that the site improved their own critical thinking: “I have learned how to better analyze and reviews news stories. It has helped me to critique articles in a professional manner.”
Rebecca Jackson wrote, “As a journalism student, looking at the work of local journalists helped me understand the things I need to look for in my stories.”
While students enjoyed learning more about the big issues of Baltimore, they also expressed interest in seeing more multimedia stories and more news that they felt was relevant to their lives.
“I think that having more news sources for young people would make me visit more often,” wrote Megan Flannery in a survey response.
Other students commented that they would like to read and review more stories about sports, entertainment, the arts, health, and beauty. In addition, 78 percent of college students said they would like to see activities specifically for college students on the site.
Kara Duffy suggested, “I would like to see News Trust have some focus on other colleges. It would be cool to have a group where journalism majors in Maryland colleges could post their articles and have other students grade them.”
Educators
Our teachers, professors and youth workers used the site primarily as an educational tool. JoAnne Broadwater, a Towson University professor, wrote in a survey comment: “I like it for its usefulness in the classroom. I think that it will help students to be more critical of what they are reading. I also like the concept of requiring them to read news and then evaluate it. Since many students do not read news stories at all, they have difficulty writing news stories and grasping the concept of a carefully constructed story. I think NewsTrust will help them to become better writers.”
While the site saw strong adoption by college students and their professors, some respondents saw a need for more focused attention on high school students. Susan Malone, executive director of Wide Angle Youth Media, wrote, “I would have liked to see more intention to create a youth-centered site where young people can start to digest news in bite-size pieces, that utilizes anonymity so young people can feel more inclined to participate.”
The convenience of aggregating local news was appealing to this group of respondents, with 86 percent of educators reporting that they like being able to find local news all in one place. They were also introduced to new local media organizations, with 71 percent reporting that they discovered new news sources via NewsTrust Baltimore.
Journalists
For journalists, NewsTrust Baltimore presented an opportunity to engage with their audience in a new way. A large percentage (86 percent) of journalists who took the survey said they visited the site more than once a week, and 46 percent visited daily, though the majority (71 percent) of journalist respondents said they rarely or never reviewed stories.
Stephanie Hughes, a producer at WYPR-FM, found the site to be valuable to her programming: “I like getting direct feedback on the segments I'm working on. NewsTrust responses are especially valuable because I know people are encountering the segments via the web, as opposed to just on air. WYPR is figuring out how to create great content for both on air and online, and it's interesting to see how reactions from online consumers differ -- it helps us to figure out what we can do to enhance the web experience. “
Howard Libit of Center Maryland, a NewsTrust Baltimore media partner and former Baltimore Sun editor, said in a survey response: “It has been interesting to see and read other people's perspectives on the different journalism taking place in the market. I am also learning about some individuals and groups involved in journalism that I was not previously aware of.”
For some journalists, the site offered new opportunities and new questions. City Paper writer and editor Bret McCabe wrote, “I'm just curious as to how best to interact with the feedback generated by this site, because if people are going to the effort of commenting thoughtfully about what they read, it should have some utility in the practice.”
Journalists were one of the more critical groups of respondents regarding the website’s usability. They described it as “cluttered” and asked for “a better job of displaying stories,” as well as “a more attractive site.” These comments were representative of the suggestions for improvements from this group.
Members
This “members group” includes members who had signed up on the website and who were not included in the other categories, such as partners and students. Their participation spanned the spectrum, with many members visiting the site weekly (43 percent) and half of all members reviewing once a week (29 percent) to once a month (21 percent).
These members said they liked the convenience of finding good journalism in one place (79 percent), and several mentioned their appreciation of the respectful environment on the site.
Gabby Knighton commented, “I like that there is a ’sane’ community of news readers out there. You don't see them as often in the ‘comments’ sections” of other news sites.
Debra Joseph wrote, “I like the transparency, the focus on smart journalism critiques, and the mutual respect among members.”
This group of members also had suggestions for improvements. Some asked us to increase the number of stories and the frequency of refreshing our pages with new articles. We also found that, while some people were critical that we sent too many emails, others wished we sent more. This suggests that a user’s email preferences and settings could be made more clear, so users could easily adjust their communication with NewsTrust Baltimore to their comfort level.
Visitors
We were fortunate to receive feedback from visitors, partners, community leaders, friends and supporters of the site, people who didn’t consider themselves members but who cared enough about our outcomes to share their thoughts in this survey.
These respondents said they did not review stories often but visited the site frequently (77 percent visited the site at least weekly). Their feedback was thoughtful and specific.
Carl Ehrhardt wrote about the challenges of adding another social network site to his regular use: “Perhaps if NewsTrust were an app for Facebook it would be easier.”
John Walters saw a challenge in the volume of participation on the site and felt that some of the tools, “like the discussion features, might be useful if there were more users.” Others also expressed this concern about the number of reviewers on the site.
Feedback by activity
In analyzing the survey results, we also looked at how people responded based on their activity on the site. People who said they reviewed stories more than once a week are defined as Active Members, those who reviewed stories weekly or monthly are Basic Members, and people who reviewed rarely or never are considered Visitors for this assessment.
Our Active Members group found the review tools to be a valuable service, with 71 percent of them reviewing stories every few days and 29 percent reviewing stories once a day or more.
Olivia Stephens wrote, “I think it’s a great tool for people to evaluate the news critically and really understand how to find reliable, credible news.”
Active Members also appreciated that NewsTrust Baltimore let them keep up with local news (54 percent) and find everything on one site (50 percent). Lauren Calva commented, “There really isn’t another website, that I know of, that collects local journalism and puts it all in one place.”
The majority (64 percent) of our Basic Members group reviewed stories once a week. They, too, like finding a variety of local news on a single website, but 60 percent also reported that they enjoyed discovering news sources they hadn’t heard about. Basic Members had the greatest number of suggestions for new content and topics.
Our Visitors group did not review stories but visited the site with some frequency, with 49 percent visiting more than once a week. They said they came mainly for the convenience of aggregated local news and are interested in reading a field guide for local news sources.
“The most significant feature for me is NT’s ability to be a trustworthy aggregator of local news,” wrote Michael Catalini, a journalist.
Conclusions
Overall in our collected survey responses, we found that NewsTrust Baltimore was valued as an aggregator of local news, introducing people to new sources and serving as a one-stop daily news site. Members appreciated the rational critique process and the sense of respect for commenters.
NewsTrust Baltimore was also found very useful as an educational tool, helping students build critical media skills, separate fact from fiction, and work on their own writing.
We also learned that there is room for improvement, through streamlining the site, increasing the frequency and diversity of stories posted, and creating more community participation opportunities. There is also a desire for more education resources, both activities for college students and learning tools that are appropriate for high school students.
All of us at NewsTrust and NewsTrust Baltimore appreciate the time and thought that respondents took in answering the survey, and we hope to continue to work together to build a robust, inclusive and relevant news community.
I am just not sure the best way to go about it. I really like your survey questions. All but about 3 are questions I would like to use. It does help in a very definitive way to have such survey results circulated. They provide a fine insight into the workings that take place. Thank you for sharing!!
Posted by: John Wodden | August 27, 2012 at 11:32 PM