“The very first thing I saw this morning was via Twitter,” says Vos, associate professor and chair of journalism studies. “All of these news cameras are in this clerk’s office, because now it’s going to happen. I was thinking of this as partly a media story, partly as a social justice story.”

The story, about a Kentucky clerk who defied the recent Supreme Court ruling and refused to issue marriage licenses to gay couples, would become a lead news item for several days. As it broke, readers, viewers and users of mobile devices joined in the fray, weighing in with reactions to what they read or saw.

After the story hit The New York Times, it generated 2,000 comments by early afternoon. Vos’s first impulse was to click on them directly, but he told himself, let’s not go there.

“But that’s sort of the temptation, because I’m wondering how people contribute to news,” he says. “What do they take away from it? What do they find useful about it? When I look at the comments, it’s partly just this entertainment value but it’s also professionally interesting as well.”

Of particular interest to Vos and his colleagues is what commenters have to say about the practice of journalism itself. The idea that online comments might serve as press criticism led to the recent study, Reader Comments as Press Criticism: Implications for the Journalistic Field, which was published this fall in Journalism: Theory, Practice and Criticism.

The study looks into what commenters have to say about journalistic performance and whether or not these remarks jibe with traditional journalistic standards. Vos, along with Stephanie Craft, an associate professor at the University of Illinois, and David Wolfgang, an MU doctoral candidate, spent two years examining the comments posted on the ombudsman columns of three media outlets: The New York Times, The Washington Post and National Public Radio (NPR).

What they found surprised them. Nestled among the cynicism, snark and incendiary remarks were more than a few pearls. Commenters took the news organizations to task for objectivity, accuracy, fact checking and balance, values long held within the industry. They even used terms like “false equivalence,” journalism-speak for giving merit to dubious sources in an effort to appear balanced.

Vos says that bias was the most common form of criticism and that audiences seem to be very good at pointing it out – even when it doesn’t really exist. But this could also be said of traditional criticism coming from within the world of journalism, he says.

“I think we were a little surprised about how willing commenters were to hold journalists to traditional journalistic standards,” says Wolfgang. “It challenges the idea that journalists think the audience is a bad influence on their work, that letting the audience in on your work could be a problem.”

The study’s commenters also held the three news organization's reporters accountable for creating social norms, calling them out for lack of transparency as well as for a number of personal shortcomings. These social criteria were new and unexpected.

“That was the big take-away for us in the study,” say Vos. “Whereas the old model of press criticism was very geared to the roles, norms, ethics of journalism that had been developed over many decades, this new kind of press criticism that comes from commenters evokes a broader range of social norms and social expectations.”

One of the most common of these was the call for greater transparency. The idea that journalists need to disclose more about how they do their work hasn’t been a journalistic norm in the past. But as a commenter on the Washington Post column put it: “Regardless of whether a writer is an ‘objective’ reporter or opinion columnist, their connections to those who might influence them – or need the writer’s influence – must be disclosed.”

Vos says that he and Craft found calls for transparency in some of their other research, too. “It’s definitely catching on to where it’s entering into the public discourse -- that’s true across all institutions,” he says. “We don’t see how the sausage is made, and hence we don’t trust it.”

Other social expectations were more personal. The commenters took journalists themselves to task, using terms such as lazy, judgmental, rude and sanctimonious. Craft, who was the study’s lead author, says that in some cases, commenters seemed to be scolding the journalists for poor etiquette.

“To have things that are part of social life as criteria for judging journalism is pretty interesting. It’s fun to speculate about those things.” Craft says that she and Vos wondered whether commenters’ new ability to interact directly with journalists has created a familiarity that didn’t exist in the past. They also wondered about the shift in journalism away from being a specialized activity.

“We’re in an environment where journalists are facing a lot of competition for tasks that used to be things that only journalists did,” says Craft. “If everyone can do journalism now, of course journalists should be behaving by the rules of everyday social behavior.”

News organizations themselves might be contributing to these social expectations, too. Many now cultivate a personality, or brand identity, which sets them up to be treated like a person, says Vos.


Of course some personal criticism crosses the line. But for the purposes of the study, the researchers were looking for big themes to emerge. Comments meant to stir the pot or hit upon a contentious issue got diluted among hundreds of thousands of remarks. This vast collection encompassed all responses to the Times, Post and NPR ombudsman columns between January 2011 and December 2012.

“We had a whole research team: three of us,” says Vos in his characteristically wry manner. Fortunately, undergraduate researchers and Discovery Fellows (members of MU’s Honors College) helped with the task of collecting reader comments and highlighting any evaluative language involving the practice of journalism. Vos, Craft and Wolfgang read the highlighted statements, noting themes and consistent remarks before meeting to discuss them. After meeting, the three researchers read the statements a second time before coming to conclusions.

Wolfgang, whose dissertation involves commenters and commenting culture, considers the research an exploratory study because no one had really looked at this kind of media criticism before. He says that the beauty of using comments to ombudsmen (also known as public editors) is that they serve as critics within the organization.

“Somebody had a great comment that if the comments are full of trolls, it’s your fault,” says Joy Mayer, associate professor and director of community outreach at The Columbia Missourian. “It’s the responsibility of journalists to be involved in the comments.”

“Our idea was that this is where a discussion about journalism practices and standards would take place,” says Craft. “That’s what the ombudsman is all about.” During the study, there were active commenting communities on the ombudsmen sites of the Times, NPR and, to a lesser degree, the Washington Post, she says. These news organizations also happened to be some of the few institutions that still had ombudsman. Many, including most recently the Post, no longer do.

Like the ombudsmen sites themselves, the people who read and comment on them are a rarified breed. Although it’s difficult to study commenters individually, Wolfgang says they’re typically outliers with very strong feelings about certain issues. There’s some early research into the psychology of heavy commenters known as “trolls,” who bait people into arguments to fight or make fun of them. But most commenters are harmless and simply want to express themselves.

“Somebody had a great comment that if the comments are full of trolls, it’s your fault,” says Joy Mayer, associate professor and director of community outreach at The Columbia Missourian. “It’s the responsibility of journalists to be involved in the comments.” Good comments foster better, more useful comments, but many journalists don’t have or take the time to read them.

Mayer, whose job with the Missourian includes soliciting and taking into account feedback of many kinds, says she thinks of commenters as collaborators. Although most readers, viewers and users of news media choose not to get involved, those who post questions and comments are doing news organizations a favor.

“Their questions make us better, and their questions make us more complete” she says. “Even the ones just giving us feedback are giving us a window into the perceptions of our work, which we want, right?”

Mayer illustrates the value of online comments in a story that The Missourian ran in early September. It was about a shooting at a local VFW post, and the reporter had quoted a veteran who suggested that if more vets had been present, he doubted the shooter would have left unharmed. The vet went on to say, “all veterans are excellent marksmen here,” implying that had they been present, the vets would have taken the shooter on.

A regular commenter responded: “That assumes one or more veterans was carrying. Does the VFW allow its members to carry weapons in the building? It wouldn’t surprise me if they didn’t.” The reporter followed up and responded a few hours later. As it turned out, neither veterans nor members of the general public are allowed to bring guns to the VFW.

“Many people might have had that question when reading the story,” says Mayer. “But someone who feels comfortable putting themselves out there and likes the idea of challenging journalists asked a really completely valid question.”

If the reporter had asked this question in the first place, he probably wouldn’t have included the anecdote. But reporters are human, and in the world of interactive media, the bigger mistake would have been to ignore the comments all together.

As witnessed by positions like Mayer’s, news organizations have begun to value audience engagement. But Vos says that it’s also important that they listen intently to what audiences say about their journalism. Whether they choose to accept or reject what they have to say, journalists can no longer treat their audiences with contempt or indifference. Unlike the insular critics of the past, commenters come from outside the journalistic field. And that might not be such a bad thing.

“We saw the ombudsman post being eliminated during the course of the study, but I think that’s what makes this study even more interesting,” says Vos. “The audience is there to be the source of press criticism. They’re a partner in this. It’s more grass roots – and in that sense more democratic, too.”

Giving the audience opportunities to talk back to journalists reinforces the concept that truth will emerge from a “marketplace of ideas,” a fundamental principle behind the First Amendment and press freedom.


As news becomes more of a conversation, another implication of the study is that journalists, once the all-powerful gatekeepers of the news, now share that role with their audience. But in sharing that authority, to whom will they listen?

“All of this fits into the sort of research that journalism scholars are reading these days,” quips Vos, holding up his book, Gatekeeping in Transition, which he co-edited with François Heinderyckx. The book was published this year and addresses the forces that come to bear on organizations that construct the news.

The commenting study, which shows quite clearly that the audience is one of those forces, is part of a broader research program that both Vos and Craft have taken on. Another of its studies examines bloggers. In addition, Vos and Craft are primary investigators in the Worlds of Journalism Study, an international comparative project looking at changes in journalism. Each brings to the research a specific focus, which for Vos involves factors shaping journalism as a social institution, and for Craft includes ethics and press criticism.

“One of the best things I’ve done in my professional career has been working with Tim,” says Craft. “He’s really great to work with. He’s so smart, and it’s so much fun to talk through this with him, to be thinking about these theories and coming up with questions. It’s been a really fruitful professional collaboration. Not just productive, but enjoyable, too.”

Before his academic career, Vos spent 12 years as a broadcast journalist and interviewed nearly every major party candidate for U.S. President. The experience appears to have stayed with him. As he explains the intricacies of qualitative research in audience feedback, he looks up from his desk and says: “I realize I’m not talking in sound bites.” And after reading the story about the Kentucky clerk in The New York Times, he says he was curious about how NBC would cover the story on the evening news.

“I still think a little bit like, well, a person my age, that 'This will be the big story tonight.' But who’s waiting until tonight?”

Post a Comment

Reader comments are reviewed by Illumination staff before they are posted, so please keep your message civil and appropriate. All fields are required.

– Will not be published

Back to Top

University of Missouri

Published by the Office of Research

© 2021 The Curators of the University of Missouri