In the third and final part of our interview with Stanford economist James Hamilton, we discuss the connection between investigative journalism, human interest stories and entertainment value, and the difference between getting the truth right and making an impact.


This is the third and final installment of our interview with Stanford Professor James (Jay) Hamilton about his recent book, Democracy’s Detectives: The Economics of Investigative Journalism. Read the first and second parts.


“Fraud and its unraveling,” writes Stanford Professor James (Jay) Hamilton in his recent book Democracy’s Detectives: The Economics of Investigative Journalism, “can both build a business.”

This observation is what makes Hamilton’s book unique—it is not a journalism professor writing about investigative journalism, but an economics professor. Hamilton’s conclusion still holds, but it is not self-evident that this will continue to be the case. In fact, he thinks that investigative journalism should be seen more as a public good—with all that comes along with that designation—and this is the result of an economic analysis.

In the third and final part of our interview with Hamilton, we discuss the connection between investigative journalism and human interest stories and entertainment value, ProPublica’s view of investigative journalism, fact-checking, and the difference between getting the truth right and making an impact, as well as mixing up journalism with advocacy.

James Hamilton

Guy Rolnik: Let me turn for a second to a niche discussion on rational ignorance and rational irrationality. When we look at the data on rational ignorance in the U.S., we see that although we have television, radio, technology, the internet, and so on, voter demand for facts has not gone up, and voters are today as ignorant as they were 50 years ago.

Some people argue that the media is a remedy that we have at our disposal in overcoming rational ignorance and the collective action problem of the dispersed public. Without media, special interests always carry the day, because politicians and regulators know that only special interests monitor them. It’s a great story, but there is a caveat: Despite the media’s efforts, we are still unable to make the electorate more informed. It doesn’t matter how much effort the media puts in, most voters will still be rationally ignorant and rely only on your “Three D’s” (duty, diversion, and drama). We will never get to a point where there is enough accountability.

Jay Hamilton: Well, economists always say, “Relative to what?” Or, “It takes a paradigm to beat a paradigm.”

First, let’s think about rational ignorance and the Catholic Church. The Boston Globe’s Spotlight series shows that if you have a strong human interest side to a story about a failing institution, that can raise the odds it is told. Telling the full story of the Boston Archdiocese’s failure to deal with cases of sexual abuse by priests took significant reporting resources. It took data journalism, it took real digging to figure out what was going on, but that literally changed the world across countries.

An interesting thing about that type of investigative journalism, or the type of investigative journalism that 60 Minutes did, is that if you piggyback off the entertainment demand, if you tell me the personal side of a story, I learn enough about the politics for it to matter, so that’s one thing. Even in a world of rational ignorance, I can still be entertained or intrigued by a story. If we go back to the Progressives, that’s what McClure’s and Ida Tarbell did. If you look at those stories, they were really fascinating from a human-interest perspective.

The second thing is that ProPublica has an interesting model where they say: “We pick a topic that we think we can make change on, but we don’t always have to get it in front of the widest public.” If they have a story, let’s call it “Traumatic Brain Injury in the Military in Iraq and Afghanistan,” they found that getting that story in Stars and Stripes, getting that story in the producer demand for information, once people see that story, sometimes they want to do the right thing and sometimes they also want to do the right thing because they know it could go to a broader public.

That’s another thing to remember: Once the information is created, it doesn’t have to be consumed by everybody to have an impact. The question is, in a world where the average story submitted to the Investigative Reporters and Editors prize contest takes six months of work, how many stories like that can a newspaper do? Even a great newspaper, like The Raleigh News & Observer, which I studied, they can do three deep dives a year, costing $200,000 or $250,000 each. So there’s a huge set of stories that are going to go untold.

I’m optimistic for a couple of reasons. Number one, we haven’t talked about how the market could change. I think computational journalism can lower the cost of discovering stories radically. If you think of investigative reporting as drilling for oil, one of the problems that’s always been there is, where do I make my bet? Where do I look for my story? If you can radically lower the cost of discovering stories, then you will get more of them, regardless of which incentives are creating it.

The second one relates to personalization. We often think of it as left-right dimension. Imagine if there were a site that knew you as well as Amazon, knew what you had read, knew that you liked data rather than personal anecdotes, that product differentiation is something that you would pay for.

I subscribe to The New York Times because, for me, there’s not a good substitute for The New York Times. If you have a site that there’s not a good substitute for, that gives you a product that you’re really interested in, you would subscribe to it. That’s what The New York Times is betting on, and I think if we get more product differentiation, partly through algorithms, you might have a higher chance of people saying I’m willing to pay for it.

GR: In an environment where liberals are only reading The New York Times to reaffirm their beliefs that Donald Trump is going to destroy America, and conservatives are only listening to Sean Hannity and David McCullough to reaffirm their belief that the threat to the U.S. is the left, voters are not truth-seekers. They are just looking to be happy. Politicians and regulators acknowledge that. For them, reelection is not about being judged by the facts, but about being on the right team.

JH: This is where we need Richard Thaler. This is where we need behavioral economists. I talked to one of the leading fact-checkers in the U.S. and I said to him, “Do you think your job ends with the correction, or do you need to go inside somebody’s head and think about the framing and, think about the context, think about what it would take to actually change somebody’s mind?” I think a lot of journalists today still believe almost in a classical truth market, that if they lay out the fact, eventually it’ll get understood, and I don’t think that’s true. That’s one place where we need a lot more research, and it may not be something that the rational economist model works well with.

Let’s take tobacco use, for instance. The Truth campaign, funded by the Tobacco Master Settlement Agreement, found that if you told a 15-year-old that each cigarette they smoke reduces your life by seven minutes, the discounting on that is such that they don’t care. If you tell them tobacco companies are “The Man” and that “The Man is trying to fool you”—if you make it about emotion and rebellion—that gets you the outcome that’s best for society: people don’t start smoking at age 15. I think that’s an area where we need a lot more research.

You’re asking whether we think of information provision just as an intrinsic good, but you also need to think of it instrumentally. You care about information because you care about changing decisions. Right now, I think politicians and tech companies are way ahead of journalists in thinking about context framing changing decisions.

GR: Should the media be looking only to tell the truth and never bother with the question, “How do we change reality?”

JH: Richard Tofel has a great white paper in ProPublica, where he basically says: “We pick out stories based on whether we think we can make a difference, but we’re not advocates in the sense that advocates start with a policy and then reason backwards.” He says, “They start with a problem, do an investigation, and then see where the debate goes from there.”

“… there are profound differences between journalism and advocacy. The most profound of these may begin with process, but culminate in much more: Journalism begins with questions and progresses, as facts are determined, to answers. Advocacy begins with answers, with the facts already assumed to be established. In short, advocates know before they begin work the sort of impact they are seeking, while journalists only learn in the course of their work what the problem is, and only after this can they begin to understand the kind of impact their work might have. Advocates seek impact based on their opinion of societal needs; journalists may identify possible steps toward reform, but should do so only from facts they have established”.

“Issues Around Impact,” ProPublica White Paper, Richard Tofel

I think you’re right. It really should make you think more about what we can learn from other industries about how people actually make decisions.

You need to recognize there are so many countervailing arguments. If you’re in the media, so many people are saying that you’re not to be trusted, that you have a bias. That has to affect how you present your information.

The top topics for investigative journalists: Police, Fraud, and Crime (Source: James Hamilton, Democracy’s Detectives: The Economics
of Investigative Journalism)

GR: Once you want to change reality, you have to have a strategic plan that resembles, but counteracts, that of the special interest group that is against you. If I want to change a market, I have to be thinking strategically, not just laying out the facts. As an airline, or an airline association, I will invest hundreds of millions in dollars in beating the legislation because I get a return. But as a journalist, I can do one story, maybe two stories, and get the Pulitzer. Once I get the Pulitzer, what kind of incentives would I have to follow through in the same way that the special interest group does?

JH: That’s where ProPublica may have an advantage. They basically say, “We’re in it for the long run,” and they do a quarterly impact report for their board. Then, at the end of the year, they have an annual report, where they tell you what their impact has been, and they’re very explicit about it.

I think in the old days, when you had 30-40 percent returns, sometimes the impact was just a side benefit, really a true spillover. Now the thing that will distinguish real public affairs reporting from opinion, from false news, is the real impact that it can have. That can be the product differentiation, and really, it’s actually good business.

I subscribe to The San Jose Mercury News, partly as a consumption act. It’s not a pure investment, partly because I feel like they have an impact in the world and I want to be part of it. That identity consumption comes from the impact that they have. Being more explicit about it can be helpful.

At the same time, you have to be transparent and you have to show your work. You’re not abandoning the provision of facts. It’s really facts plus context.

GR: I totally agree with you, and I’m also excited about the strategy of ProPublica. But ProPublica has a newsroom that costs $15 million a year. But in the U.S., on one side you have corporations and special interests that are spending tens of billions of dollars on lobbying, campaign contributions, buying science, buying academics, buying journalists and PR, and then on the other side, we have few outlets like ProPublica.

JH: I also think that if you look at the Pulitzer Prizes that came out this week, you have The Charleston Gazette-Mail, who did a great job figuring out the number of opioids going into each county.

The nice thing about investigative reporting is that, when it’s done well, it is really interesting, so you do have that entertainment demand associated with it. The other thing is, again, it’s costly today. It’s six months. If you can get that down, then you can do more of it.

GR: I don’t have a way to measure it, but my intuition is that the resources that are dedicated to this kind of impact work are minuscule compared to the resources that the other side invests.

JH: That’s true. I think that’s true, but if it has to go through the ballot box, you do have that hope. It’s always been true that there’s been inequality, yet we have had progressive ages in the U.S. We have had the consumer movement.

At some point, I don’t know if you call it punctuated equilibrium, at some times, dispersed interests do get represented. Often, it’s by political entrepreneurs who get a return at the ballot box for representing the dispersed.

GR: But then again, political entrepreneurs rarely can win if they’re not supported by media coverage. It’s a complicated world out there. People would never know what they are doing. Can a political entrepreneur really move the needle without massive backup of media outlets?

JH: Ralph Nader did it because he did it in a really compelling fashion. “Unsafe at Any Speed,” he was great at telling the industry that story. And the Boston Globe’s Spotlight, they told a really interesting story. At the end of the day, David and Goliath is a really interesting story.