- Mainstream media outlets, including notably The New York Times, have faced accusations of bias from conservatives like President Trump as the partisan climate in US politics has heated up.
- A team of academic researchers from Columbia University and Rutgers University tried to ascertain, from a quantitative perspective, whether The New York Times exhibited any bias in its coverage.
- After comparing Times articles to Reuters articles on the same topic, the researchers did not find any signs of bias.
- Even when it came to reporting on Trump, the study concluded the difference between Times articles and Reuters articles was not significant enough to indicate bias.
- Visit Business Insider’s homepage for more stories.
Mainstream media outlets have faced loud accusations of bias as the partisanship in the political climate in the US has heated up – prominently from President Trump – and perhaps none more than The New York Times.
That’s why a team of researchers from Columbia University and Rutgers University decided to puncture through the speculation by quantitatively measuring New York Times reporting to see if the outlet has any discernible bias, especially when it came to Trump.
What they found surprised one of the team’s lead researchers, Siddhartha Dalal, he told Business Insider.
“Main political figures were talking about it, so I really thought The New York Times would turn out to be biased,” said Dalal, a professor at Columbia’s applied analytics program and statistics department. “The rhetoric is all about media bias, so you think there might be some truth to it.”
But the study found the opposite: The New York Times does not appear to be biased in its coverage of Trump and in its news coverage generally.
The research team measured bias by comparing The New York Times to Reuters using ‘sentiment scores’
While there’s no definitive way to measure what is true and false in the reporting of news, researchers were able to analyse writing style in order to determine whether an article was written more favourably or unfavorably using a system called sentiment analysis, Dalal said.
In order to conduct the analysis, researchers collected 1,058 New York Times articles published over 38 days and paired each with a story from Reuters about the same topic published within the same 24 hours.
“We chose Reuters for this purpose since it is an international news agency with a policy of neutral writing and a customer base of media across the entire political spectrum,” the researchers wrote in the study.
Dalal and his team used code packages in R and Python to calculate a sentiment score for each story, which indicates if content is more positive or negative.
For instance, a sentence with words like “happy,” “satisfactory,” or “good” would be considered favourable, while words like “failure” indicate a negative sentiment, researchers wrote in the study. The codes understand modifiers as well, so a sentence that includes “not happy” would be correctly labelled as unfavourable.
Results showed that front page stories from both outlets had the same sentiment scores, meaning there was no overall difference in the favorability of front page news between The Times and Reuters.
The Times tended to be more positive in its coverage of business and slightly more negative in international and national news articles than Reuters.
‘But what about Trump?’ the study asks
To test potential bias in coverage of Trump, researchers gathered articles using a keyword search on both publications. If an article had “Trump” as a keyword, the team determined he was the main focus of the story.
Again, the typical difference between the sentiment scores was close to zero, demonstrating stories from both publications were written using similar words to convey similar sentiments, according to the study.
Reuters had a few more positive Trump stories than The Times, but Dalal said the difference was too small to be considered statistically significant based on sample size.
“Reuters is slightly positive, but when you’re comparing thousands of articles there’s going to be some randomness in there,” he said. “There was no resounding media biased we could find.”
Dalal said he hoped the findings encourage people to think more critically about media bias and how to determine if it exists regardless of their personal politics.
“I’m apolitical, I’m not a political person, but I wanted to figure out if this rhetoric was true or not,” he said. “We need to use algorithms to determine these things because data isn’t biased.”
Business Insider Emails & Alerts
Site highlights each day to your inbox.