What Are Reliable Sources of Information?
Many have noticed a profusion of “fake news” and incorrect information circulating in the media. Is there any way to distinguish information that is strongly biased, distorted, or even fabricated from information more in line with reality? This article is an attempt to answer that question. The executive summary at the end of the article contains my ranked list of trustworthy and untrustworthy sources of information.
Kim Griest: July 2020
Introduction
First, note that as a scientist, I am convinced that not all information is equally correct. Some people think that all opinions are equally valid and it is OK to say: “My beliefs are true for me and yours can be true for you”. On topics of aesthetics or even morality, this may be true, e.g. Is Matisse or Picasso the better painter? But in many cases one “fact” or “theory” is true and the others are false. In the 17th century there were two popular theories about the relationship between the Sun and the Earth. Some thought the Sun circled the Earth and others thought the Earth circled the Sun. One theory was false and the other was true, though the argument went on for around 100 years before the true theory became accepted. In the same way, for around 50 years, wealthy people and tobacco companies stated that there was no proof that cigarettes caused cancer, while almost all scientists said, yes, the proof exists. One side was wrong and the other right. Currently most conservative/libertarian billionaires and politicians say that human burning of fossil fuels is not causing global warming/climate change. Scientists almost uniformly say it is. Again, one side is wrong and the other correct.
So, I start from the hypothesis that in many cases where there is disagreement on an issue with two or more sides, one side is right and the others are wrong, especially when the disagreement is about measurable things, facts of history, or scientific ideas. I also believe there are many cases that are purely matters of point of view, morality, or opinion. These can be vigorously argued as long as all sides of the debate stick to actual true information to support their views.
But how to tell whether to believe the tobacco and oil companies or the scientific community? That is the question we ask here.
The most important thing when questioning the reliability of information is to know the reputation of the source of the information. It’s just like with people; you trust people who have a reputation for honesty and a history of telling you the truth. And you distrust information coming from people who have lied to you in the past. Thus one should believe information coming from sources with a history of truth-telling and ignore information coming from sources with a history of distortion, or strong bias, or lying.
The second most important thing is probably the motivation of the source. If a media source is wholly owned by a person or corporation, it is very likely that no information that will harm that person or corporation will ever be published. If I owned a newspaper, it would never publish anything bad about me; it’s my newspaper, and that’s what ownership means. Wealthy people and corporations, of course, use their wealth to increase their wealth, so wariness is warranted when information from a given source might help or hurt the profits of that source.
Next, information coming from multiple, independent sources is more likely to be correct than information coming from a single source.
Finally, common sense is important. Remarkable claims require remarkable evidence, so awareness should be heightened when a claim is surprising, unusual, or contradicts other, well accepted, things.
So given the above what sources are most reliable? This is the question everyone should be asking! Please make a list for yourself of reliable and unreliable sources. Then follow it! Don’t read, click on, quote from, or repost ANY information coming from sources you have decided are unreliable.
Now, some argue that even unreliable sources sometimes give true and even important information. They claim you should read things from all sources, even unreliable ones, and then “decide for yourself”. In my opinion, this is a waste of time. A broken clock sometimes has the correct time (twice a day), but it can never be used to tell time, because you never know WHEN that is, without checking a working clock. In the same way, an unreliable source MAY give correct reporting or analysis, but you can’t know when that is without checking a reliable source. So with both a broken clock and an unreliable news source, just skip the unreliable source and go straight to a reliable source. If the topic interests you, you can try to find information on that topic from a source from your “reliable” list. If you can’t find it, then just assume the information is untrue and move on. If the information is true, trust that it will eventually come out from a reliable source.
Ranking Trustworthy Information Sources
So using source reputation, source ownership, multiple sourcing, and common sense, which information sources are the most reliable? In my opinion, number one in reliability are peer-reviewed papers in reputable scientific journals.
Scientists use a kind of democracy to ensure that the information they consume is true. Other reputable scientists read each submitted paper and send comments/questions to the author. Only if those other scientists are convinced that there was no exaggeration, deception, or mistakes, is the paper accepted for publication. Of course, mistakes are made and sometimes scientists fool other scientists and get bad information published. The scientific community has a way of correcting these errors however. When it is discovered that an author falsified data or made mistakes, other authors write papers correcting that mistake. If it is shown to be falsified data, then the falsifying author’s reputation is destroyed, and typically no other scientist will ever read or consider another paper by that person. Scientists live in fear of making mistakes and tarnishing their reputations. Scientists accused of fraud whose reputations are ruined are exiled and sometimes become bitter, turning against others in the field and even science itself. So one should be careful of scientists whose reputations are not good among others in their field.
Note, that only scientists with very good reputations are asked to referee papers (and also referee grant proposals), so science is a self-correcting, truth-finding system. Random errors and unlikely chances can also cause scientific results to be wrong, so for important results, the work is repeated several times until there is almost no chance of it being incorrect. This is the gold standard for information: Repeatedly checked, peer-reviewed theories and data.
Now some people complain that science changes and old theories are proved false and replaced by new ones. This is true. Sometimes what is considered true is proved false (or more likely not as accurate as some new data or theory), and science has to change. That is a good thing! It in no way undermines the fact that peer-reviewed articles in reputable journals are the most reliable sources of information we have, even if there can be wrong theories, mistakes, and out-of-date facts published in them. Note also, that scientists are less impressed with brand new results than the general public. It sometimes takes time to repeat an experiment or calculation or to discover mistakes. So science “news” is not the best source of reliable information. Probably the absolute best sources are “review papers”; usually famous scientists reviewing the entire scientific literature on a given subject and listing what is known with high confidence, what is known with some confidence, and what is still uncertain. One typically has to be an expert in the field to be able to read these high end review papers, so the general public has to rely on reputable science reporters who try and simplify things for the non-expert. Sometimes errors are made in the simplification, so again going to the experts with specific questions is often the best method. Thus one must distinguish between real science and “science news” where the problem of oversimplification is compounded with the problem of results not yet completely vetted and reproduced.
Some also say that scientists are only in it for their grant money and so can’t be trusted. While grants are very important to scientists, in democratic nations, grants are awarded competitively by panels of scientists themselves. Having sat on many such panels, I can attest that we award grants to the scientists who make the most interesting proposals and who we think are most capable of succeeding at making original, accurate, and transformative discoveries. Fame and prestige in science come from overthrowing a standard theory or from discovering something really new. Thus scientists are bad at defending the status quo merely for the sake tradition; we all want to be the one who starts a scientific revolution. And when someone else starts a scientific revolution, we want to be among the first to work in the exciting new field, where important discoveries are easier to make. Of course, science has a very tough standard of proof, and a really new discovery requires remarkably strong evidence, so mostly we end up just adding a small detail to the body of science, or proving that the standard theory is correct yet again.
So top of our list of reliable sources is Number 1: peer-reviewed scientific papers in reputable journals, with well-received review articles being the cream of this top category.My entire list of reliable and unreliable sources is given in the Executive Summary at the end of the article.
Related to my top category of review papers are government reports that are supposed to be reviews of some topic. Examples included papers from NASA, the US CDC or Department of Energy and international reports from the World Health Organization (W.H.O.), the Intergovernmental Panel on Climate Change (IPCC), etc. We list these information sources as number 2 in reliability, somewhat below the refereed journal article category because of the “ownership” problem. Almost all reputable journals let scientists control what gets published. Many journals are published by the scientific organizations themselves and even for those that are privately owned (e.g. Elsevier), scientists are in complete control of the content. The fees that are charged by these privately owned publishers are independent of content, so there is no obvious advantage to the owners for publishing false information; especially since the journal’s reputation is of such importance. In some cases scientists themselves have patent rights to some drug or invention they publish, and the journals usually require disclosure of these “conflicts of interest”. However, still there are cases where scientists distort or fabricate things for their own financial benefit; after all, scientists are humans with human desires. One must always look carefully when financial interests are at stake, and lower one’s trust in any article where one suspects financial incentives. However, in general, especially in science areas where patents are not involved, the scientists themselves have no financial incentive to distort or lie. They are in it for pure curiosity, or fame, or promotions, and the fear of destroying their reputation by a mistake or distortion keeps the vast majority of scientists careful and honest.
So, information coming from government agencies where the reports are prepared by famous scientists is also usually very reliable for the same reasons; they are supposedly reviews of literature with no obvious advantage to authors for distorting or lying. However, this is only true for democracies and for international organizations where scientists themselves are in control, and where the line-of-command is clear. In dictatorships, such as China or Saudi Arabia, where free speech is not available, official reports should be viewed as political documents and treated with great caution. Examples include Saudi Arabia’s fabricated “official” reporting of their oil reserves, or China’s Covid-19 deaths and rates. Semi-democracies, such as Russia are also suspect since the government can easily control what is published. International documents such as the IPCC report on climate change caused by fossil fuel burning are quite reliable since scientists from many democracies write them, but still these reports do not reach the same level of trustworthiness as peer-reviewed review articles in reputable journals, since the content is subject to political review by the member nations. If the political interference is too great, then scientists, worried about their reputation, will resign, so the information can’t get too far from true. Again it is important to know the history and reputation of each publishing source. NASA and the CDC have sterling reputations and so are very trustworthy. Some review paper from an unknown source, is much more uncertain. And a review paper from a source known in the past to distort or fabricate (e.g. Saudi Arabia Oil Ministry) should never be trusted. In between we have organizations such as the W.H.O. who do have pressure placed upon them by various governments and have occasionally given in to that pressure.
Thus we list our 2nd and 3rd most reliable sources of information in the Executive Summary: Number 2. Official information coming from reputable places: NASA, CDC, US Bureau of Labor Statistics, US Department of Energy, major universities, etc. And Number 3. Government laboratories in nations with elected, democratic governments.
After reputable organizations made up of reputable people, the next most trustworthy sources are articles or books published by individuals with good reputations. If you want to know the “facts” of history, a well-known historian from a high reputation university is a good bet. There are, of course, exceptions. So we have: Number 4. Published works by reputable academics from well-known universities.
Of course, much information we are interested in is not published in scientific journals and is not scientific in nature. And usually “facts” make it into history articles and books long after they are current. How should we choose reliable information of this type (which is normally known as news)?
Again, reputation, source ownership, multiple sources, and common sense are important. Since Chomsky’s and Herman’s seminal book: Manufacturing Consent, in the 1980s, the topic of bias in news sources based on source ownership and the interests of the powerful has been studied deeply. We will not try to even recap that research here. For many topics, no regular news source is really accurate, and one must develop special sources for those topics. In the US almost all media is owned by wealthy people and/or large corporations, so one cannot expect to find information which is bad for business interests or the wealthy in any normal news source. For example, even the supposedly liberal New York Times repeated and strongly supported the lies of the G.W. Bush administration in the buildup to the Iraq invasion. Understanding the how and why of this is beyond the scope of this short article. However, here we are not writing for the sophisticated progressive, but for the average person who is confused by the contradictory “news” coming from their facebook friend’s posts. If you are interested in understanding the “why”, you might start with the most recent edition of Chomsky’s and Herman’s book, or look at my short article: https://medium.com/@kimgriest/why-so-much-pseudo-science-garbage-on-the-internet-b1ae2fb8cd5
So, here, we will take a simplified approach toward ranking normal news sources. There are several groups that rank sources on media bias and fact reliability, and there are even courses at universities on this subject. I recommend relying on these rating groups until you develop your own list of trustworthy and untrustworthy sources.
The best rating website I’ve come across so far is adfontesmedia.com. This source is recommended by the Cornell University Library and is rated as very reliable and “least biased” by other rating websites such as mediabiasfactcheck.com. I like Ad Fontes (which means “to the sources” in Latin) because it rates news sources on both the “Information Quality” and the “Liberal vs. Conservative” scales. I am not very interested in the Liberal vs. Conservative rating, because these ratings are really just opinions: they use a panel of a conservative, a liberal, and a neutral judge to rate “Liberal” vs. “Conservative”. However, I am very interested in the information quality rating, which they calculate by reading and fact checking a sample of articles from each information source. Ad Fontes is trying to make money by subscriptions, so I find their current free charts a little hard to use, but here is a plot from a year or so ago.
Please spend some time with this figure, noting especially that the lower a source is listed on the figure, the more unreliable it is. I think it is important to use mainly sources from the Green and Yellow boxes, and to ignore sources from the Orange and Red boxes. The Green Box (fact reporting) and Yellow Box (analysis and fair interpretations of the news) include many well-known sources from both the left and right: e.g. AP, Reuters, NPR, Bloomberg, Christian Science Monitor, BBC, USA Today, Wall Street Journal, ABC News, CBS News, New York Times, Washington Post, Guardian, etc.
The Orange Box (selective or incomplete stories with unfair interpretations of the news) and Red Box (inaccurate/fabricated information including nonsense damaging to public discourse) include the Daily KOS, OAN, Drudge Report, Occupy Democrats, Daily Wire, National Enquirer, Infowars, and Fox News.
So, if you like conservative analysis, I recommend switching from FOX, Breitbart, etc. which are in the Red Box, and subscribing to the Wall Street Journal or The Hill which are in the Green Box. For a left wing/neutral viewpoint, most any of the well-known media sources are OK, with the most well- known being the highest quality. Also, note that local FOX news stations would be counted as “local news” which are listed in the “reliable” area of the figure. Of course, there are differences between different local media sources, and one should check for yourself the reliability of your own local news sources.
So, in conclusion, before clicking on a friend’s facebook post, check the source byline. If it is FOX news, or one of the other sources in the “Red Box”, then I recommend not clicking on it. Why fill your mind with information that is very likely false? Try to find information on that subject from a source with a good reputation. If the post is a video from someone on youtube, see if you can find the actual source. If you can’t, then be very suspicious and don’t trust anything you hear that sounds unlikely. Hope this helps.
Frequently Asked Question:
Question: Someone quotes results published in a scientific journal. Should I believe that result? Answer: From the list in the executive summary this seems like an easy question, but it is actually a tricky question! You need to ask (and answer) two questions first.
1. Is it a reputable journal? I have a colleague who couldn’t get his nutcase theory published in any reputable Physics Journal, so he started his own online journal! It is called “Journal of Cosmology”, which sounds very respectable. Of course it published his garbage papers. This journal is now known by normal scientists as a “fake journal” and no practicing scientist ever looks at or refers to anything published in it. But unless you are an expert in the field how would you know that? You have to ask an expert! Or do research on the reputation of that journal. Unfortunately there are thousands of such fake journals now, so be careful.
2. How recent is the paper and has the result been reproduced? Very recent results are always a little suspect. There is some review done by the paper’s referee(s), but that is usually only one or two people and could be the friend of the authors. Until other scientists have the chance to read, think about, and try to reproduce a given result, it is fine to be curious, or even excited about a new result, but one should hold onto some skepticism. If the result has been around a while, has been reproduced, has other papers making use of the results, etc. then the result is much more likely to be correct and can be believed. Of course, ALL scientific results are subject to change/revision/overthrow and scientists are aware of this. However, remarkable results require remarkable evidence, so if a result is unusual and out-of-line with past results, common knowledge, or common sense, then one should remain skeptical until very compelling data is at hand. Of course, many currently accepted theories went through this stage: Einstein’s Relativity, Quantum Mechanics, Plate Tectonics, etc. were very unusual and out-of-line with accepted conventions when they first appeared. It is always an exciting time in science when such revolutions happen, but such revolutions are fairly rare and really only experts in the relevant field can tell what is going on at such times.
Executive Summary
Trustworthy sources of information in ranked order, with number 1 being the most trustworthy.
1. Peer reviewed scientific papers in reputable journals, especially review articles by respected experts in the relevant field.
2. Official information coming from reputable places: NASA, CDC, US Bureau of Labor Statistics, US Department of Energy, major universities, etc.
3. Government laboratories in nations with elected, democratic governments.
4. Published papers and books by reputable academics at well-known universities.
5. Reputable news sources not owned by conservative/libertarian billionaires, large corporations, or non-democratic governments, for example, the BBC, NPR, the Guardian, Christian Science Monitor, etc.
6. News sources owned by the wealthy or large corporations, but which have shown they care about their reputation for accurate reporting: AP, Reuters, Bloomberg, New York Times, Wall Street Journal, Washington Post, LA Times, Financial Times, etc.
[Note: Sources listed in items 7 and 8 should be considered untrustworthy and unreliable.]
7. Sources owned by the wealthy, large corporations, or their followers, that have consistently been shown to distort, misinform, or lie: FOX news, OAN, Daily Express, many AM talk radio shows, Breitbart, Infowars, Daily Wire, Project Veritas, Judicial Watch, etc.
8. High quality, seemingly professionally produced, youtube videos by “disaffected insiders” that are designed to emotionally manipulate you into accepting some strange pseudo-science or conspiracy theory. (Who is paying for these videos anyway?)