10.9: Evaluating Sources

There are five parts to the CRAAP Test:

• CURRENCY: When was the information published? For some topics, it is acceptable (or even necessary) to use an older source. For example, if you want to provide a definition of DNA, you can use a source that was published five or ten years ago, whereas in a paper on the latest DNA discoveries, a five-year-old source might be too old. Another example: if you are writing about a politician’s shifting public statements on a certain issue, you would have to cite some of those statements from sources that may be decades old, depending on how long of a career that politician had — whereas in a paper on effective political statements you should use the latest publications on public communication, reaching specific audiences, etc.
• RELEVANCE: Does this information meet your needs? For example, an article aimed at educating young children about DNA would probably not be a relevant source if you work for a tech firm and are writing a report about whether to acquire some DNA technology. On the other hand, it is acceptable to use a source that isn’t fully relevant to your research question, but only if you can make it more relevant through added analysis. For example, if you were researching the Housing Crisis in Vancouver, you might read about the ways in which other expensive cities like San Francisco and Hong Kong have been dealing with their own housing crises, and then explain which strategies used by those cities could be successfully implemented in Vancouver, as well.
• AUTHORITY: Who wrote the research material? Your source should have been written by someone who has the authority to speak on the matter. For example, you might come across a blog that offers natural health remedies for cancer or depression written by someone claiming to be a doctor. Check that person’s credentials to see if he/she is a medical doctor, has a solid research background in that specific field, etc. To determine how trustworthy an online source is, you can also look at the URL. If it comes from a .gov or .edu website, you can probably trust it, but even such sources should be verified. For instance, some universities, as well as a number of individual academics have been involved in scandals in the last decade — on issues that go from discrimination and harassment to suspicious funding practices. Check to make sure you are not citing from a compromised source (such as a paper on environmental matters produced in a university department funded by the oil industry, for instance, ).
• ACCURACY: How reliable or trustworthy is the information? Specifically, you should examine how the source uses evidence. Does the source link to other trustworthy sources? Does the source support its claims with evidence? How reliable is that evidence? If you are not sure whether a source is reliable, try Googling the source’s claim plus “criticism” or “hoax.” For example, if you read that baking soda cures cancer, you should Google “baking soda cancer cure hoax.” Sometimes, untrustworthy websites take up the top spots in Google, so if you simply Google “baking soda cancer cure,” you’ll find other untrustworthy websites telling the same lies.
• PURPOSE: Why was this information published? You should be able to identify if/how the author/ publisher benefited from publishing this information. Sources that make money aren’t necessarily untrustworthy, but they might not provide sophisticated enough content for your needs. Sources that claim to be of a certain type but are not exactly so might not be reliable. For instance, there are many so-called “academic journals” easily accessible on the internet that claim to be peer-reviewed but engage in suspicious practices and should not be trusted. If you read their instructions for authors, you’ll see that they charge authors hundreds or thousands of dollars — supposedly so they can pay the people who review submissions. No serious academic journal would ask for money from authors. Such journals are predatory enterprises that make money and provide their authors with questionable credentials (anyone who lists an article published in such a venue on his/her academic resume would not be hired by any respectable educational institution).

The SIFT test was developed by Mike Caulfield, Director of Blended and Networked Learning at WSU Vancouver. It provides a list of practical steps to take in order to figure out if you can trust a source.

• STOP: As you find a link/ page/ post, STOP and ask yourself whether you know the website or source of information. What is its reputation? If you don’t have that information, use the next moves to get more clarity. Don’t read or share anything until you know what it is.As you start applying the next steps, don’t forget your purpose. If you just want to repost, read an interesting story, or get a high-level explanation of a concept, it’s probably good enough to find out whether the publication is reputable. If you are doing deep research of your own, you may want verify each individual claim made in the text from other sources to independently verify them. Both sorts of investigations are useful, and for both, stopping periodically and reevaluating our purpose and search strategy is key.
• INVESTIGATE the Source: Make sure you know what you are reading before you read it. For instance, if you are reading a piece on economics by a Nobel prize-winning economist, you should know that before you read it; if you are watching a video on the many benefits of milk consumption that was put out by the dairy industry, you should know that as well. This doesn’t mean the Nobel economist is always be right and that the dairy industry can’t be trusted. But knowing the expertise and agenda of the source is crucial to your interpretation of what they say. Taking a minute to investigate the source before reading will help you decide if it is worth your time, and if it is, help you to better understand its significance and trustworthiness.
• FIND Trusted Coverage: Sometimes you might not be concerned about the source as much as the claim being made. You want to know if it is true or false. You want to know if it represents a consensus viewpoint, or if it is the subject of much disagreement. In this case, your best strategy may be to ignore the source you had initially found and look for trusted reporting or analysis on the claim. If you get an article that says koalas have just been declared extinct from the Save the Koalas Foundation, your best bet might be not to investigate the source but to find the best source you can on this topic, or, just as importantly, to scan multiple sources and see what, the expert consensus seems to be. In other words, find other coverage that better suits your needs — more trusted, more in-depth, etc.Do you have to agree with the consensus once you find it? Not necessarily. Still, understanding the context and history of a claim will help you better evaluate it and better define your starting point for future investigation.
• TRACE Claims, Quotes, and Media Back to the Original Context: Much of what we find on the internet has been stripped of context. Maybe there is a video of a fight between two people with Person A as the aggressor. But what happened before that? What was clipped out of the video and what stayed in? Maybe there is a picture that seems real but the caption could be misleading. Maybe a claim is made about a new medical treatment based on a research finding, but you are not sure if the cited research paper really said that. In these cases, you will have you trace the claim, quote, or media back to the source, so you can see it in its original context and ascertain if the version you saw was accurately presented.