A How-To On Reading Scientific Papers

“Be skeptical. But when you get proof, accept proof.” – Michael Specter

That quote is from Denialism: How Irrational Thinking Hinders Scientific Progress, Harms the Planet, and Threatens Our Lives, where New Yorker staff writer Michael Specter examined the distrust of science that’s turned discussion of scientific topics into a potential minefield. Some good examples of that minefield are climate change, and childhood vaccinations.

Anyone interested in scientific progress – full disclosure, I’m in that group – needs to understand the ideas being explored in scientific papers, the dispatches from the front lines of scientific thinking and discovery. To arrive at that understanding, you have to be able to understand what you’re reading, and I’ll be the first to admit that isn’t easy.

Scientific papers are written by scientists, for scientists, and follow a set of rules and formal structures that can feel like they’re designed to prevent any understanding by the average Joe/Jane “just plain human.” In this post, my goal is to help anyone interested in, but not formally trained in, science tackle reading – and understanding! – an article in any scientific journal.

10 steps to scientific (article) understanding

  1. Check the source

    • What journal is publishing the article? Check Beall’s List, and if the journal appears there, you can stop reading – it’s a fake journal.
    • Who is the lead author, and what organization or institution is s/he affiliated with? If it’s an established university or research institute (University of Chicago or Scripps Institute, for example), keep reading.
  2. Read the introduction first, not the abstract

    • The introduction will reveal the Big Question, the one that the research project worked to reveal the answer to. For instance, an article in the Christmas 2017 issue of The BMJ reports on research into the effects of pet ownership on human biomarkers of ageing; the introduction clearly lays out the Big Question as “ we examined the prospective link between pet ownership and a selected range of objective biomarkers of ageing proposed for use in large scale population based studies of older people.”
  3. Write out your own summary of what the research was examining

    • This will give you a grasp of why the researchers wanted to ask the Big Question, and a framework for assessing what their answers to that question are.
  4. Identify the null hypothesis

    • The null hypothesis could really be better termed the “nullifiable” hypothesis, since the purpose of the research project is to nullify the hypothesis that there are no differences in possible answers to the Big Question.
    • An example of a null hypothesis is “the world is flat,” which is what Copernicus worked to scientifically disprove a while back. He was successful, but there are some people who still reject his conclusions. (Warning: opening that link might be hazardous to your sanity.)
  5. Look at the approach, and the methods, used in the research study or experiment(s)

    • What did the researchers do to answer the Big Question? What specific experiments did they run?
    • Sketch out diagrams of each experiment or data crunch.
  6. Read the results section of the article

    • Look at the written results, as well as all charts and figures related to those results.
    • What are the sample sizes? Really small sample sizes are a red flag.
    • What results are listed as “significant,” and what as “non-significant”? If you want to totally geek out on this topic, this post will make your geeky day.
  7. Do the results actually answer the Big Question?

    • Using your own judgment, do you think the study authors have answered the question asked in the introduction?
    • Do this before you read the paper’s conclusion.
  8. Does the conclusion make sense, in light of everything you’ve read and evaluated while going through the paper?

    • Do you agree with the conclusion?
    • Can you identify an alternative explanation for the results in the article?
    • What are the next steps the authors see emerging from their research?
  9. Read the abstract at the beginning of the paper

    • In light of the work you did in Steps 1 through 8, does the abstract line up with what the authors said their research purpose was?
    • Does it fit with your own interpretation of the paper?
  10. What are other scientists saying about the paper?

    • Have other scientists written about this paper?
    • What other research is referenced in the paper?
    • Have the authors of that research weighed in on the paper you’re evaluating?

Reading, and understanding, scientific papers takes practice. It’s also fun, if you’re a science nerd, or just interested in new scientific discoveries. And it’s work worth doing, because the more you know, the more likely it is that you yourself might make a discovery that makes a difference.

Paying It Forward: Volunteering for Clinical Trials

Editor’s Note: This blog and video is from the Alliance for Aging Research. The Alliance for Aging Research is dedicated to accelerating the pace of scientific discoveries and their application to vastly improve the universal human experience of aging and health.

Getting medical discoveries from the research lab to patients depends on clinical trials and the people who volunteer to participate in them.   Volunteering in a trial may help society at large by bringing new treatments one step closer to patients, and could help a loved one if you have a genetic disease or condition.  Volunteering may also give you access to a cutting-edge treatment and medical team that carefully monitors your health.  But clinical trials can’t happen without volunteers, and 37% of trials don’t enroll enough patients to move forward.  Clinical trials need volunteers like you so watch this short film to find out more about why they are important, how to get involved, and what it means to participate.

How to Read and Understand a Scientific Paper

In a previous article, How to Read Beyond the Headline: 9 Essential Questions to Evaluate Medical News, I recommended you should always try to read an original study (if cited) to evaluate the information presented. In this follow-on article, you will learn how to read a scientific research paper so that you can come to an informed opinion on the latest research in your field of interest.  Understanding research literature is an important skill for patient advocates, and as with any skill, it can be learned with practice and time.

Let’s start by looking at what exactly we mean by the term “scientific paper”. Scientific papers are written reports describing original research findings. They are published in peer reviewed journals, which means they have been refereed by at least two other experts (unpaid and anonymized) in the field of study in order to determine the article’s scientific validity.

You may also come across the following types of scientific papers in the course of your research.

•       Scientific review papers are also published in peer reviewed journals, but seek to synthesize and summarize the work of a particular sub-field, rather than report on new results.

•       Conference proceedings, which may be published in a journal, are referred to as the “Proceedings of Conference X”. They will sometimes go through peer review, but not always.

•       Editorials, commentaries and letters to the editor offer a review or critique of original articles. They are not peer-reviewed.

Most scientific journals follow the IMRD format, meaning its publications will usually consist of an Abstract followed by:

•       Introduction

•       Methods

•       Results

•       Discussion

 

Let’s look at each of these sections in turn.

(a) Introduction  

The Introduction should provide you with enough information to understand the article. It should establish the scientific significance of the study and demonstrate a relevant context for the current study.  The scope and objectives of the study should be clearly stated.

When reading the Introduction, ask yourself the following questions:

·       What specific problem does this research address?

·       Why is this study important?

(b) Methods

The Methods section outlines how the work was done to answer the study’s hypothesis. It should explain new methodology in detail and types of data recorded.

As you read this section, look for answers to the following questions:

  • What procedures were followed?
  • Are the treatments clearly described?
  • How many people did the research study include? In general, the larger a study the more you can trust its results. Small studies may miss important differences because they lack statistical power. Case studies (i.e. those based on single patients or single observations) are no longer regarded as scientific rigorous.
  • Did the study include a control group? A control group allows researchers to compare outcomes in those who receive a treatment with those who don’t.

 (c) Results

The Results section presents the study’s findings.  It should follow a logical sequence to answer the study hypothesis.  Pay careful attention to any data sets shown in graphs, tables, and diagrams. Try to interpret the data first before reading the captions and details.  If you are unfamiliar with statistics, you will find a helpful glossary of terms hereClick here for an online guide to help you understand key concepts of statistics and how these concepts relate to the scientific method and research.

Consider the following questions:

  • Are the findings supported by persuasive evidence?
  • Is there an alternative way to interpret these findings?

(d) Discussion 

The Discussion places the study in the context of the broader field of research. It should explain how the research has moved the body of scientific knowledge forward and outline the next steps for further study.

Questions to ask:

•       Does the study have any limitations? Limitations are the conditions or influences that cannot be controlled by the researcher.  Any limitations that might influence the results should be mentioned in the study’s findings.

  • How are the findings new or supportive of other work in the field?
  • What are some of the specific applications of the study’s findings?

The IMRD format provides you with a useful framework to read a scientific paper. You will need to read a paper several times to understand its findings. Consider your first reading of the study as a “big picture” reading.  Scan the Abstract for a summary of the study’s principal objectives, the methods it used and the principal conclusions. A well-written abstract should allow you to identify the basic content of an article to determine its relevance to you.  In describing how she determines the relevance of a study, research RN, Katy Hanlon, focuses on “key words and phrases first. Those that relate to the author/s base proposal as well as my own interests”.  Medical writer, Nora Cutcliffe, also scans upfront “to gauge power and relevance of clinical trial data”. She looks for “study enrollment (n), country and year”. It’s important to note the publication date to determine if this article contains the latest findings or if there is more up-to-date research available. Cutcliffe also advises you should “note author affiliations and study sponsors”.  Here you are looking out for any potential bias or vested interest in a particular outcome.  Check the Acknowledgments section to see if the author(s) declare any financial interests in the research which might bias their findings. Finally, check if the article is published in a credible journal.  You will find reputable biomedical journals indexed by Pubmed and Web of Science.

Next, circle or take note of any scientific terms or keywords you don’t understand and look up their meaning before your second reading. Scan the References section – you may even want to read an article listed here first to help you better understand the current study.

With the second reading you are going to deepen your comprehension of the study. You’ll want to highlight key points, consult the references, and take notes as you read.  According to the scientific publisher, Elsevier, “reading a scientific paper should not be done in a linear way (from beginning to end); instead, it should be done strategically and with a critical mindset, questioning your understanding and the findings.”  Scientist, Dr Jennifer Raff, agrees. “When I’m choosing papers to read, I decide what’s relevant to my interests based on a combination of the title and abstract”, she writes in How to read and understand a scientific paper: a guide for non-scientists. “But when I’ve got a collection of papers assembled for deep reading, I always read the abstract last”. Raff explains she does this “because abstracts contain a succinct summary of the entire paper, and I’m concerned about inadvertently becoming biased by the authors’ interpretation of the results”.

When you have read the article through several times, try to distill it down to its scientific essence, using your own words. Write down the key points you have gleaned from your reading such as the purpose of the study, main findings and conclusions. You might find it helpful to develop a template for recording notes, or adapt the template below for use. You will then have a useful resource to find the correct reference and to cross reference when you want to consult an article in the future.

In the example below I have taken an article published in 2015, as an example. You can read the paper Twitter Social Media is an Effective Tool for Breast Cancer Patient Education and Support: Patient-Reported Outcomes by Survey on PubMed.

Template for Taking Notes on Research Articles

 

 

Further reading

Nothing About Us Without Us: Patient Involvement in Research

Until recently, patient participation in research was limited to their involvement as subjects enrolled in research studies, but there is a shift occurring as funding bodies increasingly look for evidence of patient and public involvement (PPI) in research proposals. The rationale for this is increasing evidence that PPI in the provision of healthcare leads to improved outcomes and better quality of care.

Assumptions are made every day about patients; assumptions which may lead to a failure to deliver optimum care. When these assumptions extend to research, quite often there is a mismatch between the questions that patients want answers to and the ones that researchers are investigating. As an example, the research priorities of patients with osteoarthritis of the knee, and the clinicians looking after them, were shown in a study to favor more rigorous evaluation of physiotherapy and surgery, and assessment of educational and coping strategies. Only 9% of patients wanted more research on drugs, yet over 80% of randomized controlled trials in patients with osteoarthritis of the knee were drug evaluations. PPI recognizes that patients bring a unique perspective and experience to the decision-making process in research. It is paternalistic and patronizing to rely on speculation about patient experience. By considering the actual experience of patients, researchers can make more informed research decisions. Involving patients is an important step in ensuring that the real life experiences of patients are considered when it comes to setting research priorities. This in turn will increase the relevance of research to patients and improve research quality and outcomes.

As an advocate you may be asked to become involved in a research project, so it is important to have a clear understanding of what PPI is – and what it isn’t. PPI is not about being recruited as a participant in a clinical trial or other research project, donating sample material for research, answering questionnaires or providing opinions. PPI describes a variety of ways that researchers engage with people for whom their research holds relevance. It spans a spectrum of involvement which may include any of the following:

  • Being involved in defining the research question
  • Being a co-applicant in a research proposal
  • Working with funders to review patient-focused section of applications
  • Being an active member of a steering group for a research study
  • Providing your input into a study’s conception and design
  • Contributing to/proofing of documentation
  • Assisting in the implementation and dissemination of research outcomes
  • Improving access to patients via peer networks and accessing difficult-to-reach patients and groups

Effective PPI transforms the traditional research hierarchy in which studies are done to, on, or for participants into a partnership model in which research is carried out with or by patients.  PPI should always involve meaningful patient participation and avoid tokenism. The Canadian Institutes of Health Research Strategy for Patient-Oriented Research (SPOR) describes PPI as fostering a climate in which researchers, health care providers, decision-makers and policy-makers understand the value of patient involvement and patients see the value of these interactions. Underpinning this framework are the following guiding principles for integrating patient engagement into research:

  • Inclusiveness:Patient engagement in research integrates a diversity of patient perspectives and research is reflective of their contribution.
  • Support:Adequate support and flexibility are provided to patient participants to ensure that they can contribute fully to discussions and decisions. This implies creating safe environments that promote honest interactions, cultural competence, training, and education. Support also implies financial compensation for their involvement.
  • Mutual Respect:Researchers, practitioners and patients acknowledge and value each other’s expertise and experiential knowledge.
  • Co-Build:Patients, researchers and practitioners work together from the beginning to identify problems and gaps, set priorities for research and work together to produce and implement solutions.

Derek Stewart, a patient advocate and Associate Director for Patient and Public Involvement at NIHR Clinical Research Network, sees a growing momentum of actively involving patients and public in research gathering pace worldwide. “It is really pleasing to hear researchers saying how valuable it has been to involve patients and the public in their work”, he says. “It has equally improved the quality of the research and enriched their own thinking and understanding.”

Earlier this year, PCORnet, the National Patient-Centered Clinical Research Network, announced its first demonstration study which reflects PCORnet’s aims of patient engagement and open science. ADAPTABLE (Aspirin Dosing: A Patient-centric Trial Assessing Benefits and Long-Term Effectiveness) will compare the effect of two different aspirin doses given to prevent heart attacks and strokes in high-risk patients with a history of heart disease. Seeking input at every critical step, from consent design and protocol development, through dissemination of final study results, the project represents a new research paradigm. Unprecedented in the design of clinical trials, the final consent form and protocol were shaped with input from patients, local institutional review boards, physicians, and study coordinators.

Another noteworthy example of PPI can be found in the Metastatic Breast Cancer Project a direct-to-patients initiative launched at the Broad Institute of MIT and Harvard last October. Corrie Painter, an angiosarcoma patient and Associate Director of Operations and Scientific Outreach at Broad Institute, explains that “the project seeks to greatly accelerate the pace of biomedical research by empowering patients to directly contribute to research and was built in lock step from design to consent language with dozens of patients.”

To what extent you may wish to be involved in PPI will depend on several factors. Do you have professional experience (e.g. project management, clinical experience, etc.) which would be useful? Are you happy to work as part of a team? Or would you prefer to work on your own? You should also take into consideration your other work or family commitments. For instance will you need to take time off work to attend meetings? Consider also at what point you are in your own health journey. Will participation in research place an added burden on your treatment or recovery? In making the decision to become involved in research, you should always balance your own health needs with the desire to be supportive of research and the research process.

 

Useful links

PCORI www.pcori.org

PCORnet www.pcornet.org

Metastatic Breast Cancer Project www.mbcproject.org

#WhyWeDoResearch www.whywedoresearch.weebly.com

How to Read Beyond the Headline: 9 Essential Questions to Evaluate Medical News

Ben Goldacre writing in Bad Science classified science reporting as falling into three categories – wacky stories, scare stories and breakthrough stories; the last of which he views as ”a more subtly destructive category of science story”. Whether you get your news through digital or traditional means, you can’t fail to notice the regularity with which journalists report on the latest medical breakthroughs. Some of these reports are sensationalist (“coffee causes cancer”) and fairly easy to dismiss; but do you know how to separate fact from fiction when it comes to less sensationalist headlines?

The foundation of empowered patient-hood is built on reliable health information. This means not only knowing where to find medical information, but being able to evaluate it and knowing how it can be applied to your own, or your loved-ones’ particular circumstances. Headlines often mislead people into thinking a certain substance or activity will prevent or cure chronic disease. As patient advocates we must learn to read beyond the headlines to filter out the good, the bad, and the questionable. The following questions are designed to help sort the signal from the noise next time you read the latest news story heralding a medical breakthrough.

1. Does the article support its claims with scientific research?

Your first concern should be the research behind the news article. If an article contains no link to scientific research to support its claims, then be very wary about treating those claims as scientifically credible.

2. What is the original source of the article?

If the article cites scientific research you should still treat the findings with caution. Always consider the source. Find out where the study was done. Who paid for and conducted the study? Is there a potential conflict of interest?

3. Does the article contain expert commentary to back up claims?

Look for expert independent commentary from doctors or other healthcare providers to explain the findings (there should be an independent expert source quoted – someone not directly connected with the research).

4. Is this a conference presentation?

Journalists frequently report on research presented at large scientific meetings. It’s important to realize that this research may only be at a preliminary stage and may not fulfill its early promise.

5. What kind of clinical trial is being reported on?

If the news relates to results from a clinical trial, it’s important you understand how, or even if, the results apply to you. Quite often, news publications report on trials which have not yet been conducted on humans. Many drugs that show promising results in animals don’t work in humans. Cancer.Net and American Cancer Society have useful guides to understanding the format of cancer research studies.

6. What stage is the trial at?

Research studies must go through several phases before a treatment can be considered safe and effective; but many times journalists report on early phase trials as if these hold all the answers. The testing process in humans is divided into several phases:

  •  Phase I trials: Researchers test a new drug or treatment in a small group of people for the first time to evaluate its safety, determine a safe dosage range, and identify side effects.
  • Phase II trials: The drug or treatment is given to a larger group of people to see if it is effective and to further evaluate its safety.
  • Phase III trials: The drug or treatment is given to large groups of people to confirm its effectiveness, monitor side effects, compare it to commonly used treatments, and collect information that will allow the drug or treatment to be used safely.

Source: ClinicalTrials.gov

7. How many people did the research study include?

In general, the larger a study the more you can trust its results. Small studies may miss important differences because they lack statistical power.

8. Did the study include a control group?

A control group allows researchers to compare outcomes in those who receive a treatment with those who don’t. The gold standard is a “randomised controlled trial”, a study in which participants are randomly allocated to receive (or not receive) a particular intervention (e.g. a treatment or a placebo).

9. What are the study’s limitations?

Many news stories fail to point out the limitations of the evidence. The limitations of a study are the shortcomings, conditions or influences that cannot be controlled by the researcher. Any limitations that might influence the results should be mentioned in the study’s findings, so always read the original study where possible.

Useful Resources

  • Gary Schweitzer’s Health News Review website provides many useful resources to help you determine the trustworthiness of medical news. To date, it has reviewed more than 1,000 news stories concerning claims made for treatments, tests, products and procedures.
  • Sense about Science works with scientists and members of the public to equip people to make sense of science and evidence. It responds to hundreds of requests for independent advice and questions on scientific evidence each year.
  • Trust It or Trash is a tool to help you think critically about the quality of health information (including websites, handouts, booklets, etc.).
  • Understanding Health Research (UHR) is a free service created with the intention of helping people better understand health research in context. It gives clear and understandable explanations of important considerations like sampling, bias, uncertainty and replicability.

Heading Off Cancer Growth on the Cellular Level

Cancer cells are like all the cells in our body, in that they need certain basic building blocks – amino acids – in order to reproduce. There are 20 amino acids found in nature. The amino acid serine is often found in abundance in patients with certain types of breast cancer, lung cancer, and melanoma. The overproduction of this amino acid is often required for the rapid and unregulated growth characteristic of cancer.

Scientists at the Scripps Research Institute (TSRI) wondered if there was a way to take advantage of the relationship between cancer cell proliferation and serine. Amy GrayThey examined a large library of molecules -numbering 800,000 – to find an enzyme that inhibited serine production. After much research, the group found 408 contenders that could possibly work. This list was again narrowed down to a smaller set of seven, ending with one promising candidate. This molecule, 3-phosphoglycerate dehydrogenase (PHGDH), seemed to inhibit the first step in a cancer cell’s use of serine to reproduce itself.

Luke L. Lairson, assistant professor of chemistry at TSRI and principal investigator of cell biology at the California Institute for Biomedical Research remarked, “In addition to discovering an inhibitor that targets cancer metabolism, we also now have a tool to help answer interesting questions about serine metabolism.”

What does this mean for cancer patients in the future?

Discovering an enzyme that inhibits serine production means that a key process in cancer cell proliferation can be slowed down or even stopped.   Interfering with cancer cell metabolism could be a pathway to treatment. Potentially, adding the molecule PHGDH to cancer cells disturbs the basic need of cancer cells to divide and reproduce rapidly. Obviously this finding points to years of further research and drug development. But discovering this key relationship between serine over-production and a molecule that slows it down could be a model for new cancer treatments in the future.

 

References:

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3989988/

http://medicalxpress.com/news/2016-03-team-approach-curbing-cancer-cell.html