In this post I am going to walk you through how to read and better understand cancer-related research studies. As a veterinarian, I have a lot of background in reading research and take it for granted, but for people without a medical background, these studies can be overwhelming and impossible to understand. It occurred to me that I am referencing a lot of scientific studies in my blog posts, but many of you may not know how to read these studies and understand the results and all these statistics! I hope this simple guide will help clear things up for you.
WHY YOU SHOULD LEARN HOW TO READ SCIENTIFIC STUDIES
For a cancer patient or a caregiver, reading and understanding research studies is very important for many reasons. It will help you become more informed about your disease, the treatment options available and the latest advancements in care. It also gives you an ability to filter out the real data from the hype.
I hate to say this, but many of these studies are often written in such a way as to make a particular drug or treatment sound better than it really is. More on this later. This is why it is so important to learn some basic medical terminology and statistics, which I will detail in this post.
With a better understanding of how to interpret results and how the research is conducted, you can ask insightful questions, communicate concerns about your treatments to your healthcare team and make better decisions when it comes to your care and what is best for you.
So without further ado, let’s get into the basics of scientific studies.
DIFFERENT TYPES OF STUDIES
There are many different types of scientific studies. Here I will tell you about some of the main ones used in cancer research.
BASIC RESEARCH
This type of research occurs in the laboratory. Scientists are studying cancer at the cellular and molecular level using cell cultures and animal models. The goal is to understand how cancer starts and grows and to develop new targeted treatments.
CLINICAL TRIALS
Once scientists have developed a new treatment that seems to be effective and safe in the lab, they will move to a clinical trial. Clinical trials test new drugs, new approaches to surgery or radiation or combinations of treatments and targeted therapies in human patients. Clinical trials are done in phases, (phase 1, phase 2 and phase 3). A new treatment needs to pass through all these rigorous testing phases to determine if it is safe and effective before it will be approved for use by the Food and Drug Administration in the United States.
RANDOMIZED CONTROL TRIAL (RCT)
This type of study is considered the gold standard of all cancer studies. A RCT is a clinical trial where participants are randomly assigned to different groups, also called arms. The experimental arm will receive the new treatment or intervention and the control arm will receive either a standard treatment or a placebo. This type of study has the least bias and gives the strongest evidence for the effect of a particular treatment on outcome. If a new drug or treatment has been tested using a RCT, you can be assured that the results are very accurate. Here is something else to know. When a RCT says it is double blinded, this means that neither the participants nor the researchers know which treatment or intervention each participant is receiving. This ensures that the results are more reliable and not influenced by subjective factors like the placebo effect. Double blinded RCTs are the best and most reliable type of study.
RETROSPECTIVE STUDY
A retrospective study is where researchers look back at data from the past. This type of study uses medical records and other historical data from previous studies. The goal is to identify potential associations between exposures and risk factors that can lead to disease among other things. These studies, while helpful, are not always very accurate.
OBSERVATIONAL STUDY
This is a prospective study where researchers follow groups of people over a period of time. They may include healthy people, cancer patients or people who are at high risk for developing cancer. These studies are mainly to determine risk factors for cancer, treatment outcomes, screening practices and more. No attempt is made to affect the outcome. This type of research is the least accurate of them all for many reasons I won’t go into here. When you read observational studies, have a healthy sense of skepticism and look for more accurate studies before making any judgements.
COHORT STUDY
A cohort study is where a group of people sharing a common characteristic are followed over time to determine how many develop a particular health outcome based on the data set the researchers are looking at.
META-ANALYSIS
A meta-analysis is a statistical method used to combine the results of several studies (often observational and cohort studies) to identify overall trends. They use very specific criteria for selecting the studies to include in the analysis. The researchers quantitatively combine data from multiple studies, which can provide a stronger numerical estimate of the effect of a treatment or intervention than any single study can provide. Meta-analyses are often used when there is conflicting or insufficient evidence from individual studies. These are helpful when it comes to cancer research and more accurate than an observational study, but not as accurate as a RCT.
KEY SECTIONS OF RESEARCH STUDIES
Most scientific studies follow the same basic structure. They all have a title, abstract, introduction, methods, results and discussion sections. The paper will also list the name of the journal, the date it was published and the authors and their affiliations.
IMPORTANT TERMINOLOGY USED IN CANCER RESEARCH STUDIES
Here is a simple, alphabetical glossary of terminology that will help you understand statistics. Make sure to pay particular attention to the relative risk versus absolute risk part of this glossary.
CLINICAL BENEFIT RATE (CBR or CB)
This is the total number (or percentage) of patients who achieved a complete response, partial response, or had stable disease for 6 months or more. Basically, the number of patients who had any benefit from the treatment or intervention.
COMPLETE RESPONDERS (CR)
This is the number of patients whose tumors disappeared after the intervention. This is also called complete remission.
CONFIDENCE INTERVAL (CI)
In statistics, confidence is another way to describe probability. So a study gives a confidence interval (CI) as a range with an upper and lower limit that conveys how precise the measurement is. Researchers often use a 95% confidence level, meaning they are 95% confident that the true value lies within the calculated range.
For example, a CI in cancer research may look like this. A study found that the 5-year survival rate for patients with stage 3 endometrial cancer was 60% with a 95% confidence interval of 55%-65%. This means that there is a 95% chance that the true survival rate for this patient population lies between 55%-65%. If the study were repeated multiple times with different groups, 95% of the calculated survival rates would fall within the range of 55%-65%.
CONTROL GROUP
This is the group of patients that is compared to the experimental group. This might be a group of healthy people who are similar in age and other demographics. Or, this may be a group of patients with the same disease receiving standard therapy.
EXPERIMENTAL GROUP
This is the group of patients that is getting the treatment or intervention being tested. They are being compared to the control group as mentioned above.
HAZARD RATIO (HR)
The HR is very important and is a way to summarize the difference between two survival curves. These curves, one for the control and one for the experimental group are often shown on a graph. The HR is the numerical representation comparing the two curves as the rate of survival changes over time.
- HR = 1 – there is no difference between the two groups in terms of survival.
- HR > 1 – means the experimental treatment is less effective than the control treatment and therefore does not contribute to survival. The higher the HR value, the less effective the experimental treatment is in regard to survival.
- HR < 1 – means survival is greater with the experimental treatment than with the control. The lower the HR, the more survival benefit from the experimental therapy.
IN VITRO AND IN VIVO
In vitro is latin for “in glass”. This means experiments conducted outside of a living organism, typically cell cultures in Petri dishes or test tubes. In vivo means “within the living”. These experiments are conducted within a living organism such as a human or animal.
MEAN
The mean is the average of the group as a whole. For example, let’s say a clinical trial reports a mean survival of 18 months for stage 4 endometrial cancer patients with a particular treatment. The researchers would add up all the survival times of the patients with stage 4 endometrial cancer receiving the treatment and divide by the total number of these patients giving the average survival for the group.
MEDIAN
The median is the value in the middle of a data set. In the example above, this means that half of the patients in the study lived for 18 months or longer, while the other half lived less than 18 months. In general, the median is more accurate than the mean.
OBJECTIVE RESPONSE RATE (ORR)
The ORR is the percentage of patients who had a complete response (tumor disappeared) or partial response (tumor decreased in size) to a cancer treatment. It is measured over a period of time by monitoring a tumor’s size with scans. The ORR is based on RECIST criteria, which stands for Response Evaluation Criteria in Solid Tumors. The ORR is the most common endpoint used in clinical trials to obtain FDA approval of cancer drugs for solid tumors. Unfortunately, this number is often overestimated in these clinical trials. Read More
OVERALL SURVIVAL (OS)
Overall survival measures the time from diagnosis or treatment initiation until a patient dies from any cause, not just cancer. While considered the gold standard in cancer research, it isn’t always very accurate. It can be affected by many things including, other health conditions, lengthy follow-up periods, patients lost to follow-up or patients who drop out of a study for various reasons. It doesn’t factor in quality of life or salvage therapies when patients have progression of their disease either.
P-VALUE
This is an important value, but is hard to understand. What you really need to know is it is used to prove statistical significance of an experiment. A p-value of < or = to 0.05 is considered statistically significant and proof that the treatment or experiment had an effect. Anything above 0.05 is not statistically significant. The lower the p-value (i.e. 0.001), the more convincing the result. Even though a test may not be considered statistically significant based on the p-value, it shouldn’t be completely discounted. For example, if a study had too few participants, it may not show statistically significant results, but this doesn’t mean it doesn’t have clinical importance.
PROGRESSION-FREE SURVIVAL (PFS)
This refers to the length of time during and after cancer treatment in days, months or years that a patient lives before the disease progresses or recurs. This is considered a measure of disease control and stabilization.
RELATIVE RISK REDUCTION VS. ABSOLUTE RISK REDUCTION
Risk is the chance of something happening and there are different ways of describing risk, which can profoundly affect how we perceive it. Doctors, drug reps, the media and pharmaceutical companies will often quote the relative risk reduction, which exaggerates the study results, making a particular drug or treatment sound more impressive than it actually is. They do this for monetary benefits. This is why it’s so important to be aware of this so you can get the real numbers and make better decisions.
Here is an example.
Have you ever seen a headline like this?
“Lipitor (statin) lowers the risk of having a heart attack by 36% in patients with hypertension.”
This sounds pretty impressive right? This is a statement of relative risk reduction, but it doesn’t tell you anything about the actual likelihood of this happening at all. You need to know what the absolute risk reduction is.
Here are what the actual numbers look like.
For every 100 patients, 1.9% taking Lipitor had a heart attack vs. 3.0% of patients taking a placebo. The relative risk reduction is 36%. This is how they got that number: First calculate the actual numbers, not percentages, which looks like this. (3/100 = .03) and (1.9/100 = .019); the control group (0.03) – the experimental group (0.019) divided by the control group (0.03) = relative risk reduction.
In this example, it looks like this: .03 – .019 = .011 divided by .03 = .36. Multiply .36 x 100 = 36%. I know this is a lot of math, it hurts my brain too, but this is how they came up with 36% relative risk reduction.
The absolute risk reduction is this, 3.0% – 1.9% = 1.1%. This means of those 100 people, only 1 person with hypertension was actually helped by taking the statin, Lipitor.
If you really want to impress your doctor, ask for this statistic. NUMBER NEEDED TO TREAT (NNT). This is another way of really getting to the truth about how effective a treatment is. It tells you the average number of people who would need to be treated for one person to benefit. In the example above, take the 100 patients divided by the absolute risk of 1.1% = 90.9. In other words, 91 people with hypertension would have to take the statin before 1 person benefits. That certainly sounds less impressive than reducing your heart attack risk by 36% by taking the statin.
STABLE DISEASE
This is the number of patients whose tumors did not grow or shrink within a defined period of time. For example, a clinical trial may specify, stable disease for 6 months or more.
STATISTICALLY SIGNIFICANT
This term is used to determine whether the treatment or intervention is the cause of a statistical difference in outcomes between the two studied groups or if the outcomes could have differed just by chance. This is where the p-value comes in and this term takes into account many different variables. It’s important to know that just because a treatment does not meet the criteria for statistical significance, does not automatically mean it doesn’t have some clinical importance for certain patients.
TWO MEASURES OF THE ACCURACY OF A CANCER SCREENING TEST
Many of us have cancer screening tests, for example the Signatera test is a common one. These screening tests come with two measures of their accuracy called sensitivity and specificity.
SENSITIVITY
This is the ability of a test to correctly identify patients who have a disease (true positives). A high sensitivity indicates that the test can identify most people with the disease, minimizing the number of false negatives.
SPECIFICITY
This measurement is the ability of a test to correctly identify patients who do not have a disease (true negatives). A high specificity indicates the test can correctly identify most individuals who do not have the disease, minimizing the number of false positives.
Here is an example. The Signatera test generally demonstrates a high specificity (99.9%), which means it is highly accurate at identifying the absence of cancer. The sensitivity is lower and variable based on tumor burden. This means it may be harder to detect if cancer is present if the tumor burden is very small.
THINGS TO CONSIDER WHEN READING STUDIES
- Where was the study published? Look for studies that are published in reputable scientific journals where there are rigorous peer review processes that take place before it can be published.
- Who funded the research? Was it through a large medical establishment, the government or a pharmaceutical company who had a vested interest in the drug or treatment? This can often influence the reporting of the results. Cancer research is big business so keep this in mind when looking at studies.
- How advanced is the research? Headlines can often over-inflate the science. Here is an example I came across. In 2008, a national newspaper ran a headline “New Hope of Cure for All Cancers!” The researchers had not discovered a new drug or treatment. They had discovered the molecular structure of telomerase, an important cancer protein, in a species of tiny beetle! A far cry from a cure for cancer. Read the studies to get the real information behind the story.
- Many studies will tell you the relative risk reduction. Look at the actual numbers. You want to know what the absolute risk reduction is.
- Lastly, be careful when reading mainstream media reports and information from the big name cancer hospitals. We know they don’t always have information beyond the typical standard of care protocols. Most of them want to discourage you from doing anything outside of the box. Keep digging and you will find newer and better information from naturopathic doctors and integrative clinics who really care about your health and not just profits.
I hope this post will help you when you read studies and that you have learned a lot.
When my heart is overwhelmed, lead me to the rock that is higher than I. ~Psalm 61:2