Skip to Main Content
NAU Cline Library logo

PBC Library

The NAU library website for the Phoenix Biomedical Campus Library

Overview of EBP

Definition of EBP

...the conscientious explicit and judicious use of current best evidence in making decisions about the care of individual patients. (Sackett 1996)

Optimal use of EBP involves carefully balancing: clinical expertise, the best evidence available, and patient values + circumstances.

Evidence-based practice diagram

Why do we need EBP?

Consider the following graph:

A graph showing the increase in studies on Clinical trials dot gov over time

In just 18 years' time, the number of studies available in the NIH database of clinical trials has increased by more than 23,000%

And that is only studies in Think for a moment about all of the other databases where you can find health-based information. In fact, try searching "cancer diagnosis" using many results does that deliver? On the day this was written (12/14/2018), that search query returned 842,000,000 search results! How do you think information seekers pick from all of those links?

According to a recent survey from dotHealth, 31% will opt for the very first result. Why do you think that is? 

Check out this video from WNYC to get a better idea: 

EBP helps us make sense of it all

There is an enormous amount of information out there, and it isn't going to get smaller anytime soon.

EBP can help practitioners siphon out the information that doesn't work. Doing so takes practice. We have to learn how to balance efficiency with precision. This means:

Knowing where to look for information
Weeding out unnecessary information
Successfully accessing and storing information
Analyzing the validity of information
Sharing information and putting it into practice


In EBP, we label this process: ASK --- ALIGN --- ACQUIRE --- APPRAISE --- APPLY

Please keep in mind that this is not always a linear process. Instead, learning how to get the best information using the most efficient methods takes time and is often iterative. Sometimes, you will know where you want to look for information before you know what type of keywords to use. In those cases, you'll spend time going back and forth between asking, aligning, and acquiring as you work out your research needs.

That's okay!

For now, focus on understanding why this process is important...

As you read through the following tabs, consider each of these three components. Ask yourself:

  • How do I define my own expertise, and can that change, expand, or even become more narrow?
  • How do I decide what qualifies as the "best" evidence? And how do I find it?
  • How do I ensure that my patients are treated equitably and given agency to make their own informed decisions?

i also recommend working through this checklist for conducting evidence-based research, as it can be used anytime one is looking for evidence:

Test your knowledge!

Now that you have some insight into healthcare information, and its relationship to EBP, try to answer the following questions:

  • What are the standards required to register a study in
  • Why might people opt for the first result of a Google search?

Go further:

You might also want to branch out and read through a few papers and organizational sites that define (and in some cases, problematize) EBP:


Before you start, try following along with the Guide to Forming a PICO Question.

Construct a concise clinical question.

The first step in EBP is to write a focused clinical question. That question will be translated into a database query in step two. Best practice is to define your research question and identify which types of studies or publications would be best suited to that question before searching the literature. You should also be aware of the patient, if you are seeing one, and how their needs are informing your research.

Once you've considered the foundational components of your research, the next step involves constructing a question relevant to your needs and your patient's needs. It should be framed using the PICOT acronym:

Patient / Population:

What kind of person is the patient? What is their diagnosis? What is their particular situational need?

Example: pre-adolescent children with cerebral palsy


How would you describe the specific intervention you are considering?

Example: whole body vibration

Comparison / Control:

Is there a control group? Is there are an alternative intervention? (Keep in mind: not all questions have a "comparison" component)

Example: conventional physical therapy


What is the desired outcome? Does it involve reduction, relief, or elimination of symptoms? Can it be measured with a test?

Example: increased balance

(OPTIONAL) Time / Type:

What time periods should be considered?  What study types are most likely to have the information you seek?  What clinical domain does your question fall under?

Example: randomized controlled trials

Now, we string our PICO components together:

Does whole body vibration increase balance in comparison to standard physical therapy in children with cerebral palsy?

Clinical Domains

It can be helpful to classify your question based on the clinical domain it falls under. A clinical domain refers to the type of study you wish to locate. For instance, a question involving therapy (such as whole body vibration) falls under the the clinical domain, therapy. Knowing this, you can filter your search results to return randomized controlled trials, which are the "gold standard" for studies involving therapeutic interventions.

See the examples below for more clinical domains and corresponding PICO templates:

Questions addressing the treatment of an illness or disability.

In _________ (P), how does _________ (I) compared to _________(C) affect _______(O) within _______ (T)?
In _______(P), what is the effect of _______(I) on ______(O) compared with _______(C) within ________ (T)?

In African American female adolescents with hepatitis B (P), how does acetaminophen (I) compared to ibuprofen (C) affect liver function (O)? (Time is optional).

Questions addressing the act or process of identifying or determining the nature and cause of a disease or injury through evaluation.

Are (is) _________ (I) more accurate in diagnosing ________ (P) compared with ______ (C) for _______ (O)?
In ________ (P) are/is ________(I) compared with ________(C) more accurate in diagnosing ________(O)?

Is a PKU test (I) done on two week old infants (P) more accurate in diagnosis inborn errors in metabolism (O) compared with PKU tests done at 24 hours of age (C)? Time is implied in two weeks and 24 hours old.
In middle-aged men with suspected myocardial infarction (P), are serial 12-lead ECGs (I) compared with one initial 12-lead ECG (C) more accurate in diagnosing an acute myocardial infarction (O)?

Questions on how to reduce the chance of disease by identifying and modifying risk factors and how to diagnose disease early by screening.

For ________ (P) does the use of ______ (I) reduce the future risk of ________ (O) compared with _________ (C)?

In OR nurses doing a five minute scrub (P) what are the differences in the presence and types of microbes (O) found on natural polished nails and nail beds (I) and artificial nails (C) at the time of surgery (T)?

Questions addressing the prediction of the course of a disease.

Does __________ (I) influence ________ (O) in patients who have _______ (P) over ______ (T)?
In _______ (P), how does ________ (I) compared to ________ (C) influence _________ (O) over _________ (T)?

Does telelmonitoring blood pressure (I) in urban African Americans with hypertension (P) improve blood pressure control (O) within the six months of initiation of the medication (T)?
For patients 65 years and older (P), how does the use of an influenza vaccine (I) compared to not received the vaccine (C) influence the risk of developing pneumonia (O) during flu season (T)?

Questions addressing how one experiences a phenomenon.

How do ________ (P) diagnosed with _______ (I) perceive ______ (O) during _____ (T)?

How do pregnant women (P) newly diagnosed with diabetes (I) perceive reporting their blood sugar levels (O) to their healthcare providers during their pregnancy and six weeks postpartum (T)?
How do 20 something men (P) with a diagnosis

Questions around how to select treatments to offer our patients that do more good than harm and that are worth the efforts and costs of using them.

In _______(P), what is the effect of _______(I) on ______(O) compared with _______(C) within ________ (T)?

What is the duration of recovery (O) for patients with total hip replacement (P) who developed a post-operative infection (I) as opposed to those who did not (C) within the first six weeks of recovery (T)?

Questions addressing the causes or origin of disease, the factors that produce or predispose toward a certain disease or disorder.

Are ____ (P) who have _______ (I) at ___ (Increased/decreased) risk for/of_______ (O) compared with ______ (P) with/without ______ (C) over _____ (T)?

Are ______(P) who have ______(I) compared with those without _______(C) at ________ risk for/of _______ (O) over ________(T)?

Are kids (P) who have obese adoptive parents (I) at Increased risk for obesity (O) compared with kids (P) without obese adoptive parents (C) during the ages of five and 18 (T)?
Are 30- to 50- year old women (P) who have high blood pressure (I) compared with those without high blood pressure (C)at increased risk for an acute myocardial infarction (O) during the first year after hysterectomy (T)?

Further reading


Levels of Evidence Pyramid | Click Here


Types of Study Design:


Review-type papers summarize, analyze, and synthesize published research. The most common type is the systematic review, which involves an exhaustive literature search followed by unbiased, expert appraisal of research evidence. In addition to systematic reviews, there are many other types of review papers, some of which exhibit higher levels of evidence than others. The most common are:

  • Systematic review: an exhaustive literature search followed by unbiased, expert appraisal of research evidence.
  • Meta-analysis: a systematic review with an additional quantitative component, involving statistical combination of study results to provide a more accurate understanding of the research.
  • Critical review: an extensive literature search followed by theoretical and critical analysis of, not only of findings, but of concepts and systems involved in research.
  • Mapping review: a systematic literature search culminating in a "map" of findings, which categorizes research into groups to be reviewed in further study.
  • Mixed methods review: a systematic review which allows for appraisals of mixed study types, typically quantitative and qualitative.
  • Scoping review: a preliminary review designed to asses the scope of the research available in a particular field of study. It is designed to set parameters for future systematic reviews.

These descriptions are derived from Grant & Booth, 2009.

Experimental studies involve the manipulation of populations and treatments in order to reduce opportunities for bias. This involves the use of controls, randomization, and blinding. Typically, experimental studies exhibit higher levels of evidence by virtue of their providing empirical, quantitative data. Different types include:

  • Double-blind Randomized Controlled Trials: prospective study that involves collection of data in a clinical setting. Includes double blinding, in which neither the patient-participants nor the researchers know which groups or patients are receiving treatment. Double-blinding is more common in pharmacological studies.
  • Randomized Controlled Trial (RCT): prospective study that involves collection of data in a clinical setting. Includes randomization, in which members in the control and variable groups are selected and placed at random without influence by the research team. RCTs are the premiere study type for researching therapeutic interventions.
  • Randomized Cross-Over Clinical Trial: prospective study that involves collection of data in a clinical setting. Involves the "crossing-over" of participants from one treatment to another after a determined period of time. This type of study is prone to bias, as multiple treatments can overlap.

Observational studies typically provide lower levels of evidence than experimental studies, because they do not involve strict manipulation of controls. However, they may offer valuable insight, especially if the application of control factors would lead to undue harm for the patient/s. They may also serve as preliminary studies with the goal of conducting an experimental study later in the future.

  • Cohort study: a prospective study that analyzes data collected from groups designated for observation. These studies are prone to bias because there are limited control factors involved in selection, treatment, or follow-up.
  • Case-control study:a retrospective study that is useful for epidemiological and diagnostic observation. Case-controls measure the similarities and differences between "cohorts" of patients with a common outcome alongside another, similar cohort expressing different outcomes. These types of studies require follow-up and recall, and as such, are prone to biased interpretations and memory.
  • Case series and case reports: retrospective, anecdotal studies that rely upon descriptive interpretations of outcomes based on a series(a group of similar "cases"), or a single "case". These are not necessarily generalizable, as they focus on single patients or small groups of patients, prone to bias. Researchers may also fall victim to over representation of "absurd" or "bizarre" cases

Question Type & Study Design

Some study designs are better suited to answer certain types of questions. Identifying the clinical domain your question falls under can help determine what study designs to look for in order to find the best, most suitable evidence. Systematic Reviews and Meta-Analyses are always preferred.

Clinical Domain
Preferred Study Design (after systematic reviews and meta-analyses)
Cost-Effectiveness Economic Analysis > Socioanalysis
Diagnosis Cross-Sectional Studies
Etiology or Prognosis Cohort > Case-Control > Case Series
Prevention Randomized Controlled Trial > Randomized Crossover Trial > Controlled Trial > Cohort > Case-Control
Quality of Life Qualitative Studies
Therapy Randomized Controlled Trial > Randomized Crossover Trial > Controlled Trial or Quasi-Experimental Studies

For example:

Clinical question: In adults with total hip replacements, how effective is pain medication compared to aerobic stretching in controlling post operative pain during the perioperative and recovery time?

Clinical domain: Therapy

Preferred study designs: Systematic Reviews/Meta-analysis | Randomized Controlled Trials | Conrolled Trials

Background information

Foreground vs. Background information

Before proceeding, open the Step 3: Aqcuire worksheet by clicking here.

Foreground information is the complex, precise information often needed in order to answer a PICO question.  Before searching for foreground information, however, make sure you have a good understanding of each of the PICO concepts.  

Sometimes, you will find that you need background information before you ever even begin putting together your PICO question. That's okay! This entire process is iterative, which means repeating steps is sometimes necessary. 

Put simply, background information is the general health information typically available to the public. That doesn't mean it's easy to follow, but it should be less robust, and more summative than your average peer-reviewed study.

Practice guidelines provide one useful source for background information. For the most part, they are short, designed to be understood, and closely reviewed by professionals. To get an idea what a practice guideline looks like, check out your professional organization's website or look through this sample for working with transgender patients from the World Professional Association for Transgender Health: WPATH Standards of Care.

Test your knowledge!

Before moving on, answer the following questions on your Google Form:

  • How might you assess whether you need background information or not?
  • What are some important components of good background information?
  • In what situations is having background information especially vital to patient care?

Public resources for background information:

A note about appraised resources

For practitioners who need immediate, credible information, appraised resources are highly preferred. BMJ Best Practices and UpToDate are two specific resources that can be leveraged at or around point-of-care. Keep in mind that PubMed also indexes appraised information; however, it is more difficult to parse due to PubMed's scope.

From a point-of-care perspective, we can place evidence into five categories:

Resource Types

Clinical systems that provide decision-making support. EHRs are a common example.


Textbook-like resources that summarize evidence, often intended to aid clinical decision making.

Resource examples

Databases: Cochrane Answers | DSM V TR | UpToDate


Summaries of expertly assessed studies and systematic reviews with advice for application.

Resource examples

Databases: ACP Journal Club | TRIP (open access)


Systematic reviews and clinical practice guidelines that critically assess the state of the research.

Resource examples

Databases: Cochrane Library of Systematic Reviews | PubMed Clinical Queries (Open Access)


Individual clinical studies (ie. randomized controlled trials) in peer-reviewed publications.

Resource examples


When searching for best evidence in clinical decision making, start at the the top of the hierarchy with systems and work your way down to synopsessyntheses, and studies. Opt for databases like UpToDate, which present information quickly, clearly, and efficiently. 

Remember that point-of-care tools are meant to save time in a fast-paced clinical environment. Even so, it is important to maintain proficiency in larger databases such as MEDLINE, in order to locate new research studies as they are published.

Mobile Applications can also be very helpful at the point-of-care, however, use caution since nearly anyone can develop an app! You must carefully evaluate the information & source. STAT!Ref is a library resource with a mobile app that is recommended to NAU students, as it provides access to ACP Smart Medicine, 5 Minute Clinical Consult, Red Book, and more.

6S Levels of Evidence Pyramid | Click Here


Acquiring evidence

Finding foreground information

Following from the levels of evidence pyramid (see above), information resources can be collapsed into three broad categories (with corresponding sub-categories): 1) summaries and guidelines; 2) synopses of syntheses and syntheses; 3) synopses of studies and individual studies.

When we commit to a research question and literature search, we want to get as near to the top of the pyramid as we possibly can. Depending upon the research available, we work down from there.

Looking at the pyramid, answer the following questions in your Google Form:

  • Why do we want to start at the top, and work our way down?
  • Why do we use a pyramid rather than a square? Why is the top smaller than the bottom?
  • What does it mean if you don't find any research published for your clinical question?

(NOTE: Don't worry about picking a resource just yet. You might want to look through summaries, but otherwise, move on to the next step.)

EBP resources



summarizes the evidence from a topical level, can be used as practice guidelines

Resource examples

Open access: Natural Medicine | ECRI Guidelines Trust | Professional Organizations

Databases: BMJ Best Practices | DSM V TR | Rehabilitation Reference CenterUpToDate

Synopses / Syntheses

(Review-type papers)


Summaries of high-level evidence concerning a specific research topic

Resource examples

Open access: Annals of Internal Medicine | NARIC PLOS MedicineEpistemonikos

Databases: CINAHL | Cochrane | OTSeeker | PubMed (MEDLINE, PMC, NCBI) | PEDro

Synopses of Studies / Studies

(Clinical trials, case reports, etc.)    


Primary original research that has not been pre-appraised

Resource examples

Open access: Google Scholar | PubMed Central | PLOS Medicine | TRIP

Databases: CINAHL | OvidMD | PubMed (MEDLINE, PMC, NCBI)

Click here to review NAU's complete journal listings.

Complete list of NAU health science databases

Conducting a search

After constructing your PICO question and finding the resource that best suits your needs, it's time to build and conduct a search. Listed below are strategies and tools that will make it easier to find information that is most applicable to you and your patients.

Primary concepts

Your PICO question should help you identify specific search terms that you can use to search the databases. This, combined with what you learned while searching background information, will help you form the "backbone" of your search question.

Take a few moments to list your PICO "primary" terms following the table below:

Primary Search Terms
P - Patient / Population

ex. children with cerebral palsy

Primary Search Terms                                                                                        

ex. "cerebral palsy" "children" "pediatrics"

I - Intervention

ex. whole body vibration

Primary Search Terms

ex. "whole body vibration" "vibration"

C - Control / Comparison

ex. conventional therapy

Primary Search Terms

ex. "stretching" "strength training"

O - Outcome

ex. balance

Primary Search Terms

ex. "balance" "gait"


More than likely, PubMed is the most common place you'll go for your research. It's an an enormous platform, and houses the MEDLINE, PMC, MeSH, and NCBI Bookshelf databases. It also has its own citation management tool, which you should access before using any other part of the service. 

This tool can be accessed as soon as you've set up a MyNCBI account. Doing so is fairly simple. Simply click Sign in to NCBI from the top right corner of the PubMed homepage. From there, you need only create a new account.

For more help, check out this walkthrough video from VCU Libraries on setting up an account: 


It can be challenging to find exactly what you're looking for when searching databases. For that reason, it's valuable to adhere to the controlled vocabulary in the different databases you search. One of the most well-regarded is MeSH, which stands for "Medical Subject Headings". MeSH terms are an important part of the search experience in PubMed, one of the largest research platforms in the world. 

MeSH terms are created by the National Library of Medicine. Using them will help you formulate keywords for your search. Follow this link to go to the MeSH database:

Consider the following questions in your Google Form (Step 3: Acquire):

  • Why is it important to at least browse MeSH terms before conducting a full-on search?
  • Do you think that MeSH terms are valuable in databases outside PubMed? Why or why not?
  • At what point would you want to consider a MeSH search?

For help with MeSH, see this short tutorial:

PubMed Clinical Queries

Now that you've put together your primary search concepts using MeSH, let's conduct a search in PubMed's Clinical Queries tool.

The clinical queries tool is designed to provide more efficient access to studies and systematic reviews which correlate with a specific clinical domain. It is not exhaustive! Rather, it is designed to give you more immediate insight into the amount of studies available in your field of study.

As you think about your clinical queries search keep in mind the following:

  • When you begin formulating a search strategy, aim for a search with high recall rather than precision. In many cases, this will generate a larger number of results than is necessary; however, it is sometimes easier to narrow your search over time instead of doing so from the outset.
  • Avoid using too many search concepts; concepts are the primary terms that align with your search needs. At the same time, a variety of synonymous search terms should be combined in conjunction with each concept.
    • IE: Intervention (concept) = hip strengthening | "hip flexor" OR "hip extension" OR "hip exercise" OR "hip stretch" OR ""hip stretches" OR hip stretching" OR "foam roll" OR "foam roller" OR "foam rolling" (search terms)
  • Familiarize yourself with Boolean operators AND, OR & NOT, and apply them within your search string, both between concepts and between search terms.
    • AND separates concepts | "cerebral palsy" AND "whole body vibration" narrows your search, such that you receive papers about both cerebral palsy and vibration.
    • OR separates search terms | "children" OR "pediatrics" broadens your search to include both of these key words.
    • NOT excludes unwanted concepts | "stroke" NOT ("heat stroke" OR "breast stroke") makes it more likely your search results will be limited to cerebrovascular accidents.
  • Two other rules to keep in mind involve parentheses and quotation marks:
    • Parentheses include phrases in your search, but will not limit to exact phrasing.
      • IE: (disorders of sexual development) will return any combination of those keywords.
    • Quotation marks limit search results to exact phrasing.
      • IE: "disorders of sexual development" will return this exact phrase.

A well-constructed initial search question:

"cerebral palsy" AND "children" AND (whole body vibration)

For more help building these strategies into a search string, check out the University of Arizona search strategy builder.

Advanced Search Strategies

Read through this entire section before answering the questions in your Google Form (Step 3: Acquire)

Creating a Zotero account

Once you have compiled at least one promising study or review, you should consider a citation management platform like Zotero. This will help you keep your research organized, which is particularly helpful if you're writing a literature review or annotated bibliography!

Getting started with Zoteroo is easy (although it does require a NAU login ID). Simply navigate to: and create an account from there. 

If you need help with Zotero, visit the Library's Research Support Services page, and click on Citation Management Apps.

Journals in NCBI Databases

Sometimes, it is helpful to initiate a journal search to find information. This is particularly useful when you:

  • know which journals are most likely to publish studies similar to your topic;
  • want to identify the type of studies being published in a specific journal;
  • want to explore the "impact" of a specific journal within the discipline.

Journals can be searched by:

  • topic (ie: developmental disabilities)
  • journal title or abbreviation (ie: title = Transgender Health | abbreviation = Transgend Health)
  • ISSN (ie: 2380-193X)

If planning to publish a study, it is very important to identify a journal that focuses on the type of research you want to present. In these instances, authors should also check out a journal's home page, which can be found from the journal listing in PubMed.

See this link to view the complete listing for Transgender Health.

PubMed Central (PMC)

PubMed Central is the NLM's open-access archive of biomedical literature. PMC indexes published papers, manuscripts, retractions, and other forms of gray literature (conference proceedings, dissertations, etc.). It is an enormous resource, and particularly useful when performing systematic analyses of research, which often require scouring gray literature as well as published studies.

See this link to access PMC:

Document Delivery Services

When you find studies that you can't access through NAU, that doesn't mean you can't read them! Instead, you'll need to use the library's Document Delivery Services tool to secure the paper from another source.

DDS is easy to use once you've set it up. Typically, it only takes 1 - 2 days to receive a requested study, which will be sent as a .PDF for you to save and use later.

To learn more about DDS, see this link:

Additional health science databases

Aside from PubMed, NAU provides access to a range of health science databases. See the table below (replicated from Step 3a) for specific databases that align with various study types.

In order to further narrow and refine your search, remember to:

  • Use search filters to limit results, particularly when delineating between reviews, clinical trials, and/or other paper types
  • Look for information that is timely when assessing for current trends
  • Do not limit your search to Full Text, as you can request articles not owned by NAU through Document Delivery Services
  • Be prepared to look in more than one database, and never expect that one database is a one-stop shop for information
EBP resources



summarizes the evidence from a topical level, can be used as practice guidelines

Resource examples

Open access: Natural Medicine | ECRI Guidelines Trust | Professional Organizations

Databases: BMJ Best Practices | DSM V TR | Rehabilitation Reference Center | UpToDate

Synopses / Syntheses

(Review-type papers)


Summaries of high-level evidence concerning a specific research topic

Resource examples

Open access: Annals of Internal Medicine | NARIC PLOS Medicine | Epistemonikos

Databases: CINAHL Cochrane | OTSeeker PubMed (MEDLINE, PMC, NCBI) | PEDro

Synopses of Studies / Studies

(Clinical trials, case reports, etc.)    


Primary original research that has not been pre-appraised

Resource examples

Open access: Google Scholar | PubMed Central | PLOS Medicine | TRIP

Databases: CINAHL OvidMD | PubMed (MEDLINE, PMC, NCBI)

See below for a list of select NAU Health Science databases:

Evaluating your findings

Once you have collected your findings, they must be properly assessed. This page will assist you as you go through that process.

Before you begin, you may use the following form to help you annotate and synthesize your research findings:

Alternatively, a shorter fillable form can be accessed from this link:

Grading scales

You may find it necessary to use a grading scale when assessing a study. Try using one of the following:

Assessing an abstract

Before you ever get into a paper, it is possible to obtain some quality assessment data from an abstract.

While not all abstracts present with the same information, they are increasingly more uniform, and you should at least be able to do the following:

  • Ensure that the paper was not published in a predatory journal
    • FYI: one easy way to do this is check and see if the paper has been indexed in MEDLINE using PubMed!
  • Look for any notifications that the paper has been retracted
  • For systematic reviews: look for evidence of registration in PROSPERO and adherence to PRISMA standards
    • NOTE: you may find that neither of these qualities are present in older systematic reviews
  • Check the date of publication to ensure that the research is still applicable
  • Ensure that the research can be generalized to other research, including your own

Validity and reliability


To find out if a finding is valid, ask:

  • What was the study design, and is it appropriate for this clinical domain?
  • Did experimental and control groups begin the study with a similar prognosis?
  • Did experimental and control groups retain a similar prognosis after the study started?
  • What are the results? Specifically, how large and precise was the treatment effect?
  • How prevalent was bias? (NOTE: see below for more information regarding bias)
  • Are results clinically significant in addition to being statistically significant?

To find out if a finding is applicable, ask:

  • Were study patients similar to my patient?
  • Were all patient-important outcomes considered?
  • Are the likely benefits worth the potential harms and costs?
  • Does the intervention impose upon my patient's values and circumstances?
  • Does this resource help me answer a PICO(T) question well enough to analyze and write about it myself?
  • Does this study have wider significance to my field?
  • Can this study be improved upon by further study?

Assessing for bias

Particularly for those conducting reviews (literature, scoping, systematic, etc.), it is important to assess for risk of bias. The following bullets, attributed to the Cochrane Handbook for Systematic Reviews, can help you ascertain the prevalence of bias in a given study:

  • A bias is a systematic error, or deviation from the truth, in results or inferences. Biases can operate in either direction: different biases can lead to underestimation or overestimation of the true intervention effect. Biases can vary in magnitude: some are small (and trivial compared with the observed effect) and some are substantial (so that an apparent finding may be entirely due to bias). Even a particular source of bias may vary in direction: bias due to a particular design flaw (e.g. lack of allocation concealment) may lead to underestimation of an effect in one study but overestimation in another study. It is usually impossible to know to what extent biases have affected the results of a particular study, although there is good empirical evidence that particular flaws in the design, conduct and analysis of randomized clinical trials lead to bias.
  • Bias should not be confused with imprecision. Bias refers to systematic error, meaning that multiple replications of the same study would reach the wrong answer on average. Imprecision refers to random error, meaning that multiple replications of the same study will produce different effect estimates because of sampling variation even if they would give the right answer on average. The results of smaller studies are subject to greater sampling variation and hence are less precise. Imprecision is reflected in the confidence interval around the intervention effect estimate from each study and in the weight given to the results of each study in a meta-analysis. More precise results are given more weight.
  • A study may be performed to the highest possible standards yet still have an important risk of bias. For example, in many situations it is impractical or impossible to blind participants or study personnel to intervention group. It is inappropriately judgmental to describe all such studies as of ‘low quality’, but that does not mean they are free of bias resulting from knowledge of intervention status.
Types of bias
Selection bias

systematic differences between baseline characteristics of the groups that are compared

Performance bias

Systematic differences between groups in the care that is provided, or in exposure to factors other than the interventions of interest.

Detection bias                 

Systematic differences between groups in how outcomes are determined.

Attrition bias

Systematic differences between groups in withdrawals from a study.

Reporting bias

Systematic differences between reported and unreported findings.


The GRADE handbook breaks down the official grading rating scaled used for Cochrane reviews. The Handbook itself is lengthy, but it's easy to navigate across sections according to your needs. For appraisal, focus specifically on sections 4 and 5, Summarizing the evidence and Quality of evidence

See the image below for a typical journey using the GRADE approach. Note that it begins with framing the PICO question, and from there, goes through the literature search into appraisal, grading, and recommendations for guidelines.

Access the GRADE Handbook available through Cochrane. 

Cochrane's GRADE Framework for describing the certainty of evidence and justifying downgrading or upgrading:

Domains for assessing certainty of evidence by outcome

Results section

Examples of reasons for lowering or increasing the certainty of evidence

Risk of bias

Describe the risk of bias based on the criteria used in the risk-of-bias table.

Downgraded because of 10 randomized trials, five did not blind patients and caretakers.


Describe the degree of inconsistency by outcome using one or more indicators (e.g. Iand P value), confidence interval overlap, difference in point estimate, between-study variance.

Not downgraded because the proportion of the variability in effect estimates that is due to true heterogeneity rather than chance is not important (I= 0%).


Describe if the majority of studies address the PICO – were they similar to the question posed?

Downgraded because the included studies were restricted to patients with advanced cancer.


Describe the number of events, and width of the confidence intervals.

The confidence intervals for the effect on mortality are consistent with both an appreciable benefit and appreciable harm and we lowered the certainty.

Publication bias

Describe the possible degree of publication bias.

1. The funnel plot of 14 randomized trials indicated that there were several small studies that showed a small positive effect, but small studies that showed no effect or harm may have been unpublished. The certainty of the evidence was lowered.

2. There are only three small positive studies, it appears that studies showing no effect or harm have not been published. There also is for-profit interest in the intervention. The certainty of the evidence was lowered.

Large effects (upgrading)

Describe the magnitude of the effect and the widths of the associate confidence intervals.

Upgraded because the RR is large: 0.3 (95% CI 0.2 to 0.4), with a sufficient number of events to be precise.


Dose response (upgrading)

The studies show a clear relation with increases in the outcome of an outcome (e.g. lung cancer) with higher exposure levels.

Upgraded because the dose-response relation shows a relative risk increase of 10% in never smokers, 15% in smokers of 10 pack years and 20% in smokers of 15 pack years.

Opposing plausible residual bias and confounding (upgrading)

Describe which opposing plausible biases and confounders may have not been considered.

The estimate of effect is not controlled for the following possible confounders: smoking, degree of education, but the distribution of these factors in the studies is likely to lead to an under-estimate of the true effect. The certainty of the evidence was increased.

Additional evaluation resources

EBP/EBM databases


eBooks from the collection