"When it comes to our science, we have to be vigilant in our pursuit of excellence; we have to set a high standard and we just can not accept anything else."

~ Kenneth Flowers, 1994 Nathalie Barr Lecture


Critically Appraising Research

Evidence Based Medicine and Critical Appraisal Topics

Evidence Based Medicine

The Centre for Evidence Based Medicine (CEBM) defines Evidence-Based Medicine (EBM) as the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. Healthcare providers should use both individual clinical expertise and the best available external evidence collectively. Without clinical expertise, practices risk becoming inundated by evidence. Even excellent external evidence may be inappropriate for an individual patient. Without current best evidence, practices risk becoming rapidly out of date. There is no restriction to randomized trials and meta-analyses in best EBM. The process involves tracking down the best external evidence to answer our clinical questions.

Critical Appraisal Topics

The CEBM (CEBM, 2009) defines Critical Appraisal Topic (CAT) as a short summary of an article from the literature, created to answer a specific clinical question while testing evidence for validity, clinical relevance, and applicability. The following attributes are also considered:

  • Asking Focused Questions: creating an answerable question
  • Finding the Evidence: systematically retrieving best evidence available
  • Evaluating Performance: assessing evidence-based decisions
  • Making a Decision: applying results in practice

The CAT is a powerful tool for best EBM. As therapists in the revolutionary healthcare reform, this tool will be instrumental to properly support and justify clinical decisions in practice.

CAT Objectives

  1. Provide resources that facilitate the process of conceptualizing and implementing a research study.
  1. Introduce and define strategies for locating information pertaining to the question.

EXAMPLE:  Justification (for CAT)

  • Why use splints for patients with CMC joint arthritis?
  • What outcome are we seeking?
Clinical Question or PICO or PIO (MeSH terms)
Population patients with CMC joint arthritis (osteoarthritis AND thumb)
Intervention splinting (orthosis)
Outcome decrease pain, improve function (therapy)
  • Do patients with CMC joint arthritis who use thumb spica splints (versus those who use no splinting) have decreased pain and improved function?

Additional Resources

May include but are not limited to the following:

  • E-books
  • Online workshops
  • Networking
  • Experienced researchers to mentor novice investigators

  1. Beginning Your Research

    1. Choosing a Topic

      1. Consider your personal interests
      2. Discuss your ideas with mentors and colleagues
      3. Make an appointment with a reference librarian
    2. Focusing Your Topic

      1. Limit to a specific time injury or condition
      2. Limit to a specific treatment protocol
      3. Limit to a specific age group or gender
      4. Compare two interventions
      5. Find a good review of the scientific literature which points to future research needs
    3. Getting Background Information

      1. Check with a university librarian to see if there is a published guide to the resources in the area of interest
    4. Gathering Information

      1. Textbooks
      2. Scholarly (or peer-reviewed) journals
      3. Statistics and data sets
      4. Government information
      5. Manuals or other reference materials
      6. Information Timeline
  2. Searching for information

    1. Web Search

      1. Free
      2. Broad results

        1. Popular
        2. Sometimes scholarly
      3. Infrequently provides full-text access
      4. Text, images, and video
      5. Content not organized for efficient searching
    2. Library Databases

      1. Subscription required but provided through libraries
      2. Broad or focused results

        1. Popular
        2. Scholarly
      3. Often provides full-text access
      4. Text with some images and video
      5. Content organized and structured for efficient searching
    3. Choosing Search Terms

      1. Library databases (i.e. EBSCOhost, Ovid etc.) generally don't lend themselves to phrase searching, unlike web search engines.

        1. Search through specific fields in the record of an article: title, author, publication name, abstract, etc.
        2. If your exact phrase isn't in one of those fields, you won't get any results from your search.
      2. To get results, choose key words ("lateral epicondylitis" and "ultrasound") rather than searching with entire phrases ("treatment for lateral elbow pain").
      3. How can I find good search terms?

        1. Choose the key words and phrases in your question
        2. Find synonyms and related terms (i.e. teenager, youth, adolescent)
        3. Investigate whether there are scholarly terms for any of your keywords (i.e. trigger finger vs. stenosing tenosynovitis)
        4. Do a preliminary search in a broad multidisciplinary database (such as Academic Search Complete) and explore the list of subject terms that is generated.
    4. Connecting Keywords

      1. Once you have your list of keywords use them to search through the library databases for scholarly materials.
      2. Boolean Operators

        1. Many databases allow you to connect your keywords with the words “AND”, “OR”, and “NOT”.  Knowing how to use these will help you search smartly and efficiently.

          1. Narrow your search: if your search turns up a huge number of results, and the results are either varied in scope or not relevant, connect your original keyword with another in your list using “AND”. Doing this will search for both of the keywords and only brings up the records in which both are found.

            1. example: search for "pregnancy" and then search for "pregnancy" AND "carpal tunnel syndrome" and then "pregnancy" AND "carpal tunnel syndrome" and "function" to find articles which focus on the intersection between pregnancy, carpal tunnel syndrome, and function
          2. Broaden your search: if your search turns up a very few number of results, you can broaden your search by connecting up keywords which are synonyms using “OR”. Doing this will search for either of the two keywords within the record.

            1. example: search for "osteoarthritis" AND "females" then search for "osteoarthritis" AND "(females OR women)" to find articles which focus on osteoarthritis among in the female population, whatever term is used to name them. Note: it is always good to nest the keywords you connect with OR in parenthesis, just to make sure they are connected together and the database doesn't try to do something like "osteoarthritis OR females"
          3. Get rid of irrelevant results: if your search keeps turning up results that are irrelevant to your topic, use the connector “NOT” to filter out those results.

            1. example: search for "adhesive capsulitis" AND "therapy" then search for "adhesive capsulitis" AND "therapy" NOT "surgery" to find articles which focus specifically on therapy for adhesive capsulitis, not surgical interventions for adhesive capsulitis.
    5. Finding and Using Subject Terms

      1. Articles in library databases have been categorized and assigned terms known as subject headings or descriptors

        1. Look through the subject headings to see if there is a more scholarly term or subject heading ("stenosing tenosynovitis" instead of "trigger finger").
        2. Check for subheadings that might help you decide to narrow or focus your topic ("stenosing tenosynovitis - trauma")
        3. After a search you'll find that most library databases give you other suggestions for terms.

          1. In EBSCO databases look at the box on the left to find other terms.
          2. In other databases, click on the title of the article to go to the record for it and look at the subject field to find other relevant terms.
        4. Check to see if the database has a thesaurus to view the list of the "official" word when multiple synonyms possible ("teenager," "youth," "teen," "adolescent," etc.). Using the thesaurus term will also save time.
  3. Finding Articles

    1. Searching Article Databases

      1. After collecting topic ideas and a few keywords, a great place to start is by searching through an article databases, to search for what is published in popular, professional, and scholarly journals. Journals have the advantage of presenting more current information than is found in books and because articles are shorter than books the information in them is more focused.
      2. Search article and journal databases by keywords; the search will return a list of articles that contain the words you typed in (the result list). The result list has a brief record - basically the citation to the article. If you click on the title of the article (in most cases) you'll go to a more complete record that will usually have an abstract or summary of the article. An abstract is useful for deciding if the article is worth pursuing or not.
    2. Getting the Article Text

      1. HTML or PDF Link-Most databases can provide you with immediate access to some of the articles you find; availability will be noted by a link to either (or both) the HTML or the PDF formats of the articles.
      2. Print version on the shelves: In other cases you'll see "Notes: This title is available in Library stacks" at the bottom of the result list entry.
      3. Borrow from another Library: And if the library doesn't own or subscribe to the journal at all, libraries can order it for you for free from another library.
    3. Searching for a Specific Article

      1. Specific article - one that your professor told you to find, or maybe one that you found listed in the references or works cited list of another article or book - how might you go about finding it? Using the example citation below, follow these steps in order to find a specific article.

        1. First, find the name of the journal. In this case, the journal's name is Journal of Hand Therapy.
        2. Using the library’s database, do a title/journal title search for the journal's name.
        3. The next part of the citation you need to look at is the year, volume & issue information. Look to see if the year your article was published is covered by either the print or electronic subscription.
        4. At this point, whether you have the physical journal in your hand, or are looking at the digital version online, you should browse to the right volume, issue, and page number, you should look for the title and author, and you should find the article you are looking for.
  4. Using Information

    1. Evaluating Information

      1. Read the information (articles, books, websites, etc.) that you have gathered, and critically evaluate what you find. Some information may seem to fit the criteria, but may not be appropriate upon evaluation. Some criteria by which to evaluate the information you find, and questions to ask in your evaluation:
      2. Credibility

        1. Who is the author of the material?
        2. What are the author's credentials?
        3. Is the author considered an expert in the field in which he or she writes?
        4. What is the author's reputation among his or her peers?
        5. What else has the author written?
        6. Who is the publisher of the material?
        7. Is that publisher well known?
      3. Bias

        1. Is the information presented in an objective manner?
        2. Are all sides of the issue presented?
        3. If not, can you determine the side of the issue the author takes?
        4. Does the author acknowledge a bias?
        5. Is there any inflammatory language in the material?
        6. Does the author verify statements with facts and cite his or her sources?
        7. Does the publisher stand to benefit from any research published?
      4. Accuracy

        1. Does the author cite his or her sources?
        2. Does the material provide a description of its research methods?
        3. Does the information contradict other published information?
      5. Currency

        1. When was the material published?
        2. Does this work have a more current edition or update?
        3. Does your topic require more up-to-date information (i.e. is it a scientific or medical topic or about a current event?
        4. Relevance

          1. Does the information add to the topic you are writing about, or is it peripheral to your discussion?
          2. Is the information significant and valuable, or trivial and common knowledge?
          3. Does the material provide references that will also be useful?
    2. Citing Information

      1. Some of the most common ones are

        1. MLA - Modern Language Association
        2. APA - American Psychological Association
    3. Plagiarism

      1. Here are some great resources to look at in thinking about how not to plagiarize

        1. Purdue University's OWL: Avoiding Plagiarism
        2. David Gardner at the University of Hong Kong: Plagiarism and How to Avoid It
      2. Introduce and define strategies for locating information pertaining to the question

        1. Recommended Search Strategy: Analyze Your Topic & Search With Peripheral Vision
        2. Search Tools

          1. Search Engines

            1. Comparison table of recommended search engines
            2. Google

              1. Research-quality Web Searching: Google and Beyond
              2. Free to search
          2. Database

            1. PubMed

              1. More than 5400 biomedical journal titles created by the U.S. National Library of Medicine

                1. Basics
                2. Searching PubMed with MeSH
                3. Free to search
            2. Ovid

              1. More than 4500 ebooks, 1200 peer-reviewed journals, & 100 bibliographic & full-text databases

                1. Basic Search
                2. Advanced Search
              2. High cost but available through most large institutions
            3. EBSCOhost

              1. More than 1,700 journals covering the social sciences, humanities, general, 1,090 journals covering business, management, economics, 500 general interest and current events periodicals, and 340 military related periodicals: Training Tutorials home

                1. Basic Search
                2. Advanced Search
              2. High cost but available through most large institutions
            4. Google Scholar
            5. OT Seeker

              1. abstracts of systematic reviews and randomized controlled trials relevant to occupational therapy
              2. Free to search
          3. Subject Directories

            1. Table comparing some of the best human-selected collections of web pages
          4. Meta-Search Engines

            1. Use at your own risk: not recommended as a substitute for directly using search engines Advanced Boolean Searching Tutorial
          5. Advanced Boolean Searching Tutorial