Critical appraisal skills: evaluating information

This is the third in a series of articles to illustrate how to critically think and reflect upon practice problems. (Read our introductory article here.) Future articles will explore how you can communicate your decision to your patient or other healthcare professionals.

The previous article in this series focused on finding information relevant to your patient or practice scenario and discussed the usefulness of summary resources for this purpose. Recall that our pharmacist received a prescription for mirabegron for her pediatric patient. She confirmed with the patient’s father that the prescription was for overactive bladder (OAB). He said his daughter had tried another medication already. The pharmacist’s review of the patient’s Netcare profile indicates recent use of oxybutynin; she also notices a recent negative urine culture. However, the patient’s father said that the medication had made his daughter constipated and she stopped taking it.

The pharmacist checks one of her bookmarked summary resources, RxTx. The drug monograph for mirabegron does not have dosing information for children because safety and efficacy haven’t been established in this population.[1] She reviews the therapeutic information on adult and pediatric urinary incontinence but does not find any information that supports dispensing this drug for her patient. What now?

The 6s pyramid[4]

Recall the 6S pyramid that we introduced in the last article. See here for definitions of each level of the pyramid as well as some examples of different resources you will find in each level. If you cannot find the information you need near the top of the pyramid, start working your way down. The pharmacist works her way down the pyramid and eventually reaches the “Single Studies” level and checks PubMed Clinical Queries with relevant search terms. She finds the following results:

There is nothing listed in the “systematic review” section of the query so there are no summary resources the pharmacist may consult. Before beginning to evaluate the evidence she sees under the “clinical studies” section in detail, she first considers the following questions developed by the Centre for Evidence-Based Medicine (CEBM)[2]:

1. Does this study address a clearly focused question?
2. Did the study use valid methods to address this question?
3. Are the valid results of this study important?
4. Are these valid, important results applicable to my patient or population?

If the answer to all these questions is yes, the more likely it is that the evaluation will be productive. Recall from the previous article in this series that the most useful information is directly applicable to your patient, evidence based, and obtainable with a reasonable amount of time and effort.[3] Since the pharmacist is applying critical appraisal skills at the point of care, she needs to be selective in choosing the best information to help her make a decision.

The pharmacist considers these questions and chooses the second study to evaluate in detail, “Prospective Pilot Study of Mirabegron in Pediatric Patients with Overactive Bladder,”[4] since it seems the most applicable to her patient. She considers the following after reading the study:[5]

1. The population and intervention. How closely does the study population match her patient? Does the treatment intervention used in the study match what she sees in her practice?
2. Biases. What biases did she discover while reading the study? What impact will this have on her overall decision?
3. Apply the findings to your patient or your practice scenario. To do this she had to reflect on the study sample size, summary statistic used, and confidence of the result. The CEBM has developed this resource to assist health professionals.

All this information must be considered when making clinical decisions when caring for your patients. Even good quality evidence has limitations. After reviewing the available information, the pharmacist has some concerns. The study sample size was relatively small (only 14 girls were enrolled) and there was no placebo control group. Unfortunately, there were no summary resources and only a limited number of other single studies available to review. She thinks this may not be an appropriate treatment. Her next step is to discuss her findings with the prescribing physician and the patient’s father.

Practice completing a PubMed search and review the materials available on the CEBM website. What do you conclude from your appraisal of the available information? Do you agree with our pharmacist? Why or why not? Compare your answers with a colleague.

In our next article, we will take a closer look at communicating information obtained through your critical appraisal process to make it meaningful for your patient and other healthcare professionals.

*NOTE: This article was updated on October 14, 2020.