| Evidence-based Medicine Newsletter | |
| EBM Resource Center of the New York Academy of Medicine and The American College of Physicians, NY Chapter | July 1999 |
EBM Resource Center Sponsors "Retreat" on Educational Interventions
On Friday June 6th, the Evidence-based Medicine Resource Center hosted a retreat for tutors and members of the EBM Steering Commitee for the Course "Teaching Evidence-based Medicine". The Retreat, which featured reports from major educators, was designed to begin the process of evaluating the impact of evidence-based educational actitivies at the Resource Center.
Speakers at the event were: Dr. Rosanne M. Leipzig of the Mt Sinai Medical Center, Dr. Geoffrey Norman of McMaster University, Dr. Ian Hart of the University of Ottawa, Dr. Michael Green of Yale University, and Dr. Cheryl Walters of Columbia University. After each of the presenters discussed their perspective on the ways in which ebm education could, or could not, be evaluated, a lively discussion between the speakers and program attendees followed.
The EBM Committee is instituting a series of assessment activities and will report on them in the fall.
EBM Educational Programs at the Resource Center
The EBM Resource Center continued its series of Educational Programs with three further sessions. The first, held on March 2nd, discussed "The Cochrane Collaboration: What it is, Where Can I Find it, Should I Get Involved?" The program, facilitated by Rosanne Leipzig, MD, and Jean Sullivant, MLS provided an overview of the database, The Cochrane Library, and the Cochrane Collaboration, and their work on systematic reviews. (A fact sheet on the Cochrane Library is available on the EBM Resource Center page at: http://www.ebmny.org/whatcoch.html)
"How to Introduce Evidence-based Medicine at Your Institution", held on April 6th, 1999, was the second program, and featured discussions and slide presentations by Rosanne Leipzig, MD, Thomas McGinn, MD, and Eleanor Wallace, MD. The program featured tips on disseminating ebm information to leaders, colleagues, and trainees within hospital and academic institutions. Slides from the program are available for downloading from the EBM Resource Center home page at: http://www.ebmny.org/
ebm/sindex.html.
The final program in this series, "A Walking Tour of Our Web Page: Key Sites That will Help You Practice EBM," which was held on May 4th, 1999, focused on the EBM Resource Center Home Page, and some web sites that are useful in evidence-based practice. Eleanor Wallace, MD and Patricia Gallagher, MLS demonstrated major websites such as the JAMA Contempo Page, The ScHARR Home Page, The POEMS Home Page, and The Center for Evidence-based Medicine. All sites are available from different places withing the EBM Resource Center Home page at: http://www.ebmny.org
CATS, Part II
by Ilene Wilets, MD
In the previous Evidence-Based Medicine Newsletter, Dr. Peter Wyer's article entitled "Letting CATs Out of the Bag" discussed the impetus, utility, and potential pitfalls of CATs (i.e., Critically Appraised Topics). Today there exists numerous evidence-based websites on the Internet containing CATs and variations of CATS. These sites are organized differently with varying levels of detail and quality. To obtain a sense of the scope and attributes of existing CAT-Banks, a review of was conducted using the search term "Critically Appraised Topic" with Medline, Hotbot and AltaVista search engines.
The Medline search yielded only 3 documents, whereas the search with Hotbot produced 26 web matches, and AltaVista offered 155 web pages pertaining to CATs.
Upon review of the many CAT websites, I was struck more by their commonality than diversity. A cursory inspection led me to CATegorize them into three major breeds:
A purebred CAT is an organized review of the best evidence pertaining to a specific clinical question or topic. Purebred CAT websites tend to be well organized, incorporating EBM principles. They begin with a clinical scenario, leading to a specific question. The "clinical bottom line" usually follows, suggesting the clinical course of action based on the best available evidence. The search databases are indicated (e.g., Medline or Cochrane), as are search terms and delimiters. Next, "the evidence" is presented, often as a data summary table displaying the number of study subjects, the number of subjects with a particular outcome, relative risk, 95% confidence intervals, and the number of patients a clinician needs to treat to avoid one additional adverse outcome (NNT). Several of the websites provide a data table, which functions as a calculator enabling the CAT browser to add data to recalculate these statistics.
A mixed-breed CAT indicates an evidence-based review of the literature, however, the study search strategy and inclusion parameters are often not articulated, or with limited detail. Some sites have reviews of systematic reviews. These reviews are CAT-like in nature, and therefore may be viewed as a mixed-breed.
Club CATS follow the journal club format with a critical review of one or two articles addressing a clinical question or topic.
Characteristics of Individual Sites
University of Washington
The CAT-Bank from the University of Washington, a Pediatric Evidence-Based Medicine website (http://weber.u.washington.edu/~ebm/) is organized as a purebred. CATS are grouped in 13 clinical areas, one of which is dedicated to emergency medicine. There are 11 more clinical areas in development, but these are not yet available for browsing.
Center for Evidence-Based Medicine
Another popular CAT-Bank is that from the Center for Evidence-Based Medicine in Oxford (http://cebm.jr2.ox.ac.uk/docs/catbank.html). Sixty-three CATS are listed alphabetically by disorder, ranging from alcoholic hepatitis to venous thromboembolism. Each CAT poses questions as to the definition of the study sample, the length and completeness of follow-up, objectivity of outcome measures and adjustment for potential confounders. The search and selection strategies, however, are not specified, rendering this more of a mixed-breed type of CAT-Bank.
The Bandolier
The Bandolier (http://www.jr2.ox.ac. uk:80/Bandolier/) is another comprehensive site devoted to evidence-based health care. It is also produced in Oxford for the NHS R&D Directorate. The site offers critical appraisals of systematic reviews in particular clinical areas. A large part of the Bandolier is dedicated to summaries of systematic reviews within the field of Palliative Care. The search strategy, the evidence and a brief critique of studies are provided. This CAT-Bank also may be thought of as a mixed-breed.
The ACP Journal Club
The ACP Journal Club (http://www.acponline.org
/catalog/cbi/best_evidence.htm) can be considered a journal club CAT. It is an easy-to-search database of over 1000 abstracts from issues of ACP Journal Club (1991-1998) and Evidence-Based Medicine (1995-1998). Journals from internal medicine, family practice, pediatrics, obstetrics, psychiatry, gynecology and surgery are reviewed by ACP editors. Articles meeting strict selection criteria for study design are selected and summarized in a structured abstract. Included within the abstract is commentary on study quality and the clinical bottom line based on this evidence.
SORAHSN
The SORAHSN (Southwestern Ontario Regional Academic Health Science Network: http://ahsn.lhsc.on.ca/cat/ contains CAPS (i.e., Critically Appraised Papers) derived from the McMaster's Users Guides. The CAPs have the same areas as McMasters; including therapy, prevention, harm and diagnosis. Each review of a paper follows the Users Guide critical appraisal strategy of questioning the validity of study results, determining what were the results, and whether they will help with caring for patients. A results and clinical bottom line section follows. A handy feature of this journal club CAT is that one can directly link to the McMaster site from any given CAP.
There are numerous journal club CATS dedicated to medicine, nursing and mental health. A few of the many useful sites are: "POEMS" (Patient Oriented Evidence that Matters): http://www.infopoems.com/; Journal Club on the Web: http://www.journalclub.org; and the Journal of Family Practice: http://jfp.msu.edu/jclub/ indexes/jcindex.htm#New.
In summary, CATs offer bedside access to scientific evidence pertaining to a specific clinical question. It is to the advantage of the CAT browser to become knowledgeable as to the strengths and limitations of the various CAT-Banks on the Internet. A well-conceived CAT can serve as an informed guide for clinical decision-making. If not well researched and properly articulated, a CAT can relay misleading information or be misinterpreted by those accessing the information. The CAT browser should understand that the information gleaned from a CAT is only as good as the studies contributing to it. Additionally, it is imperative to recognize that CATs serve to support, not replace, sound clinical judgement.
Providing Meaningful Feedback About EBM Activities
by Carol Martinez Weber, MD
Whether one is precepting students, working with colleagues or participating in a journal club, or critical analysis of the literature session, feedback plays a key role in enhancing the EBM learning experience for all involved. Feedback is the sharing of information that describes an individual's performance in a given activity. Evaluation is distinct from feedback: evaluation involves judgement; feedback involves description. One uses precise and neutral language composed of nouns and verbs in depicting a learner's behavior. For example:
We can integrate constructive feedback into any type of learning situation or environment. Choose the proper setting for the feedback session, based on who the learners are and the nature of the teaching environment. Some sessions require privacy to maximize learning; others are done within a group, in which all participants understand the meaning of feedback and participate in turn as givers and receivers, observing basic guidelines. Open the session by asking the learner to do a self-assessment that begins with positive statements or descriptions of things done well, followed by concrete and specific comments on opportunities for improvement. Then, when others give feedback, insist that the learner remain silent. Close with a summary that allows one to understand what the student learned and ask the learner, "What are we going to work on or do next?"
Feedback is interactional and collaborative, and helps to instill self-assessment as a lifetime process. Address only particular behaviors that can be changed; do not focus on personality characteristics, nor on environmental constraints.
Good feedback creates options. The use of positive language communicates hope that meaningful change can occur. Through mutual development of solutions to problems, common goals are set between the learners and the teachers; performance can improve as you work together cooperatively.
Remember, the key points on feeback are:
Three New Users Guides Published in JAMA
JAMA has published three new Users' Guides, as part of this continuing series:
Barratt A, Irwig L, Glasziou P, Cumming RG, Raffle A, Hicks N, Gray JA, Guyatt GH. Users' guides to the medical literature: XVII. How to use guidelines and recommendations about screening. Evidence-Based Medicine Working Group. JAMA 1999 Jun 2;281(21):2029-34
Guyatt GH, Sinclair J, Cook DJ, Glasziou P. Users' guides to the medical literature: XVI. How to use a treatment recommendation. Evidence-Based Medicine Working Group and the Cochrane Applicability Methods Working Group. JAMA 1999 May 19;281(19):1836-43
Richardson WS, Wilson MC, Guyatt GH, Cook DJ, Nishikawa J. Users' guides to the medical literature: XV. How to use an article about disease probability for differential diagnosis. Evidence-Based Medicine Working Group. JAMA 1999 Apr 7;281(13):1214-9
Librarian's Report: A Residency Program Introduces EBM
by Roberta Bronson-Fitzpatrick, MLS
Residents and interns in CIMA, the Cornell Internal Medicine Associates, will begin an evidence-based medicine training program in Fall 1999. There will be at least three sessions which deal with online searching techniques. The first, "Introduction to Searching Techniques," will begin with an exercise, in which each trainee will be asked to answer some questions about a clinical scenario to be collected prior to the teaching session.
Helen-ann brown [sic] and Roberta Bronson Fitzpatrick, librarians at the Weill College of Medicine, devised various search strategies for that scenario, executed in a variety of electronic resources, such as MEDLINE (via OVID, PubMed, and Internet Grateful Med), Cochrane, and Best Evidence. The teaching sessions will address why various sources were chosen and will analyze formulation of the resulting search strategies, as well as how best to develop clinical questions likely to yield meaningful results.
First Session Scenario and Activities
Mrs. P is a 45 year old woman who presents with gradual onset of dyspnea over the last two weeks. There is a slight dry cough. She also reports occasional sharp chest pain on the right when she takes a deep breath or coughs. She denies fever, chills, weight loss, URI symptoms, joint pain, leg swelling, calf tenderness, history of heart disease or reflux. She has no relevant past medical history. Her surgical history is notable for C-section 20 years ago without complication. Her only medication is oral contraceptive pills. She has no allergies to medication. She quit smoking five years ago and has a ten pack-year history. Her family history is notable only for breast cancer in her mother at age 60 and hypertension. On physical exam she is afebrile, pulse is 94, BP is 120/80, RR is 18, her O2 sat on RA is 96%. Her lungs are clear, cardiac exam is normal, there is no edema or Homan's sign. Her EKG is sinus rhythm with normal intervals and axis and without ST abnormalities. Her CXR is clear without cardiomegaly. Her cbc is normal except for a slight left shift (70% PMN's.) SMA7 and pt/ptt are all within normal limits. She refuses ABG.
You are unable to obtain a V/Q scan overnight. The only available modalities are bilateral LE doppler and high resolution spiral CT scan of the chest.
Initial Question:
Based on your current knowledge base, which would you choose? If the result of this test is negative, what is the liklihood that the patient actually does have PE?
Evidence Based Search
Estimate Negative Predictive Value
Perform a literature search to decide, based on best evidence, which test you would use and what its negative predictive value (NPV) is. Please keep a record of your literature search technique so that you can compare it with the librarian's methods. Based on your search and taking into account the clinical scenario above, what method would you choose? What is the NPV? Please cite the best references upon which you based your decision.
Further EBM Sessions
Two other searching sessions will focus on specific techniques used for finding answers to therapy, diagnosis, prognosis/harm, and etiology questions in the biomedical literature. Plans for the final course content are being developed as of this writing. The course is being developed as a pilot project by two CIMA residents, with assistance from Library and CIMA staff.
Gentler Error Messages
Contributed by Eleanor Z. Wallace, MD
The cryptic, and sometimes threatening error messages characteristic of Microsoft Windows operating system prompted a response which appeared in my mailbox which attempts to put a human face on these messages by using Japanese Haiku poetry.
Some examples:
Chaos reigns within.
Reflect, repent, and reboot.
Order shall return.
First snow, then silence.
This thousand dollar screen dies
so beautifully.
The Tao that is seen
Is not the true Tao, until
You bring fresh toner.
Stay the patient course
Of little worth is your ire.
The network is down.
Three things are certain:
Death, taxes and lost data.
Guess which has occurred.
New EBM Resources
by Eleanor Wallace, MD
Clinical Evidence
http:// www.evidence.org
Clinical Evidence is a new resource for clinicians, offering a pocket-sized book, updated every six months, containing concise summaries of the best available evidence on clinical interventions with a focus on answering clinical questions. The first issue (June 1999) contains summaries on prevention and treatment of about 70 common conditions. These summaries are written by practicing physicians using rigorous searching and evaluation methods looking for systematic reviews or, alternatively , individual randomized control trials and are peer reviewed. The articles present the evidence but do not make recommendations, leaving practice decisions to the health care provider.
A sample of one of their reviews appears in the BMJ which publishes this new resource(Charman, Carolyn Clinical Evidence: Atopic Eczema BMJ: 1999, 318:1600-1604). The full text is available at: http://www.bmj.com/cgi/content/full/318/7198/1570. Further details about the book and how to purchase it can be found on their web site listed above.
MDConsult: a Fee-based Online Library
http://www.mdconsult.com
The NYAM/ACP Resource Center is adding MDConsult to its database accessible by password to those who have taken our EBM courses, available through September of 1999. This resource is a comprehensive (but not complete) collection of about 40 books, 50 journals, practice guidelines, patient education and drug information databases which can be searched collectively or individually and will provide full text of any hits in the MDConsult database. It does not include internal medicine journals which have their own web sites (New England Journal of Medicine, Annals, for example). It has an excellent user interface, a useful collection of resources for the practicing physician and may serve as an effective one-stop-shopping background site for the practitioner attempting to answer a clinical question in real time. In combination with the Cochrane Library and Best Evidence, both available through OVID on our site, it allows for a rapid search for background and foreground (recent systematic reviews or structured abstracts) answers to clinical questions. Information about subscribing personally for those not eligible for a password can be obtained from their web site (see above).