Clinical Reference Road Test briefly compared the efficiency and accuracy of several online references used in completing the home study portion of the American Board of Internal Medicine's recertification program. Here, these references are individually described and compared in detail.

Harrison's Online

The centerpiece of subscription-based Harrison's Online is the classic medical textbook, Harrison's Principles of Internal Medicine. The core text and color atlas are from its 14th edition, but content is periodically updated with new references, some of which are summarized within the text itself. References are linked to Medline abstracts where available.

The site's newest clinical information is presented in well over a hundred Updates, Clinical Trials, and Concise Reviews & Editorials, usually written by Harrison's staff. All articles can be accessed independently or from within each chapter, and they are well integrated into the site's search feature. The most recent are showcased on the home page and, more extensively, under What's New. Also available are Harrison's fine PreTest Q&As, although there is no CME credit available. Other site features, such as Patient Notes (handouts), Related Links, and a discussion Forum, are far less satisfying.

Navigation through clinical content is three-pronged: the Table of Contents, Index, and Search Engine. The Index uses letter links, drop-down menus and lots of scrolling, but it is the quickest way to access simple topics within Harrison's itself; unfortunately, it does not directly link to new articles. The full Table of Contents loads slowly. It is good for browsing, however, and includes links to new articles alongside chapter headings. The Search Engine does the best job of accessing and prioritizing more complex topics in both Harrison's and new articles. It allows for Boolean operators and limits hit results using advanced features, but it has trouble with synonyms. For example, searching for "medullary thyroid carcinoma" and "medullary thyroid cancer" should result in the same hit list, but doesn't. Within a chapter, related new articles and site features can be found using a series of buttons located on the left navigation panel, which also links to the chapter subheadings.

SAM

The main competitor to Harrison's Online is subscription-based Scientific American Medicine (SAM) Online, which is presently accessible for free through the medical library link in WebMD Practice. SAM's unique text maintenance system, in which whole chapters are periodically updated in rotation, translates effortlessly into electronic media. SAM's strengths work well on line; each revised chapter is dated at the of the page, and almost all of its wonderful illustrations are nicely presented as embedded thumbnail images that enlarge within their own windows. Like Harrison's, SAM's references are linked to Medline abstracts.

SAM's updating is less extensive than Harrison's but, as described above, is wholly integrated into the core text. Chapter updates are summarized each month in a scrollable list with links to content and references. Otherwise, there is no "new content" feature except recent FDA drug approvals, when available. SAMSAQ, whose monthly program content is fully on line, competes favorably with Harrison's PreTest, although its CME credits are available only at an additional cost.

Finding content within one current text rather than a dated text plus current articles would be a real time-saver were it not for SAM's less sophisticated and efficient navigational tools. The basics are here; the expandable/collapsible Table of Contents on the left navigation screen permits access to individual chapters (confusingly, some identified using arabic and some, roman numerals), but there is no Index. The keyword Search Engine links to chapters with search terms highlighted in red, a nice touch that is necessary since the entire chapter is presented as one scrollable text. As with Harrison's, searches using Boolean operators are allowed, although overall the engine here feels less powerful. Results are prioritized and include introductory descriptions of each chapter that are as often incomprehensible as they are helpful. An ever-present navigation toolbar permits access to the advanced search engine that filters out references, tables, and figure captions. It also has a thesaurus option but, here too, "medullary thyroid carcinoma" and "…cancer" yield different results. Within a chapter, one can move directly to next and previous search hits, an option that Harrison's site lacks. As expected, the left navigational panel within a chapter provides links to the chapter's subheadings.

MD Consult

The 38 textbooks within MD Consult comprise the main component of the site's Answers section. Few physicians will find fault with the selection of medical classics, which include Cecil's (Medicine), Mandell's (Infectious Disease), and DeVita (Oncology) among many others. Unfortunately, all textbooks share the same inelegant interface: content is presented in the right window as a large scrollable screen, complete with print page numbers and embedded tables and graphs that may or may not be available. MD Consult's attraction is the number and breadth of its textbook offerings, not timeliness, since the textbooks are not updated between editions in any fashion.

The four other components of the Answers section are more forgettable. Journal Search, Practice Guidelines, Clinical Reference System's Patient Handouts, and GenRx. Free PubMed is a better online resource than Journal Search, even given MD Consult's occasional full-text article, and National Guideline Clearinghouse, also free, is better than Practice Guidelines. Together with Books, Answers is its own searchable database, which may prove fruitful if a separate textbook search, as described below, comes up empty.

MD Consult's other section, Updates, has several interesting although flawed features, none of which is comparable to the clinical update sections of Harrison's and SAM; PraxisMD (see below) has very similar features that are far better realized. Appropriately, Updates is not linked to clinical content and uses its own search engine.

Navigation of textbook content is much improved since my last review, but still has problems. Each textbook can be accessed using its Table of Contents or Index via a left navigation panel of expandable/collapsable menus. The Search Engine searches all 38 books at once, but is relatively quick and provides an intermediate list by textbook in descending order of number of hits, so it's easy to find one's book and concentrate on its hits exclusively. Other search engines within MD Consult contain more features, but here the engine accepts up to six search terms with several Boolean operators. The search engine is quite good, and similar to SAM's in that search terms are highlighted in red within a large scrollable text. Synonyms are especially problematic. Here, changing "medullary thyroid carcinoma" to "…cancer" eliminates all hits from DeVita's classic cancer tome. Hits are linked to each text's Index, which makes the Boolean operators especially useful.

eMedicine

A full series of medical and surgical specialty textbooks is promised from eMedicine, but only the emergency medicine text on this innovative site is presently on line. Still, there is enough content crossover, and it is of sufficient depth and breadth, for eMedicine to be reviewed alongside these clinical references. Emedicine is a free resource, with multiple corporate sponsors and flashing banner ads ever-present atop the content pages.

Unique to the Web, eMedicine's many hundreds of chapters are created, edited, and updated entirely on line by physicians using eMedicine's proprietary software. Emedicine's content also has the simplest structure of all clinical references listed here. Chapters are based on a common template divided into sections with text written in a combination of narrative and outlines. One section, Differentials, hyperlinks to other chapters and another, Medication, lists drugs in table format. Each chapter concludes with study questions (CME credit is coming soon) and a Bibliography containing links to Medline abstracts when those are available. Embedded graphics grouped together towards the end of the chapter allow for graphs, figures, and images that enlarge to fill the entire right window, temporarily replacing the text.

The left navigation panel links to other emergency medicine chapters, accessible by section (Table of Contents) or alphabetically (Index). A simple search engine prioritizes hit results by chapter only. Although chapters are divided into sections, navigation is limited to scrolling down the text, which can be exhausting even when one is familiar with the chapter template.

Praxis MD

Praxis MD's physician reference, Best Practice of Medicine, is updated by section in a manner similar to SAM's chapter rotation updates. Freely accessible for the present, Best Practice of Medicine articles (chapters) appear on line in two parts: Briefs, one-page summaries of practical points, and Reports, the core content comparable to other sites. Praxis MD features Field Reports, which update topical articles that are oriented more to primary care practice than are Harrison's. Overall, Praxis MD's physician reference is the least comprehensive of those reviewed here, with many content areas missing. However, the areas that are covered are solid and thoughtfully presented from a basic, primary care perspective.

Praxis MD is a medical portal site that also contains an extensive patient reference, a magazine for physicians, PubMed and Web links, and free e-mail service. MDAlerts (also sent as weekly e-mail) and Journal Scan are solid features not part of the physician reference area. Influences from evidence-based medicine abound, although I (and perhaps Archie Cochrane) might argue over some of its editorial board recommendations.

The attractive structure and navigation tools are unique and certainly a matter of taste. Others may love them, but to me they feel overly cumbersome without enough satisfying content to justify the effort. Best Practice of Medicine can be accessed using a Table of Contents on the left navigation panel, or by one of two search engines. Site Search accepts Boolean operators, and MDQuickAnswer is a topic (keyword) search using mouseover menus to help visitors locate information more efficiently. These are faster than the Table of Contents perhaps, but neither functions as a powerful search engine, which Praxis desperately needs. For example, a site search on "birth control" brings up Best Practice of Medicine articles on rhinitis, diseases of hair, and similarly unrelated topics, and a search for "medullary thyroid cancer" misses a number of references listed under "…carcinoma." Once within an article the left navigation panel and assorted toolbars help sort through the complicated maze of information. Each article has its own search engine, which is a very nice touch other sites should adopt. Search terms are highlighted in red, another nice touch. References are not only linked to Medline abstracts but to related articles as well.

Conclusion

My main concerns while completing the home study program were the quality and ease of accessing clinical information. From that perspective, I found Harrison's and MD Consult to be the best clinical references. SAM's and eMedicine's navigation and search tools were not nearly as efficient, and either Praxis MD lacked the content or its search tools did not work for me as expected. Importantly, eMedicine's content was highly variable from chapter to chapter; some are excellent but just as many provided outdated or inaccurate content (unlike the other four references, which at worst provided incomplete or no information).

References varied somewhat with regard to timeliness of content. I found Harrison's articles contained the most current information. Surprisingly, a number of MD Consult's textbooks were as up-to-date as many of Harrison's articles and SAM's ongoing content renewal. Emedicine prides itself on its near-instantaneous updates, but in practice I didn't find the content more current compared to these other resources. I wasn't able to access enough content on Praxis MD to appreciate its timeliness.

So, which reference is the best overall? That answer may need to wait until the next part. UpToDate's online database looks like the next generation of electronic clinical references and the online reference to beat at present, but it's far from perfect.

Disclosure: Dr. Lazoff wrote three chapters for eMedicine and submitted an article to PraxisMD.

continue on to Part 3: Clinical Odyssey

Related resources on MCToday:

• Frequently Updated Medical References
• Online Medical Services