Boards Review Software
Marjorie Lazoff, MD
Emergency Medicine
Philadelphia, Pennsylvania
Medical Editor
Medical Computing Today

accepted for publication in Medical Computing Today October 1997
[updated] February 2000
Originally published in edited form Sepember 1997 in Medical Software Reviews

Criteria - Ratings - Limitations - Summary - Requirements - Comparative Chart

This article explains the selection criteria and review process as applied to the individual software packages. Following the article is a comparative chart of software features, which links to the reviews individually and by specialty. Reviews will be appended as other appropriate software is identified. Physicians are invited to contribute their own reviews, which should include information to complete the categories in the Table, as well as the version number, ratings, and other information placed at the beginning and end of each review. Remember to include the name of the software package when responding with comments or questions using the link provided at the bottom of each page.
Like most physicians, I regard the inconvenience and expense of certification and recertification exams as second only to the onerously time-consuming, anxiety-producing and megalomaniacally intellectual task of preparing for them. Fortunately, many specialty organizations, medical institutions, and software companies have created electronic Boards review programs. But to those of us predisposed to using a computer for review and study, are these software products worth our time and money?
For the purposes of Boards preparation, experts recommend candidates simulate test conditions. Ideally, preparations should be made by sitting at a desk before an open book with a sharpened No. 2 pencil in hand; until electronic testing is an option, software users will be at a disadvantage in this regard. Including questions and an answer form, both printable, is one option. But by employing computer-assisted instruction (CAI) techniques, electronic Boards review software can offset any disadvantage by increasing the user's understanding and powers of recall and retention, quickly assessing strengths and weaknesses, and decreasing subliminal cues by manipulating the order of questions. This is in addition to the educational value of searching through vast quantities of text or a large bank of question in seconds.
Some electronic Boards review programs are designed specifically for physicians just completing residency, while others are geared more for the recertification exam. On one end of the spectrum lie computerized core content reviews filled with multimedia; on the other end lie banks of simulated Boards questions. Many of these software packages were derived from course reviews or texts, and others were created solely for the medium. All but one are available in CD-ROM format; of these, two contain five disks each, and one offers up to four specialty reviews on a single CD. This diversity is laudable since it addresses the wide range of physician needs and preferences, but it complicates a consumer's purchase choice.
Sections Selection Criteria
To be selected for this introductory review article, the software's major (not necessarily sole) objective had to be to prepare the user for certification or recertification in a general specialty field. Effort was made to include the most popular packages in each specialty, and/or those that came from or were recommended by the specialty organization. Originally, four internal medicine, three family practice, three pediatric, two emergency medicine, one ob/gyn, and one surgery program were selected for inclusion. At the time, no appropriate psychiatry/neurology package could be located, and several packages were rejected either because their course content required specialty education or unique multimedia needs (such as nuclear medicine or radiology), or because they represented subspecialties (such as cardiology).
Beginning with the February 2000 update, the online version was expanded to include board review software beyond the primary specialties. MCToday is actively soliciting software to review, and reviewers; contact our editor.
Software was reviewed solely as preparatory material for the written exams. Tabular information is provided for health care professionals also looking for self-study materials, clinical resources, or CME credits.
Regardless of specialty, all Boards review software seeks to accomplish the same task. With the important exceptions of software developers and users looking for broad-based overviews outside their primary discipline, inter-specialty comparisons are of limited benefit. Such comparisons are included when appropriate, but the majority of this review will focus on describing each program and pitting it against others in the same specialty, if any.
Sections Criteria for Ratings
The following five criteria incorporate aspects of certain grids, studies on and reviews of educational software, and objectives as supplied by several of these vendors.
  1. Content: The goal is for content to match the authority, and the depth and breadth of material to be tested. Although software that goes into greater detail than the test is likely to ask, or presents all sides of a subtle controversy, is a wonderful resource, it is distracting given the immediate task at hand unless this material is presented as an adjunct.
  2. Programming: The software should have an intuitive interface or be easy to learn after a short tutorial. Practical features expected in software of this type include strong navigational tools such as appropriate search engines and intraprogram links; personalization tools such as bookmarks, highlighting and addition of personal notes without destroying the original reference; and task-specific tools such as maintaining scores and allowing multiple, password protected users.
  3. CAI (computer-aided instruction): Not just presenting graphics and multimedia, but incorporating them within thoughtful education. Good Boards review software should allow users to interact with, not just navigate through, content, and should individualize feedback for inappropriately-answered questions.
  4. Error handling: Software part of the original review was tested then gamed on a PC with a 90 MHz Pentium processor, 16 Mb RAM and a double-speed CD-ROM player. As technology develops, more powerful computers are utilized for updates and new software.
  5. Support: This includes manuals and/or online help, and availability of technical support, especially the availability and appropriate responses of telephone support.
Sections Limitations of Review
Opinions of respected peers regarding many of these packages were sought informally. Since feedback from even one such user can be helpful to potential buyers, educators, and software developers, I encourage readers to e-mail our editor with experiences regarding these or similar software packages.
Even though a physician's medical education and clinical experience afford some familiarity with every specialty, such background is clearly insufficient to judge the adequacy of course content for every electronic Boards review. Each package presented here has an overt affiliation with a medical institution, specialty organization, or major vendor company that ought to know what content should be covered. In other cases, content strengths and weaknesses were obvious, and the reasoning behind such descriptions is provided for the reader's consideration. Even more than with other preparatory materials, potential purchasers need to compare the subject covered in each package against the subject covered by each certification or recertification examination, which is available through the respective specialty Board.
Sections In Summary
Most Boards review packages offer one of three categories of content: lectures, articles, or outlines from Boards review conferences; booklets, monographs, or articles prepared for Boards self-study; or authoritative specialty reference texts with or without accompanying study guides. Several packages are without didactic content. Most but not all programs possess the minimum requisite of software design and multimedia.
Second, a physician may approach the recertification process as a refresher course, or may desire a fair amount of self-instruction -- in which case CAI is a desirable feature. Given the limited CAI in this group, such physicians may want to investigate other categories of medical education, at least until software developers directly address our professional needs.
Finally, physicians have access to a variety of self-testing materials, some more appropriate than others at various stages of their review: pretesting and other formats designed to self-assess strengths and weaknesses; subject questions with explanatory answers serving as information review; large banks of randomly generated questions to improve test-taking strategies and increase endurance; or practice exams to gauge one's progress. Many programs include more than one form of self-testing, but a surprising number of programs have not taken advantage of computer programming in this area, leaving users without the ability to manipulate questions or obtain automatic scoring.
More and more specialties are offering time-limited certifications requiring recertification exams, and institutions are now demanding physicians document their professional skills and continuing education. Together with the increasing popularity and educational advantages of computers, and with electronic test taking becoming a reality, electronic Boards review should receive more and more attention and support. Even now most specialties offer a variety of Boards review software impossible five years ago, including software designed or supported by five major specialty organizations (AAFP, AAP, ACEP, ACP, ACS). Still, too many of these packages include dated content, are limited from a design perspective, use less-than-excellent programming and token interactivity, and do not take advantage of CAI techniques. We physicians should encourage specialty organizations and software companies to incorporate the best from software developers and medical educators, and we should encourage our peers, departments, and medical schools to purchase those products that reflect the best in content, programming, test-taking features, CAI, and price.
Sections Software Requirements
Most packages run on both Macintosh and Windows platforms. Minimum specifications beyond a 486 (PC) or 68030 (Mac) processor, 4 MB RAM, 4 MB hard drive space, mouse, SVGA graphics card/monitor, and a 2X CD-ROM drive. Exceptions, additional requirements, the number of disks, and any extra items included are noted at the end of each review, as are the sources.
Comments or questions for posting?
Archives of other articles

Sections Comparative Chart of Software Features
This table is organized by specialty, with links to specialty and individual software reviews. For those using browsers that cannot read tables, scroll down or use the following links to the non-tabular comparative list containing the same information:
Cardiology: Non-table listing of Software Features - Software Reviews
Emergency Medicine: Non-table listing of Software Features - Software Reviews
Family Practice: Non-table listing of Software Features - Software Reviews
Internal Medicine: Non-table listing of Software Features - Software Reviews
OB/GYN: Non-table listing of Software Features - Software Reviews
Pediatrics: Non-table listing of Software Features - Software Reviews
Psychiatry & Neurology: No products reviewed yet.
Surgery: Non-table listing of Software Features - Software Reviews

# of
Didactic Material
ACCSAP 2000 115 ACP
19 sections and ECG interpretation. $345 5-4-2-4-4
CathSAP 65 ACP
282 15 chapters $395 4-4-4-2-5
Emergency Medicine
Emergency Medicine Plus 200 ACEP
1st 10 free
Tintinalli's EM 4th Ed. $179 5-3-2-5-4
Med-Challenger EM 200 ACEP
1st 50 free
4,000 None $615 3-5-3-5-5
Family Practice
AAFP Home Study Self-Assessment 60 for
1,000 72 monographs $195
updates $95
Family Practice Recert.
30 AAFP 2,400+ 7 years FPR $295 individuals
$395 inst./network
updates $150
Family Practice Review 70 AAFP 350+ 86 audio & slide lectures $295 individuals
Med-Challenger FP 192 AAFP
1st 50 free
3,600+ None $615 3-5-3-5-5
Internal Medicine
ACP MKSAP 10 Electronic 146 1,700+ 17 MKSAP Booklets $4954 5-4-4-5-5
CCF Intensive Review
of IM 1996
54 350 75 audio & slide lectures $989 5-4-3-5-3
CCF Intensive Review
of IM 1997
64 361 88 audio & slide lectures $1,295 N/A
IM Interactive Text and Review 0 1,100+ Kelley's IM 3rd Ed. $1504 5-4-2-5-4
Med-Challenger IM 143
1st 50 free
2,700+ None $425 3-5-3-5-5
Obstetrics and Gynecology
Interactive Review of
Obstetrics and Gynecology
0 0 80 lecture articles and outlines $199 5-3-1-5-4
AAP PREP 0 1,500 10 yrs. PIR $195
CCF Pediatric
Board Review
44.5 300 66 audio & slide lectures $699 4-4-3-5-3
Med-Challenger Peds. 155 AAP
1st 50 free
3,000+ None $520 3-5-3-5-5
Psychiatry and Neurology (no products reviewed yet)
ACS SESAP 60 650 None $4504 5-5-5-5-4
Surgery 0 1400 Greenfield's Surgery 3rd Ed.
Nyhus' Surgery 2nd Ed.
$2494 5-4-5-5-4

1.All AMA Category 1 with others as noted. CME costs included in list price except where noted.
2.Questions are those designed specifically for review purposes, even if eligible for CME. Text here refers to articles, chapters, or lectures independent of test questions or answers. Those without text intend the question/answers to be used as content database.
3.Rated for content-programming-CAI-error handling-support on a scale from 1 = poor to 5 = superior. Full explanation in body of article.
4.Price less for members or students.
N/A = not applicable

Return to Top of Comparative Chart of Software Features
Emergency Medicine
Family Practice
Internal Medicine
Obstetrics and Gynecology
Psychiatry and Neurology (no products reviewed yet)
1.All AMA Category 1 with others as noted. CME costs included in list price except where noted.
2.Questions are those designed specifically for review purposes, even if eligible for CME. Text here refers to articles, chapters, or lectures independent of test questions or answers. Those without text intend the question/answers to be used as content database.
3.Rated for content-programming-CAI-error handling-support on a scale from 1 = poor to 5 = superior. Full explanation in body of article.
4.Price less for members or students.
N/A = not applicable

Return to Top of Comparative Chart of Software Features