The American College of Laboratory Animal Medicine (ACLAM) undertook an intensive review of its certification examination program in 2006. As a result of this process, there have been two significant changes to the examination process:

 

1. The first was to more closely align the examination to the Role Delineation Document (RDD) [63 KB PDF]. This was accomplished using the 2002 Role delinieation document for recent examinations through 2008. During 2007-2008, ACLAM reviewed and revised its RDD. The 2008 RDD describes six domains. Within each domain are knowledge and task proficiencies that are required by an ACLAM Diplomate. In addition, the RDD defines the animal species that make up our patient population. The RDD was used to create a Test Template [55 KB PDF] for the certification examination, which provides examination weightings for the domains and species defined in the RDD. The 2008 Test Template will be used to formulate the ACLAM certification examination beginning in 2009.
   
   
2. The second change was to adopt a peer-reviewed criterion referenced passing standard for the examination. The purpose of this change is to assure that successful candidates possess the knowledge necessary for entry level members of our specialty and that the passing standard for the examination is the same for each year the examination is administered. ACLAM conducted criterion-referenced standard setting sessions in 2006 and in 2008.

 

Frequently Asked Questions (FAQ)

Terms, abbreviations and definitions:

ABVS:  American Board of Veterinary Specialties – the group which advises the American Veterinary Medical Association and sets standards for groups of veterinary specialists, such as ACLAM.  ACLAM has a representative on ABVS, and must make yearly reports to ABVS.

Angoff modified technique:  one of a number of techniques used by psychometricians to conduct criterion-referenced standard setting studies.

BOD:  Board of Directors

COC:  Certification Oversight Committee – advises BOD on all matters related to certification, including oversight of Examination Committee, and others

Criterion referenced:  exams in which the passing point is based on adequate demonstration of established skills, tasks or knowledge areas (as distinct from norm referenced exams, which are “graded on the curve”)

Cut score:  the score a candidate must obtain to pass an exam – also known as the pass(ing) point – not to be confused with the pass(ing) rate.

Equating:  any of a number of statistical methods used to compare one exam with another, based on evaluation of test items in common between the 2 exams;  linear equating is one such method.  Equating allows different exams to be made equivalent in difficulty level, within the limits of the statistical analysis.

Equator items:  A set of questions that is identical in form and placement on 2 different exams, necessary to conduct a linear equating procedure.

Norm referenced:  exams scored such that the passing point is set by statistical analysis of the performance of the candidates, regardless of the abilities of the candidates, i.e., “grading on the curve”.

Pass point:  the score a candidate must obtain to pass an exam – also known as the cut score – not to be confused with the pass(ing) rate.

Passing rate:  the percentage of candidates passing the exam, i.e., the percentage of candidates taking an exam who attained the passing point (cut score).

Psychometrician:  a professional with training in test design, construction, administration, and evaluation – often holding a PhD in statistics, or educational theory.

RDD - Role Delineation Document = the job analysis of the board certified Diplomate member of ACLAM, established in 1997 by vote of ACLAM members, with major revision in 2002, and a current major revision pending completion and analysis of a members survey. 

SSS – Standard Setting Study = the meeting of a panel of Diplomates selected to conduct a criterion-referenced establishment of the cut score (passing point) for the 2006 Exam.  This took place in the week after the 2006 Exam was administered, during the 2006 ACLAM Forum, and was facilitated by the CASTLE consultant.

QUESTIONS:

What is the rationale underlying ACLAM’s reconfiguration of the certifying examination?

ACLAM must review our examination on a regular basis to make certain that our examination process is fair, standardized, legal, defensible, valid, and effective in certifying laboratory animal veterinarians who have sufficient knowledge and skills to conduct the safe and effective practice of laboratory animal medicine, as defined by our Role Delineation Document. (These concerns were made more important by the advice from the American Board of Veterinary Specialties recommendations on this issue, as reported by ACLAM's representatives to the ABVS.)  In the past, the pass rate has been highly variable and ACLAM has been unable to determine if the cause is the exam or the quality of the candidates' preparation.  Several years ago, the ACLAM BOD approved the retention of a professional examination consultant firm to provide ACLAM with expert advice on its certification examination and examination process.  This resulted in hiring CASTLE Worldwide, Inc., who evaluated our examination with respect to established "psychometric" standards in the industry of certification examinations, and spent a full day with a select panel of Diplomates at the 2004 ACLAM Forum to review the certification process used by ACLAM.  CASTLE presented 12 recommendations to the BOD in 2005 for improvement of our examination process, including 6 that the consultant considered High Priority.  In June of 2005, President Kathryn Bayne appointed a committee to provide an implementation plan for changes to the examination process recommended by the consultant.

What have been the major steps taken to implement the consultant’s report?

The Reconfiguration Committee met through teleconferencing multiple times with the consultant in 2005, eventually submitting action items to the BOD giving recommendations for the implementation of consultant recommendations.  Chief among these were (1) the need to closely align the certifying examination with the Role Delineation Document (RDD), in order to base the exam on our accepted list of skills and knowledge used by a certified laboratory animal veterinarian, and (2) the need to base the exam scoring (pass point) on criterion-referenced standards through the use of a Standard Setting Study (SSS), eliminating the fixed 66% pass point, which the consultant advised was not legally defensible. (Details on each of these below.)

How was the exam aligned with the RDD?

The Exam Committee and Exam Review Committee met with the consultant in early 2006 in order to review the effective alignment of test items with the current RDD, which describes the tasks a board-certified laboratory animal veterinarian should be able to complete, and the knowledge and skills a board-certified laboratory animal veterinarian should use to complete them.  The alignment of test items was accomplished with no more than 1% variance from the RDD task distribution. All reference to “traditional” 4 categories was removed, in order to emphasize reliance on the RDD. The BOD approved the balancing of the total exam (not the individual written and practical exams) to the RDD, effective with the 2007 exam, at the recommendation of the CASTLE consultant.

Why do we need the role delineation document for the exam, and how is information on the RDD used in designing the exam?

The CASTLE Report (2005) stated:  “…ACLAM is vulnerable to criticism and perhaps litigation in several areas, especially the lack of direct linkage between our task analysis (the RDD) and the exam…”  Recommendation #2 from that report stated:  “It is very important when assembling new forms of the examination for ACLAM to employ the specifications derived from the Role Delineation Document (High Priority)” .  Recommendation #3 from that report stated:  “In implementing specifications based on the Role Delineation Document, ACLAM should use only the percentages expressed, without varying within the limits indicated.  Following this recommendation will serve to limit content sampling bias even further.”  After the BOD voted to implement these recommendations for the 2006 Exam (and subsequent exams), the Exam Committee, with help from the Exam Review Committee, spent considerable time properly assigning a single task from the RDD to every test item in the ACLAM database, and assuring that each examination is balanced to the RDD as indicated.

Are both the written and practical exams individually balanced across the RDD?  Did this change recently?  Why?

The issue of changing the RDD distribution to be spread over both exams was a recommendation from the Exam Committee and the Reconfiguration Subcommittee, and it was recommended by our consultant. A report from the CASTLE Worldwide consultant in 9/06 stated:  “In discussions that followed the 2004 Report, CASTLE advised ACLAM that each part of the test (written and practical) should adhere to test specifications. Now, having attended an Examination Review Committee meeting and conducted the standard setting study, CASTLE has gained greater familiarity with the examination and modifies its advice to ACLAM: the optimal strategy would be a single examination containing both item types, fully aligned with specifications based on the Role Delineation Document and having a single, overall passing standard. The rationale for the new advice is that various areas of the Role Delineation Document can not reasonably be tested in the practical portion of the examination. By combining the two parts of the examination, ACLAM can assess specified content using the most appropriate item type.” This recommendation was submitted to the BOD in 9/06, and was approved . Along with this, the BOD approved the elimination of the allowance to pass one part of the exam only, effective 2010,  thus allowing a full 3 years for anyone already planning to sit the 2007 exam for the first time. This announcement was made on the website.

Is the RDD still relevant?

Since the first implementation of the RDD in 1997, the Certification Oversight Committee (COC) has been charged with reviewing the document annually, and convening an appropriately constituted review group every 5 years for more comprehensive review.

For the 2007 RDD review, ACLAM has retained the services of a professional testing and consultation service to assist the COC in this process. The most important step in the process was to design and construct a Role Delineation Survey that will be administered to all members of the College. A task force of 19 ACLAM Diplomates who represent a cross section of the College demographics met with the consultants on May 9 and 10, 2007, in Tucson, Arizona, at the site of the Forum.  This group was charged with ensuring that the content of the survey appropriately probes the specialty of laboratory animal medicine, such that the results will be the most accurate representation possible for our specialty.  The RDD survey was delivered soon after the Forum to each Diplomate in an electronic format for response within about one month.  The results of the survey will instruct the efforts of the panel to reconvene, and revise, as necessary, the current RDD.

Why the emphasis on the RDD?  What else does the BOD do with the RDD?

Most of the following description was sent to all Diplomates by the Webmaster, Steve Fisk, on 3/16/07, as part of a communication from the Certification Oversight Committee:  The RDD defines the knowledge, skills, and abilities required to be considered an ACLAM-certified laboratory animal specialist. The RDD was first developed and written in 1997 during a meeting of a diverse group of ACLAM Diplomates who identified and characterized the job tasks and knowledge areas of our specialty. In addition, the RDD defined the species distribution best representing the “patient population” of an ACLAM Diplomate. The RDD was finalized after a survey of all Diplomates was administered and analyzed. In 2002, the first comprehensive RDD review resulted in several changes to the original, and is the version currently used by the College. The current RDD is available on the ACLAM web site at:  http://www.aclam.org/print/RDD_11-15-2005.pdf.

Originally, the RDD became the standardizing template for designing ACLAM-recognized training programs. Since that time, the RDD has evolved into an examination template against which the number and categories of questions included on the ACLAM certification examination must be aligned. The RDD defines our specialty, and will continue to be used in the future to balance our certification examination by tasks, knowledge areas, and species, and to provide the standard for our recognized training programs. It is also possible in future years that the RDD will be used more specifically as a basis for Continuing Education efforts by the College. As you all know, our specialty changes over the years, sometimes quickly and sometimes slowly, depending on the issue. A periodic review and update of the RDD insures that our specialty is defined accurately and appropriately in the future.

What is criterion referencing, and how does it apply to the ACLAM Certification Exam?

Criterion referencing means that an exam is based on specific knowledge criteria, and not on comparative performance of the candidates (norm referencing – that is, “grading on the curve”). It is contrary to recommendations from the American Board of Veterinary Specialties and to recommendations of the expert community of certification examination experts, such as our consultants, to use norm referencing – otherwise known as “grading on the curve”.  General veterinary licensure examinations eliminated “grading on the curve” over 15 years ago.  Norm referencing (“grading on the curve”) is simply no longer acceptable for high stakes certification examinations in any field.  Our exam consultants recommended that our exam must be strongly linked to the Role Delineation Document, and have the passing point set by psychometrically accepted methods, in order to accomplish this.

How was criterion referencing accomplished?

A 14 member Standard Setting Study panel, comprised primarily of recent Diplomates of the College (on the advice of the consultant), was selected. The CASTLE  consultant strongly recommended that panel members closer to the point of certification themselves would be better able, with instructions and discussion, to formulate and judge the required concept of performance of a candidate that is at the minimal level for certification.  Other criteria that members of the panel filled, in compliance with deliberations of the COC Subcommittee to Implement Exam Consultant Recommendations, were the following:

No member, or at most 1, will have any involvement in ACLAM Exam production and development within the last 10 years.



• At least 2 members will be involved in training programs for ACLAM candidates.
• At least 2 members will be Diplomates who work in government (including military) or industry (including animal vendors).
• At least 2 members will have qualified for boards through the alternate (experience) route.
• Members will fit multiple categories, but we will make certain at least one member has some emphasis in each of the primary species (especially a non-human primate person and a mouse person).

This panel met one week after the administration of the 2006 Examination, in order to set a criterion-referenced passing score (cut score) for the 2006 Exam.  (See FAQ #11 below for description of the Standard Setting Study)  The consultant reviewed the scoring of the 2006 exam, conducted a statistical analysis of the resulting data, and made statistical recommendations for the passing score to the BOD.

What is a cut score  - also known as the passing point?  Why was the old cut score of 66% not good enough?

A cut score is the score that determines pass or fail on the certifying examination.  The historic standard of 66% was the judgment of our predecessors, based on their assessment of the performance of a minimally-competent, but board-certifiable laboratory animal veterinarian on the exams they used for certification.  The problem with this method is that it is an entirely arbitrary standard.  Also, we were not controlling the difficulty level of the exam, other than selecting test items with Difficulty Index of 30-70% (which means the percentage of candidates who correctly answered the test item in previous administration).  In addition, at times in years past, the 66% cut score was adjusted on a given examination, based on the performance of that year’s candidate group.  The CASTLE Report (2005) stated:  “…ACLAM is vulnerable to criticism and perhaps litigation in several areas, especially …..the lack of a defensible method to determine the cut-score;  the latter has been identified also by ABVS.”  Recommendation #4 from that report stated:  “ACLAM should conduct a professional Standard Setting Study (SSS) using an appropriate criterion-referenced method, with a full and complete report of the study and its results (High Priority).”  The high stakes certification decision of pass or fail needs to be based on specific criteria and the standard should not vary from year to year.  Thus, ACLAM changed from a fixed passing point (cut score) of 66% to the use of a Standard Setting Study for the 2006 Examination.

How do we determine the cut score now?

The Angoff Modified Technique is the method ACLAM used in 2006 to determine the passing standard. This procedure is the most widely employed and one of the most thoroughly researched in the field of licensure and certification testing. Using the Angoff Modified Technique, a group of ACLAM Diplomates, chosen to represent the spectrum of activities in our specialty (see FAQ #9), served as subject matter experts in a peer review session to evaluate the exam. The panel evaluated each item on the exam by estimating the percentage of the minimally acceptable candidate group that would answer the question correctly. In other words, each member of the selected panel conceptualizes whether a hypothetical candidate they would consider to meet the minimum standard for the ACLAM profession would be likely to answer the question correctly. These estimates, which the panelists made independently, were then combined across all panelists and questions, the result of which represented the minimally acceptable score. The peer review session for standard setting took place at the 2006 ACLAM Forum with the assistance and guidance of CASTLE Worldwide, which has conducted many of these studies for specialty certification organizations since 1987. CASTLE tabulated and analyzed the findings, and presented the results to the BOD for approval.  The consultant’s advice was that the Standard Setting Study should be repeated again whenever there are significant changes in the Role Delineation Document that alter the character of the examination.  This may be expected to occur every 5 to 7 years.  The method to assure consistency of the pass point from year to year (between standard setting studies) is called equating, and a description is found in FAQ #12. 

Because the Standard Setting Study (SSS)’s establishment of the cut score (passing point) is so critical to validity of the exam, the process was recently reviewed by a second consultant psychometrician. Interviews with members of the 2006 SSS have confirmed the confidence of that group in their work product. The second psychometrician’s opinion is that the SSS was properly conducted.

How does ACLAM ensure that there are no major differences in exam difficulty and fairness from one year to another?

Based on the recommendations of the Exam Consultant, ACLAM uses a process called equating.  The following description from our consultant, CASTLE Worldwide, explains the process:  “ACLAM established a criterion-referenced passing standard for each part of the test in June 2006, using versions of the test administered at that time.  Because new versions of the examinations may be either easier or more difficult than the 2006 versions, ACLAM contracted with CASTLE to equate the new versions introduced in 2007 to the 2006 examinations in order to correct for these differences. On behalf of ACLAM, CASTLE uses accepted statistical methods to ensure that candidates are not penalized or rewarded when one version of a test is more or less difficult than another version.”

In preparing for the study, ACLAM and CASTLE worked together to review item analysis statistics and identified a set of high quality questions for both the written and practical from the anchor versions of the test (2006 versions) to be used as the equating pool (embedded equator items) for the 2007 Examination. In addition, each new version of the examination has questions that are unique to it because these items were not included in the 2006 forms. CASTLE instructed ACLAM to place the equator items in the 2007 tests in the same location that they occupied sequentially in the 2006 examinations to control for the effect of candidate fatigue.  (Equating questions do not need to stay the same on every subsequent form of the exam, as long as they are unchanged from the anchor exam (in this case, the 2006 Exam) – in other words, we may choose a different set of equators for the 2008 Exam versus the 2007 Exam.)
The next step in preparing for the study involved scoring the anchor examination in CASTLE's system. ACLAM provided CASTLE with the answer strings (electronic files containing the answers candidates provided for each question along with the candidate identification number) for the written and practical tests for 2006. Using the answer keys, CASTLE scored these examinations. ACLAM administered new versions of its examinations on July 8, 2007, scanned the answer sheets, and sent the 2007 answer strings to CASTLE. Using the answer key for the 2007 examinations, CASTLE scored each test in its system to generate raw scores for each candidate and then conducted the equating study, consistent with the Tucker model for linear equating for nonrandom groups and an embedded group of equator items. The calculation employs a complex linear regression formula to identify and control variance due to the difference in the ability of the two groups, and to isolate the difference in item difficulty between the two versions. Based on these calculations, CASTLE determined the score that candidates taking a new version would have achieved if they had taken the anchor version of the examinations. The BOD approved the equating studies for the written and practical examinations and approved equated scores for 2007 candidates, determining the pass/fail decision.

What other changes have been made in the exam?

The number of items on the written test was reduced from over 400 to 240.  The number of items on the practical test was reduced from over 200 to 140.   All questions were written in multiple choice format.  The restrictive advice of 50% newly written questions was removed, in order to use more questions with known performance characteristics.      The 2007 Examination had a number of exact duplicate questions from the 2006 Exam to use for statistical equating (see FAQ #12 for a description of the equating process), and (for the first time) the use of color photographs in the practical test booklet, rather than projected images. The EC made this decision because there was a record number of people taking the practical, and they didn’t think they would be able to have equally good seats in the test room to see projected images. Secondarily, it fits in with the plan to combine the two exams in 2010. The EC also believed this would be considered a positive change by the candidates, by removing the time pressure.  There has been no change in the allotted time allowed for candidates to complete the examination, in spite of the shortening of the exam.  The BOD has approved the requirement that the written and practical components of the exam will be combined in 2010, so that passing “all or none” will be necessary.  This was a recommendation of the consultant, realizing that the only differences in the exams were the addition of pictures, and the balance across the RDD could only be effectively secured by balancing across both exams.  The introduction of this requirement in 2010 assures that all candidates who were promised “3 tries” at the exam, with partial passing as a possibility, will have that opportunity using the existing system. 

Why did ACLAM eliminate the Exam Review sessions which used to provide
Continuing Education to Diplomates?

Our consultants have advised us that the dual purpose of certification AND continuing education is not appropriate for a high-stakes certification exam.  This exam should serve a single purpose – that is, to distinguish a board-certified specialist from those who do not qualify.  Failure to adhere to adequate security measures can result in sharing, with some candidates, of test context, materials covered, and potentially even test items, intentionally or unintentionally, making the examination unfair for others, especially those seeking certification by the experience route.  In addition, the requirement for members of the College to produce large numbers of new test items every year strains the abilities of the Exam Committee and other associated committees to produce high quality test items, threatening the quality of the overall exam.  Most importantly, the consultant advised that we are NOT in compliance with accepted (and legally defensible) psychometric standards if we continue to use the exam for multiple purposes.  While releasing sample questions may be appropriate, proper exam security demands that only those committees involved with preparing and reviewing the exams for content validity be given access.

Why can’t Training Program Directors review the exam each year?  If they can’t see the exam, how do they know what to teach?

Our testing consultants strongly recommend that the examination should be used only for certification purposes, and NOT as Continuing Education.  Since the exam is directly related to the RDD, Diplomates do not need to see the exam to know what is covered by the exam.  Training Program Directors, who are among the most experienced Diplomates in the College, and have many years of experience in preparing candidates, should continue to prepare those candidates in the ways that have a history of success, using source materials that, in their experience, cover the material in the RDD well.

Does the reuse of test items to a greater extent mean that there will be no questions, or fewer questions, covering recent journal articles?

Recent retrospective analysis indicates that the 2005 Exam contained 41.6% new questions, the 2006 Exam contained 23% new questions, and the 2007 Exam contained 25.5% new questions. The advice given on the ACLAM website this year was that “Roughly 66% of the questions are derived from the test data base.  This allows for 33% of the questions as new.  Often these new questions are drawn from current literature and recent publications.  Additionally, new questions may also be added from standard laboratory animal resources (i.e., textbooks and regulations).  Prior to 2005, exams were considerably longer with a higher percentage of new questions.  The current exam format of less questions and therefore less new questions may have given the impression that current literature was not important or necessary to review.  This is not true.  Questions are still generated from current literature but we attempt to focus all questions to the important topics of laboratory animal medicine as highlighted in the ACLAM Role Delineation Document.”  In future years, there is no guarantee of new question content, but, depending on the significance of new publications in the field, new questions will certainly be included to cover material appropriately linked to the RDD.  The candidate should continue to study new publications with a critical eye on the RDD. The consultant’s strong recommendation was that no new questions be used for scoring on the exams.  Although we are not yet able to follow this recommendation, future Exam Committees will build the database with “test items” (also known as test market items or preliminary test items)used to gather statistics for evaluation of future use – if any such items are used, they will NOT be scored for or against candidates scores – this could potentially add a number of questions to future exams, making them slightly longer.  The 2006 Exam did not contain any new items that were scored only for statistical purposes, and the 2007 Exam contained only 4 such items – these items were not scored for or against the candidates, and were completely ignored in initial scoring of candidates and the equating process used to establish the passing point for the 2007 Exam. The consultant has also recommended that there be no questions based solely on new research that does not form part of our professionally-accepted body of knowledge. If we were to ask a question based on recent research which was subsequently shown to be invalid, this would jeopardize the validity of the examination.  This advice was followed for 2006 and 2007 Exams, with the result of eliminating questions that simply require rote memorization from a recent journal article.

What is the recommended list of study references?  The suggested reference lists on the ACLAM web page are not the most current and are very broad.  Likewise the lists of references in the RDD are very broad and not always the most realistic (e.g., the IATA manual – how many people have access to that?).

ACLAM has published lists of references recommended for study on multiple occasions, and has tried to update this list intermittently. The recommended list of references is on the ACLAM website under the “Studying for Exam” section located within the “Certification” tab. References are categorized as “Primary” and “Supplemental” materials/resources.” Candidates should continue to concentrate on traditional study materials from which the majority of test questions are derived, including the list indicated as Primary Materials in this document.  The list of Primary Materials is up to date, including the most recent editions of those materials.  The list of Supplemental Materials is admittedly very broad, and some of the references are 10-20 years old, but they represent important sources of material for the laboratory animal veterinarian.
In addition, the Exam Committee (or other assigned committee) is considering a project in which they would review recent exams and post the commonly used references used for exam questions. 

What are the core references?

See the ACLAM website, under the list entitled Primary Materials.

What percentage of test questions cover the core references?

There is no longer an Exam Committee policy/process to delimit “core” questions, or to fix a certain percentage of core questions on the exam. The distribution of exam questions must follow the RDD, and certainly the majority of exam questions will come from the Primary materials referred to above.

Is it true that every question must have 2 references?

Yes.  In recent years, guidance within the Examination committee has been to assure that 2 references exist for answers to questions, or that the information contained within the single reference is widely accepted as important, confirmed knowledge within the field.  The Exam Committee’s aim has been to eliminate any questions covering material that is not yet widely accepted in the field, because such questions might need to be discarded in future years if the source is disproved.

Which journals are important, and for how many years back are they covered?

The recommended list of references is on the ACLAM website under the “Studying for Exam” section located within the “Certification” tab. References are categorized as “Primary” and “Supplemental” materials/resources.   Important topics or information contained in journals that are more than 3 years past publication usually will have been incorporated into a newer text edition that appears on the Primary Materials list, or will have been mentioned in subsequent, and more recent journal articles elaborating on the subject.  For this reason, it is unlikely that a single “fact” found only in a journal more than 3 years old, would form the basis of a question linked to the RDD.

Are classical animal models and AFIP Fascicles covered?

Knowledge of spontaneous and induced animal models is part of Task 9. The AFIP Fascicles are no longer used as primary references for exam items. Nearly all current test items have more than a single reference, i.e. from a recent journal article with support from a primary source text.  ACLAM does not ask “history” questions simply for the sake of knowledge of history, but some older animal models are still used, or form an important knowledge base for animal models that are later developed.

What reference can be used to study the statistics that may be on the exam?

Any good college level statistics textbook should provide the basic biostatistical information important for a laboratory animal veterinarian to function.  In addition, the current edition of Laboratory Animal Medicine (pp. 384-386, in Chapter 10) and many of the older ACLAM Blue Book series texts have chapters describing sampling techniques, sentinel planning, epidemiology, and other topics containing applied statistical methods important for the field.  ACLAM does not list a specific text for many general subjects that laboratory animal veterinarians should be familiar with.

Why do the exam questions focus on obscure information instead of focusing on the overall conclusions of published articles?

The Exam Committee in recent years has made a concerted effort to eliminate questions that ask “in a recently published article, what was …………?”.  Questions are not asked about journal articles just to “cover” the journals, but rather, questions are asked about important concepts detailed in the RDD that are elucidated by journal articles. We may actually use a reference to a commonly-known fact that any minimally qualified Diplomate should know, backed up by other references, as a test item.

Since most current animal facilities do not have parasites other than pinworms/mites, why does this exam test for historical ones?

 Parasitology, especially parasitic diseases that can become established in a colony and zoonotic parasitic diseases” is part of Task 1 of the RDD.  Although modern animal facilities obtain most of their animals from well-managed vendors who provide parasite-free animals in order to minimize research variables, the world at large is not so controlled.  Laboratory animal veterinarians still need to work with non-purpose bred animals or wild-caught animals in some situations.  Opportunities for cross-contamination or import of parasitic organisms no longer frequently seen makes the knowledge of parasites relevant for the certified laboratory animal veterinarian.  One might ask the same question about bacteria, viruses, fungi, or other pathogenic organisms successfully screened by our high quality vendors and facilities, and the answer would be the same.  Unless and until such pathogens are completely eradicated, the trained professional must have some knowledge of them, although the emphasis will certainly be heavier on those encountered regularly.

Why does the exam not reflect information/skills needed to be a laboratory animal veterinarian?

The College has tried very hard in recent years to assure the fairness and relevance of the certifying examination, by making certain that the RDD lists the tasks, knowledge and skills required in the field, and that the test items are carefully linked to the RDD. Diplomates should participate in the review/revision of the RDD, as discussed above.

Could current ACLAM Diplomates really pass the examination?

The simple answer is yes.  The conditions would be that the Diplomate would have to study in order to pass the certifying examination.  This is a high stakes examination designed to qualify entry level specialists into the field, and it is intentionally difficult in order to assure the capabilities of any who hold the title of Diplomate.  As with any field after the certification hurdle has been cleared, the Diplomate may have concentrated on any of a myriad of subspecialty areas in the field, such as administration, facility design, special species, surgery, viral research, etc.  Without studying the broadly based material required for entry into the field, that experienced Diplomate might not pass the rigorous entry exam.  However, each and every Diplomate has demonstrated the necessary entry level knowledge and skills once in their career, and each continues to meet the College’s approved standards for recertification in order to maintain Diplomate status.  The College decided a number of years ago that reexamination was not a necessary standard for recertification.

Is the tested information necessary to be a successful laboratory animal veterinarian?

The exam is based on the RDD, which opens with the following statement: The board certified laboratory animal medicine specialist is a veterinarian with specialized training and experience whose mission is to advance the humane care and responsible use of laboratory animals. This mission is fulfilled through the application of specific knowledge to safely and effectively perform the tasks listed [in the RDD].

Is there any correlation between pass rate and completion of a training program versus experience route?

This interesting question has significance for the College’s “pipeline” issue, especially in the face of changing support for many approved training programs and the appearance of many new training programs.  Unfortunately, correlative statistics needed to answer this question beyond speculative opinions are minimal.  The BOD has recently directed a select group of Diplomates to attempt to retrieve data from past years and analyze this question.  The BOD hopes to be able to report on this study at the national AALAS meeting in November, 2007.

Is there any correlation between job performance as rated by one’s employer and pass rate on the examination?

This is an interesting question.  We all hope that Diplomates perform well in their jobs, and are rated highly.  We believe that our approved training programs or experience route evaluations, and the subsequent certification process, identify individuals who have proved their competence in the field of laboratory animal medicine.  The purpose of the exam is only to distinguish the non-qualified from the qualified based upon standards set by the College.  Individual job performance is so dependent on local factors that an objective analysis such as suggested by the question would be difficult or impossible.

What are some other veterinary specialties doing?

This was one of the first questions from our BOD to our initial consultant. The following information is quoted from the 2005 CASTLE Report, commissioned by ACLAM:  “The first objective specified by ACLAM relates to benchmarking its examination program to the test development practices of other ABVS-approved entities….to determine the degree to which other approved veterinary specialty examinations adhere to generally accepted psychometric standards….Generally speaking, veterinary specialty boards are not well aligned to NCCA standards for certification examinations.  Only the American College of Veterinary Ophthalmologists meets the majority of standards, while others have significant shortcomings.” ( NCCA is the National Commission for Certifying Agencies, which is the accrediting arm of the national Organization for Competency Assurance (NOCA), established in the 1970s under direction and financing from the Bureau of Health Professions of the (then) United States Department of Health.  References to these contemporary psychometric methods (and their professional organizations) including websites, books, key journal articles have been obtained and studied by members of ACLAM involved with these procedures to assure that we are using consultants to employ contemporary statistical methods for exam development, scoring, and interpretation. A variety of consulting organizations exist, in addition to the 2 major consultants who were chosen, but ACLAM members are convinced that other consultants would have used similar accepted methods.)

The American Board of Veterinary Specialties has hosted workshops for their constituent colleges (including 4 ACLAM Diplomates each year) in February, 2006, and February, 2007, on the subject of certification processes, in an attempt to encourage the specialty colleges to better meet ABVS standards.  The 2006 day-long presentation was given by our CASTLE Worldwide consultant, who discussed how specialty examinations should meet commonly accepted psychometric standards, especially as related to certification examination standards.  The 2007 day-long presentation was given by our Thomson Prometric consultant, discussing job analysis (role delineation).

ACVP is creating a Role Delineation Survey, will create Role Delineation Documents for each specialty, and will evaluate their examination process based on their Role Delineation Document. The pathology certification process must address the validation of minimal passing scores just as ACLAM has done recently. Creating a fair, relevant, legally defensible examination is now one of ACVP’s highest priorities. ACVP faces similar challenges explaining to Diplomates why this process is necessary and why the examination process must be periodically evaluated and occasionally modified.
It is apparent that ACLAM is being proactive, compared to other veterinary specialties, but that all will be following in the footsteps of our efforts.

Will changes in the exam reduce or raise the bar to entry into ACLAM?

The changes in the exam and examination process were not implemented with a goal of either lowering or raising the bar for ACLAM certification, but rather, at the problems of fairness, consistency, and legal defensibility. The actual pass rates on the exam in 2006 and 2007 (22% practical, 58.5% practical, and 24% written, 54% practical, respectively) were lower than most Diplomates would like. However, the standard for passing the exam is now established by psychometrically accepted methods, based on the efforts of a select Standard Setting Panel (see FAQ #11 on Standard Setting Study).  Future exams are assured of consistency with the process of equating (see FAQ #12 on equating), thus assuring a level playing field for all candidates. The COC has been charged recently with conducting a comprehensive analysis of the exam success rate, to include not only exam-related factors, but also possible changes in the candidate pool, training programs, and availability of training resources.

Won’t a lower exam success rate impact the critical need for new ACLAM Diplomates?

The “pipeline issue” is of concern to all Diplomates, and remains a priority of the BOD, the Career Pathways Committee, and others. As above, the changes taking place in the certifying examination are not aimed at addressing this problem, but at improving the exam process itself.

Will any sample questions be made available?

This question is still under consideration.  Unlike the SAT, ACT or MCAT exams, or other standardized exams with large databases, ACLAM does not have an extremely large database of validated test items.  Some of the study group organizations, such as LABSG and Camp ACLAM, do a good job of producing “exam-like” questions. 

Can the candidates get additional information regarding how they did?  Providing information about each task, especially since the RDD is touted as the guideline to be used in studying, would be more valuable.

The Exam Committee produced reports of performance on groups of tasks from the RDD which were sent to failed candidates following the 2006 and 2007 Exams.  Numeric score reports were not issued, in accordance with the statement posted nearly two years ago, after vote by the BOD: “ The purpose of the ACLAM examination is to determine qualification in laboratory animal medicine. Consistent with this purpose, the passing standard for the test must separate individuals who possess knowledge at the level commensurate with Diplomate status from those whose knowledge is below that level. Essentially, the examination is used only to make pass/fail decisions. ACLAM’s position is that all who pass are equally qualified, and so no numeric scores will be released.”  Candidates who fail to attain certification have expressed a wish to have more definitive information provided regarding their performance on individual RDD tasks, in order to help guide future studying.  The Exam Committee is investigating statistical and scoring software packages that would allow for a more detailed analysis to be shared with candidates who fail.  The BOD will consider other suggestions for release of information, such as release of  the size of the gap between the passing point and the candidate’s performance.

Why hasn’t the College made the exam changes public?

The College has endeavored to keep Diplomates apprised of all changes in the exam process, through announcements in the newsletter and in General Business Meetings, special Town Meetings held at AALAS and AVMA meetings and the ACLAM Forum (3 times since the 2006 Exam was given), and changes posted on the web site. This FAQ document is the latest of these efforts, and will be followed shortly by the dissemination of a public Exam Committee Procedures Document which will include this and additional information intended to fully disclose how the ACLAM exam is constructed and scored.   Another Town Hall meeting is planned for the National AALAS meeting in October, 2007.  Continued opportunities to become involved in efforts aimed at certification examination improvements, and related issues, are available to all Diplomates

Copyright © ACLAM. All Right Reserved.
Copyright and Disclaimer | Credits and Acknowledgements