|
 |
News@WSLH
Your
Source For News & Events From Wisconsin's Public Health
and Environmental Laboratory |
 |
|
| |
Interlaboratory
Surveys To Assess Performance: More than Proficiency Testing
The following article is from the
Medical Laboratory Observer (MLO) for January
2006 (PDF, 199 KB), pages 20-21, 24-25.
By David J. Hassemer
With the implementation of the Clinical Laboratory Improvement Amendments
of 1988 (CLIA 88), proficiency testing (PT) programs assumed an
even greater regulatory role than held previously. PT programs —
initially provided for educational purposes — were replaced
by programs designed solely to meet regulatory or accreditation
needs. Unfortunately, many of these educational programs, ones often
developed by states to meet a specific need, were eliminated. It
became more difficult for laboratories to justify participating
in an interlaboratory PT program on a voluntary basis or for educational
purposes. Accreditation organizations would require the lab to participate
in PT, to the extent available, whether or not it was for a “regulated”
analyte. The focus of proficiency-testing programs changed to meet
these new mandates. Successful laboratory performance, as defined
by the new rules, became the important issue. “How well did
we perform?” and “What did we learn?” were replaced
with “Did we pass or fail?”
Several states established
proficiency testing programs to
meet regulatory and educational objectives. Wisconsin saw its PT
program, available to laboratories throughout the United States,
as being more than a program designed to meet CLIA requirements.
As a full-service proficiency-testing provider approved by HCFA
(Health Care Financing Administration; now, the Centers for Medicare
and Medicaid Services or CMS), the Wisconsin State Laboratory of
Hygiene proficiency
testing (WSLH PT) program also made changes to bring it into compliance
with CLIA 88. The bigger challenge facing WSLH PT was how to maintain
interlaboratory PT programs that had been developed for a specific
population (or need) of laboratories and where the focus was more
educational than regulatory. Important components of WSLH PT programs
have always been the education element and the readily available
consultation that is a vital part of PT service. Our PT program
is a division of WSLH — the state’s public and environmental
health laboratory.
The WSLH is attached to the University of Wisconsin-Madison, and
many of our senior scientists are faculty and academic staff at
the university. This relationship provides WSLH PT with access to
the full array of a large laboratory’s resources. This enables
us to continue to offer unique interlaboratory PT programs along
with those traditional programs that are designed to meet federal
and state regulations, and
accreditation organizations such as The College of American Pathologists
(CAP), Joint Commission on Accreditation of Healthcare Organizations
(JCAHO), and COLA. Our range of analytes in the PT program covers
the usual laboratory disciplines including chemistry, hematology,
immunology, and microbiology (see listing on our website, www.wslhpt.org).
The following describes some of the special programs offered by
WSLH PT and the role they play in the clinical laboratory
community.
Blood Lead
Lead poisoning has long been recognized as a major public health
threat for children and occupationally exposed workers. A Centers
for Disease Control and Prevention (CDC) statement issued in 1991
lowering the blood-lead concentration level deemed harmful to children
had the dual effect of 1) rendering the erythrocyte protoporphyrin
test unsuitable for lead-screening purposes because of poor specificity
and
sensitivity, and 2) challenging laboratories to use a direct test
for measuring blood-lead concentration that would produce accurate
results, both for screening and diagnostic purposes. This challenge
occurred about the same time the CLIA 88 regulations were being
implemented, hence the need for a proficiency-testing program —
not only to help labs meet the regulatory requirements, but also
to help demonstrate improved analytical performance and assess new
methods.
In 1987, the WSLH, in
cooperation with the federal Bureau of Maternal and Child Health,
assumed administration of a blood-lead PT program, which had previously
been run by the CDC. The WSLH PT program was originally set up to
send out three unknowns monthly. The samples, obtained from lead-dosed
cows, are considered more reliable than materials prepared by spiking
with an inorganic salt. The program is
grant-supported and provided at no cost to participating laboratories.
Because a large number of laboratories in the WSLH PT program also
needed to meet CLIA 88 proficiency-testing requirements, the program
was modified in 1995 to send out five samples three times per year.
Three unknown samples continue to be sent out in the remaining nine
months.
Embedding a regulatory PT program within a voluntary interlaboratory
program is a unique approach to providing proficiency testing. PT
data indicate that laboratories have attained better performance
(as determined by successful PT results) when they participate in
the monthly external assessments, compared to laboratories that
participate in a PT program providing only the CLIA-mandated three
events per year.
Enrollment in the WSLH lead program has shown steady growth, increasing
from 273 participants in 1998 to more than 580 today. Much of this
growth is attributed to the introduction of hand-held analyzers
in 1997. The PT performance of these analyzers lags behind that
of other methods, and improving their performance has been a recent
program focus. Analysis of data from the WSLH PT program has led
the manufacturer to initiate changes in test kits and the calibration
process. These changes have resulted not only in improved PT performance
in 2004-2005 over previous years but, more importantly, increased
reliability of patient test results.
In 1997, the WSLH developed a PT lead program for blood lead on
dried filter-paper blood spots. Because no PT program existed to
provide an independent assessment of accuracy, the reliability of
the filter-paper methods was unknown, and its use generated controversy.
The pilot program was granted approval for CLIA-testing purposes
in 1999, and the test methods have gained general acceptance.
Blood-lead poisoning is well characterized as a public-health issue,
and the importance of accurate blood-lead measurements has been
well established. Through the combined efforts of WSLH PT and the
WSLH Clinical Metals Program (Toxicology) these issues have been
addressed by 1) offering an interlaboratory program that exceeds
the three yearly events mandated by CLIA; 2) using high-quality,
well-characterized
materials as unknowns; and 3) providing readily available technical
expertise and consultation to laboratories experiencing performance
issues or considering alternate analytical methods. Future plans
include the establishment of PT for other toxic elements —
notably, mercury and cadmium.
Rabies
The seriousness of rabies requires that those laboratories performing
this test provide the most reliable results possible. Testing for
rabies does not fall under the auspices of CLIA 88. This further
underscores the importance of these laboratories participating in
an interlaboratory PT program and including this survey in their
laboratory’s quality-assessment program. As a working public-health
laboratory, the WSLH is well situated to include a rabies PT program
among the full spectrum of PT programs offered. Since 1994, this
unique program — evolutionary
in format — has utilized the expertise and services of the
WSLH Virology Laboratory to identify and provide challenges to rabies-testing
laboratories throughout the United States. In the program, a minimum
of five challenges, consisting of microscope slides of brain-tissue
impressions, are prepared in house and distributed to laboratories
three times per year. This non-regulatory program encourages laboratories
to self evaluate their performance and identify opportunities to
improve their testing procedure. In addition to the analysis and
summary of test results, the program also provides a vehicle for
the sharing of ideas, questions, and comments among participants.
The event summary that accompanies a laboratory’s
performance report includes timely questions and answers to
issues identified and submitted by the participants.
In the early years of the WSLH rabies PT program, participants were
surveyed regarding their testing procedures. Responses indicated
that few laboratories adhered strictly to a published method. Despite
the use of numerous modifications to the fluorescent antibody test,
laboratories appeared to perform well and were able to correctly
identify the unknowns. The failure of some laboratories to consistently
identify weak positives, however, has provided the impetus for development
of a standardized testing method. This work has been an important
component of a workshop developed and presented biannually through
the National Laboratory Training Network. This training, provided
to laboratory professionals from throughout the country, will continue
to have an impact on laboratory practices, which will lead to improved
test performance.
Implied Consent Alcohol
A forensic alcohol PT program was developed in the early 1970s to
provide an external quality-assurance mechanism for Wisconsin laboratories
and for individuals who perform the implied consent (legal) alcohol
testing in these laboratories. There are a limited number of laboratories
in Wisconsin approved to perform this testing. Analysts working
in approved laboratories are certified based on their educational
background and training. The WSLH Forensic Toxicology
Program approves the methods used for performing the analyses. One
need only watch any of the several CSI programs on television
to appreciate how important reliable test results from these forensic
laboratories are to (Wisconsin) law-enforcement agencies and the
important role laboratory-test results play in operating while intoxicated
investigation and successful litigation. Five samples, consisting
of both urine and whole-blood challenges are prepared in house and
sent to laboratories five times per year. In order to receive a
passing grade, participant results must meet performance criteria
established by five out-of- state referee laboratories. Because
the program functions
under the auspices of well-trained and experienced professionals
in the WSLH Forensic Toxicology Laboratory, consultation is available
— not only for the technical aspects of the alcohol analyses,
but also for the individual called to testify as an expert witness
in the courtroom.
Specimens for the forensic
alcohol program are prepared to closely mimic clinical samples.
They are very stable, lasting many months. Dr. Kurt Dubowski, a
world-renowned forensic toxicologist and former director of alcohol
testing for the State of Oklahoma, has stated that “these
in house specimens were so well prepared and exquisitely stable
that they were used in our laboratory as controls, often as long
as a year after receipt, for our quality-assurance activities and
as an external reference laboratory for NBS, the CDC, the American
Association of Clinical Chemistry (AACC)/CAP Blood Alcohol Survey
program, and several state blood alcohol PT programs.”
The program is an excellent example of a PT program designed to
meet the needs of a targeted population, to ensure that participants
comply with regulatory requirements and produce results that can
withstand the scrutiny of the legal system.
Tuberculosis
Tuberculosis (TB) remains a leading cause of death worldwide and
a public-health priority in the United States. The confirmation
of the disease, now considered to be preventable and treatable,
depends on early and accurate diagnosis. To accomplish this requires
the utilization of state-of-the-art laboratory services and technology.
The development and expanded use of molecular methods has provided
laboratories with another tool to support successful strategies
for diagnosis, infection control, and patient isolation.
Since 1996, under contract with the CDC, WSLH PT has provided an
interlaboratory performance-evaluation program at no charge to laboratories
performing nucleic acid amplification testing for TB. The success
of this program relies on the participation of our WSLH TB Laboratory
and the professional staff who perform this testing.
Staff from the WSLH TB
laboratory identifies organisms to be used as unknowns in the program,
adjusts the concentrations of the organisms to challenge the performance
of the amplification methods, and provides feedback to participants
based on both individual laboratory results and the aggregate data
of all participants.
Because this is a non-regulatory program, the focus is educational,
and laboratories participate without concern for receiving an “unsatisfactory”
performance report. For example, through the analysis of data and
customer surveys, it was established that a significant number of
laboratories were not using unidirectional flow to minimize the
possibility of contamination; this laboratory practice is critical
to
successful use of molecular (PCR) methods. Another practice identified
was that some laboratories were not using inhibitor controls to
monitor for specimens that may be falsely negative. Over the years,
changes by participating laboratories in these practices have been
incorporated, resulting in an improvement in the reliability of
the test results.
Bioterrorism
An important issue facing the clinical laboratory today is the need
to prepare for the threat of a bioterrorism (BT) event and emerging
diseases that present broad population challenges. Hardly a day
goes by without the evening news warning of the threat of a natural
or man-made disaster. Is there a role for interlaboratory surveys?
Who should carry them out? How should they be implemented?
In order to answer these
questions, several issues need to be addressed. It is essential
to know the technical capabilities and competencies of clinical
laboratories in the event of a terrorism incident. An interlaboratory
PT program would be a useful tool in this evaluation. Prior to the
assessment of
a laboratory’s technical capability, it is important to ascertain
its ability to appropriately respond to whether the lab has, or
suspects, a BT agent. The WSLH has played a central role in creating
a laboratory network within the state, the Wisconsin Laboratory
Response Network (WLRN), and in providing necessary training to
these laboratories through site visits, teleconferences, and personal
communications.
Recently, educational
proficiency exercises were conducted by the WSLH that tested the
ability of laboratories to recognize and rule out the presence of
a BT agent (through testing of surrogate organisms) and to initiate
the appropriate response.
Response measures included:
- location and use of the Integrated Laboratory Response
Plan developed for clinical labs by the WSLH;
- contacting the WSLH via the 24/7 emergency-alert system
when a BT agent was suspected;
- following appropriate protocols for packaging and shipping
of suspect isolates to the WSLH, which serves as a national
Laboratory Response Network reference laboratory; and
- awareness of the location of specimen-shipping kits, which
are stored in repositories statewide.
The exercise also tested the timeliness of the WSLH response to
the emergency-alert system: how long it took for suspect isolates
to be received at the WSLH for reference testing, how the isolates
were handled within the WSLH, and whether the shipping-kit repositories
successfully functioned as designed.
Laboratories were not graded; however, the data submitted was summarized
and reported back to the participants; target results were provided
that allowed them to self assess. In addition to the written report,
the WSLH provided a teleconference for participants (presentation
also available as a Webcast) that reviewed relevant information
about the particular BT agents, the appropriate application of the
diagnostic-testing protocols, and whether laboratories adhered
to established emergency-response procedures.
Future surveys will be carried out to further assess laboratory
capabilities; the outcomes of these surveys will also be used by
the WSLH to identify additional training opportunities for WLRN
laboratories. The impact of these exercises goes beyond BT events.
Laboratories that participate in these exercises will be better
prepared to deal with the threat (or emergence) of SARS, monkey
pox, avian flu, and other agents that pose significant clinical
and public-health consequences.
Educational PT Programs Still Serve Important Purpose
WSLH PT has been able to adapt its full range of regulatory PT programs
to meet CLIA 88 requirements, while at the same time maintaining
(and adding) several programs where education has been the primary
focus. It has been able to do this successfully because of its unique
position within a working laboratory — the WSLH — and
its access to the laboratory’s staff of working professionals
who share their expertise in data analysis, method evaluation, and
a wide range of performance and technical issues faced by today’s
clinical laboratory. Each of the survey programs described could
not have been carried out
in an environment that did not have access to this laboratory staff
and the resources provided in their departments. This richness of
staff and expertise is available for WSLH PT’s regulatory
programs as well. Proficiency-testing programs should go beyond
the important role they play in the regulation of laboratories.
They must also serve as a vehicle to educate and prepare laboratories
— not only for new technology and
specialized areas of testing, but also for future challenges that
will assess their capability to respond to needs that have not yet
been clearly defined. It is apparent that this latter role will
not be fully realized through the implementation of “traditional”
proficiency-testing programs.
Author’s note: Thanks to the staff at the
WSLH for their overall support of WSLH PT, and especially the following
for their contributions to and review of the specific programs detailed
in the article: Noel Stanton, James Powell, Patrick Harding, Dr.
David Warshauer, Dr. Peter Shult,
and Carol Kirk.
David J. Hassemer
is the director of the WSLH Proficiency Testing Program and director
of the Laboratory Improvement Division of the Wisconsin State Laboratory
of Hygiene at the University of Wisconsin-Madison.
###
Posted By:
Jessica D. Burda,
WSLH Public Affairs
Date:
January 25, 2006
Related Stories:
Return
To WSLH News & Events
|
