Becoming Proficient in Proficiency Testing

On July 11, 2024, the latest update to CLIA regulations comes into effect, raising the bar for clinical labs’ analytical quality and proficiency testing (PT).¹ It’s the latest in a list of regulatory changes to which labs must now devote time and resources—but will more stringent requirements lead to improvements for labs and patients? Lab quality expert Sten Westgard, director of client services and technology for Westgard QC, thinks so.

CLIA’s quality update

The change brings in stricter proficiency criteria for a wide range of tests across clinical chemistry, endocrinology, hematology, immunology, and toxicology.² “There are previously unregulated analytes that will now be directly regulated, requiring more specimens per survey and more surveys per year,” explains Westgard. “There are also existing regulated analytes for which the new criteria are significantly tighter—20 to 40 percent reductions in allowable error for some tests.” The update, the first in the requirements’ 32-year history, is intended to reflect the improved performance of modern instruments and heightened standards for patient care.

“For many labs, PT up until now was pro forma. Failures were rare because the goals established in 1992 were wide and permissive. This has generated such an air of complacency regarding PT goals that many don’t believe the new goals are anything to worry about,” says Westgard. “But why would CLIA update the goals if they will have no impact? If your budget for anything else in your laboratory shrank by 40 percent (or more), wouldn’t that be cause for concern?”

With these new regulations, the FDA’s recent final rule on lab-developed tests (LDTs),³ and updated requirements for laboratory directors,⁴ the clinical lab has undergone a lot of change in the past few months. But, as Westgard points out, “These regulations target different aspects of the laboratory; nothing is getting a double dose. CLIA’s direct regulation and updated PT criteria are focused on standard, well-established test methods. LDTs are novel, more esoteric test methods that, by definition, don’t have established PT criteria. LDTs need some form of proficiency testing, but the goals for success will not be written by CLIA.”

Preparing for compliance

As with all regulations, Westgard says, preparation is the best remedy for the new requirements. “Look at your PT data and impose the new goals on the results. Estimate whether you would pass.” Some labs may want to hire consultants to evaluate their testing; others may need to devote additional resource to the calculations. “If you have methods that are failing PT now, the new CLIA criteria are unlikely to make life easier for you—but you may have some methods that were perfectly fine using the old goals, but will become more failure-prone under the new ones. It’s worth figuring out which methods those will be.”

Westgard recommends the analytical sigma metric⁵ as a quick shorthand to predict whether your methods are at risk of PT failure, warning laboratories to focus on improving those whose results fall below the three-sigma mark. “The new CLIA criteria are publicly available; so is the sigma metric. PT programs have customer portals in which you can review your results. All the ingredients are there for any laboratory to predict their future PT successes or failures. Having the time and staff to do it, however, is the eternal laboratory challenge.”

Not all labs will be equally affected by the update—and resource limitations may prove one of the biggest challenges. “Some labs and instruments will fail more PT than others,” says Westgard. “The right instrument in the right laboratory will feel minimal impact. The regulations, like many laboratories, proceed with the assumption that all instruments are the same—but anyone with enough years under their belt knows that different diagnostic manufacturers provide different levels of quality. If you sacrifice quality for low cost, don’t be shocked when the new PT criteria reward your frugality with more frequent PT failures.”

No matter how labs approach the new requirements, one change is guaranteed: most laboratories will need to spend more money and expend more effort on PT.

Common quality challenges

“Some of today’s laboratory problems are still the same ones that plagued the lab back when my father started in his laboratory half a century ago,” says Westgard. “The level of knowledge and skill for quality control (QC) is possibly worse than it was 25 years ago. QC is one of the blind spots in laboratory operations, ignored equally by diagnostic manufacturers, regulators, and laboratory professionals alike.”

In his work, Westgard routinely encounters labs whose staff admit they don’t understand QC, which makes him worry that even the latest CLIA update won’t advance their expertise. “Build a chart, throw some dots on it, and you’ll be in compliance. That’s not a recipe for deep understanding. What happens far too often is that the bench-level implementation of QC is closer to superstition, where the remedy for outliers is to repeat controls over and over again until they ‘fall in,’ as if the act of repetition solves the root causes of method performance.”

The solution? Better training for clinical lab professionals so they are equipped with the technical skills to properly design and interpret QC.

“Laboratory professionals get almost no training in QC during their education,” Westgard says. He anticipates that the CLIA update will lead to improvements—but the process may not be quick or painless. “Because PT testing under CLIA is a relatively infrequent event (three times a year for the directly regulated assays, twice a year for others), the failures will not happen instantly; they will be spread out over time. But some instruments that were designed and built with 1992 goals in mind will find it increasingly difficult to pass PT. In contrast, instruments where the R&D invested in improving performance above and beyond CLIA specifications will prosper. Much like the tightening of HbA1c goals fueled method improvements and better clinical use of results,⁶ the long-term upside of new CLIA PT criteria will be better test results for clinicians and patients. In the short run, however, there will undoubtedly be an uptick of pain.”

References:

1. 1. Clinical Laboratory Improvement Amendments of 1988 (CLIA) Proficiency Testing Regulations Related to Analytes and Acceptable Performance. Federal Register. July 11, 2022. https://www.federalregister.gov/documents/2022/07/11/2022-14513/clinical-laboratory-improvement-amendments-of-1988-clia-proficiency-testing-regulations-related-to.

1. 1. 2024 CLIA Proposed Acceptance Limits for Proficiency Testing. Westgard QC. July 11, 2022. https://westgard.com/clia-a-quality/847-2019-clia-requirements.html.

1. 1. 21 CFR Part 809. Medical Devices; Laboratory Developed Tests. Federal Register. May 6, 2024. https://www.govinfo.gov/content/pkg/FR-2024-05-06/pdf/2024-08935.pdf.

1. 1. Clinical Laboratory Improvement Amendments of 1988 (CLIA) Fees; Histocompatibility, Personnel, and Alternative Sanctions for Certificate of Waiver Laboratories. Federal Register. December 28, 2023. https://www.federalregister.gov/documents/2023/12/28/2023-28170/clinical-laboratory-improvement-amendments-of-1988-clia-fees-histocompatibility-personnel-and.

1. 1. Kumar BV, Mohan T. Sigma metrics as a tool for evaluating the performance of internal quality control in a clinical chemistry laboratory. J Lab Physicians. 2018;10(2):194–199. doi:10.4103/JLP.JLP_102_17.

1. 1. Little RR, Rohlfing CL. The long and winding road to optimal HbA1c measurement. Clin Chim Acta. 2013;418:63–71. doi:10.1016/j.cca.2012.12.026.