Now Even the “Good” Tests Are Coming Under Fire

The inaccuracy of serology tests that detect antibodies to the SARS-CoV-2 virus has become an issue of mounting concern. Even though the lack of sensitivity and specificity of these tests is well documented, the vast majority of the more than 150 SARS-CoV-2 serology tests currently provided in the U.S. reached the market through a pathway that did not require Food and Drug Administration (FDA) review. The supposed silver lining is the group of 11 tests that did actually receive Emergency Use Authorization (EUA) clearance and are thus deemed the most reliable. However, a new study casts suggests that one of the fastest and most widely used of these gold standard tests has major sensitivity problems, missing as many as half of the cases that another test found to be positive. The NYU Study Researchers from New York University (NYU) set out to evaluate one of the newly approved SARS-CoV-2 serology tests used in their laboratories. While those tests also included the Cepheid Xpert Xpress and Roche Cobas assay, Abbott Laboratories’ ID Now point of care test was an attractive target for study because of its fast turnaround—the Abbott test produced results in as little as five minutes, as compared to 45 minutes for the Cepheid test and 3.5 hours for the Roche assay. Of course, these are the very same qualities that make ID Now the go-to test for so many end users, including the White House which uses the assay to test officials and screen visitors. The researchers assessed 101 dry nasal swabs and 101 nasopharyngeal swabs from the same patients to compare the ability of the ID Now and Cepheid Xpert Xpress tests to detect SARS-CoV-2 viral nucleic acids. They found that the ID Now test performed well at first but generated more false negatives as the viral load decreased. According to the study, which was published on the BioRxiv preprint platform and which has not undergone peer review, the Abbott test missed one-third of the samples detected positive by the Cepheid test when using nasopharyngeal swabs, and 48 percent of the samples detected when using dry nasal swabs. Based on these results, the researchers found that those who used ID Now for first-step screening, would have to confirm over 80 percent of the tested samples “to be confident that the negative results are truly negative.” On the bright side, they concluded that the test had high specificity and positive predictive value, meaning that “a positive result can be interpreted as a true positive.” The Reaction Abbott and the FDA reacted swiftly by questioning the researchers’ testing methods and noting that the findings were not consistent with other studies and accounts of the ID Now test’s accuracy. “Abbott has distributed more than 1.8 million ID Now tests and the reported rate of false negatives to Abbott is at .02 percent, which we’ve previously shared with the FDA,” according to a company spokesperson. The NYU study provides excellent ammunition for those who have criticized the White House and FDA for relaxing regulatory scrutiny and allowing unproven serology tests to reach the market. At the same time, it is not the first study to call the Abbott test’s accuracy into question. The NYU researchers cited a separate study by the Cleveland Clinic finding that ID Now COVID-19 had a false negative rate of 14.8 percent—although that might have been influenced by the fact that study samples were stored in viral transport media instead of being directly inserted into the ID Now platform. In response to the Cleveland Clinic study, Abbott recommended using dry swabs for best results. But the NYU researchers found that sensitivity of ID Now using dry swabs was actually lower than when using viral transport media—51.6 percent vs. 66.7 percent. Study Finds Abbott Serology Test More Accurate than Euroimmun Serology Test Two days after the NYU study came out, researchers from Washington University in St. Louis and Barnes Jewish Hospital published a study in Clinical Chemistry finding Abbott’s SARS-CoV-2 IgG assay more accurate than another EUA-cleared serology test, the Anti-SARS-CoV-2 ELISA assay produced by PerkinElmer’s Euroimmun business. Testing samples from 103 patients found by polymerase testing to have COVID-19 and 153 patients without the disease, the study found that 14 days after the onset of symptoms:

• The Abbott test had a specificity of 99.4 percent and a sensitivity of 93.8 percent; and
• The Euroimmun assay had a specificity of 94.8 percent and a sensitivity of 85.4 percent.

However, the researchers characterized the differences in sensitivity and specificity between the tests “statistically insignificant” and said more studies would be necessary to confirm that the Abbott test had higher sensitivity and specificity than the Euroimmun test. Takeaway Even the FDA has admitted that it was a mistake to allow SARS-CoV-2 serology antibody tests to market without agency review and is now requiring all test makers to submit their products for independent validation by federal laboratories. What makes the NYU report so disconcerting is that the test it questions is one that is not only so widely used but also among the small group of assays that came through the more rigorous pipeline requiring agency review and EUA clearance. And if even that test is not reliable, can any of the COVID-19 serology trusts currently being provided be trusted?