Theranos Continues to Confront Thunderheads Regarding Accuracy of Its Testing Platform

Whether or not in its uncertain future it manages to survive or even thrive, Theranos will at least receive credit for never being boring.

The Theranos saga has just entered its most critical state, as the Centers for Medicare & Medicaid Services (CMS) recommended the company’s CLIA certification be revoked for its California lab only days after its scathing inspection report was made public. Meanwhile, a group of researchers from one of the most prestigious teaching hospitals in the nation questioned the validity of Theranos’ testing platform.

Theranos responded by deflecting the former while attacking the latter, all the while appointing a new scientific and advisory board jam-packed with health care luminaries. Promising to allow low-cost lab testing with just a few drops of blood that can be drawn without needles, Theranos was valued at $9 billion just a few months ago. But as a deal to install testing facilities at Walgreens pharmacies throughout the country is placed on hold, it is experiencing a storm of criticism all but unprecedented for a laboratory.

The CMS’ conclusions were the worst news. It recommended that Theranos lose its CLIA certification by May 17. That was based on an inspection that took place late last year where the agency determined Theranos’ hematology testing, analytics and high-complexity testing were not accurate and posed an immediate danger to patients. CMS has concluded that the corrective actions taken by Theranos were not credible, pointing out numerous instances where Theranos plans were non-specific. Theranos can appeal the revocation if it is imposed. CMS also recommended limitations on the lab’s performance of hematology assays that would take place almost immediately. Theranos faces a $10,000 daily fine for non-compliance. CMS has also requested a list of all providers who have used the lab since January to notify them of the ongoing issues at the facility.

The CMS’ conclusions were the worst news. It recommended that Theranos lose its CLIA certification by May 17. That was based on an inspection that took place late last year where the agency determined Theranos’ hematology testing, analytics and high-complexity testing were not accurate and posed an immediate danger to patients. CMS has concluded that the corrective actions taken by Theranos were not credible, pointing out numerous instances where Theranos plans were non-specific. Theranos can appeal the revocation if it is imposed. CMS also recommended limitations on the lab’s performance of hematology assays that would take place almost immediately. Theranos faces a $10,000 daily fine for non-compliance. CMS has also requested a list of all providers who have used the lab since January to notify them of the ongoing issues at the facility.

• Preanalytic systems and relevant documentation when specimens were referred to other labs for testing failed.
• Corrective actions for chemistry quality control were not in place.
• The proper calibration of equipment failed.
• The laboratory director failed to sign off on mandated procedures or changes to procedures.
• Appropriate temperature ranges were not maintained for freezers.
• Appropriate guidelines to maintain the integrity of quality control materials were not followed.
• The laboratory failed to notify a patient or patients of errors in their test results in a timely fashion.

There were some redactions to the report, including the annual test volume of the laboratory. According to a recent report by the Wall Street Journal, quality control efforts failed 22 to 87 percent of the time, depending on the kind of test being run.

Theranos noted it had hired a new laboratory director to overhaul operations, and that it was retraining staff, along with taking other corrective actions CMS has since concluded were not satisfactory.

Just days before that correspondence was made public, a study appeared in the Journal of Clinical Investigation. Researchers from the Icahn School of Medicine at Mt. Sinai Hospital New York had 60 patients undergo common testing via Theranos’ retail sites in Arizona last year, and compared the results against more traditional venipuncture testing at LabCorp and Quest Diagnostics. The study included more than 18,000 data points.

The results were not gratifying for Theranos: It flagged tests outside its normal range of results 1.6 times more often than Quest and LabCorp. Of the 22 lab measurements evaluated, 15 (68 percent) showed significant interservice variability. And 2.2 percent of Theranos’ data results were missing, compared to 0.2 percent for LabCorp and no missing results for Quest.

“While most of the variability we found was within clinically accepted ranges, there were several cases where inaccurate results would have led to incorrect medical decisions,” said Joel Dudley, the study’s senior author and director of biomedical informatics at the Icahn School of Medicine at Mount Sinai Hospital in New York City, in a statement.

Theranos did report more precise results in leukocyte subsets, although the study’s authors noted that that would not likely impact clinical decision-making. But the variances were stark enough in lipid panel testing that the study recommended Theranos be more transparent regarding how it calibrates its Edison testing platform.

Theranos expressed deep criticism not only of the test results, but of the integrity of some of the researchers behind the study. In a letter sent to the journal’s editor and signed by its four laboratory managers, it questioned whether the researchers had contacted Theranos as they had purportedly claimed in media interviews; and noted that both Dudley and Schadt failed to disclose their participation in a Palo Alto, Calif.-based company called NuMedii, which uses some genomic testing to determine the efficacy of some pharmaceuticals under development (Theranos labeled NuMedii as a “potential competitor”).

And Theranos also questioned the integrity of the study, including how the blood was collected. It noted that the larger draws of blood for the Quest and LabCorp testing could have caused blood flow issues that may have compromised the Theranos test results. It also raised questions because the study did not disclose which fingers of the patients had been pricked for collections.

“These fundamental problems with the conduct of the study reflect the lack of understanding of the study investigators concerning basic sample collection procedures that are critical for the integrity of their study and its conclusions,” the letter said. Neither Schadt nor the Icahn School of Medicine responded to requests seeking comment.

Other observers were less critical. Michael Cherny, an analyst with EverCore ISI, raised concerns in a recent report that Theranos had tried to halt the publication of the study.

“We have refrained from piling on to the negative Theranos narrative too much ... while we have moved past the point of viewing Theranos as a potential near-term risk, we continue to point to studies like this as further evidence of our standing, and we continue to wait on Theranos to release the data it promised in order to try to change the narrative,” Cherny wrote. “All-in, this is another crack in the Theranos armor.”

In its latest act of damage control, Theranos has appointed an eight member scientific and advisory board. Among them are physicians from New York Presbyterian/ Weill-Cornell Hospital, a former director of the Centers for Disease Control and Prevention and a former president of the American Association of Clinical Chemistry.

“It is clear that Theranos has done what people thought was impossible,” said David Helfet, M.D., director of the orthopedic trauma service, for New York Presbyterian Hospital, and a cochair of the advisory board along with Chief Executive Officer Elizabeth Holmes, who left Stanford University as a teenager to start the company. “Theranos invited groups of independent experts in the fields of pathology and laboratory medicine and literally took the lid off of the box. Experts were shown Theranos’ technology, met with Theranos scientists, and had access to any data.”

Takeaway: Although the storms buffeting Theranos continue unabated, the company’s determination to succeed so far appears undaunted.