GDR23 May 2020 11:52
MAY 19, 2020
expert reaction to PHE laboratory evaluations of Roche and Abbott antibody tests
Public Health England (PHE) laboratories have conducted analysis on antibody tests produced by the companies Roche and Abbott.
Prof Jon Deeks, Professor of Biostatistics and Head of the Test Evaluation Research Group, University of Birmingham, said:
“COVID-19 serology tests from Roche and Abbott were reported by the manufacturers and government last week as being “100% accurate” and as game-changers in the identification of past infection with COVID-19. These claims were based on studies undertaken by Public Health England (PHE) but the release of these statements preceded any official study reports being made available, and thus their validity could not be scrutinised. These reports have now been released (1,2) and it is possible to check the veracity of the “100% accuracy” claim.
“The two studies have notable limitations: they are based on samples and not patients (almost certainly some patients will have contributed multiple samples which will make the results look more precise than they actually are); the origin and severity of disease in the COVID-19 samples is not known, so we can’t check whether the samples are representative of typical patient groups; and non-COVID-19 patients with similar respiratory illnesses were not included. As the studies were undertaken in expert PHE laboratories, their performance may not be so high when used in practice.
“The accuracy of a test relates to whether it makes errors: whether there are individuals with the disease who wrongly get negative test results, and whether there are individuals without the disease who wrongly get positive test results. Saying a test is 100% accurate implies to the public that neither of these two types of error occurs. The reports from PHE make it clear that this statement is misleading.
“Whilst both tests make no or very few false positive errors – they only very rarely wrongly state a non-COVID-19 sample as showing antibodies to COVID-19, both tests sometimes miss detecting COVID-19 in samples which are from infected patients.
“For the Roche test, 93 samples with COVID-19 were tested, of which 78 (84%) gave positive test results. Thus 16% were missed. For the Abbott test, 96 samples with COVID-19 were tested, of which 90 (94%) gave positive test results. Thus 6% were missed. Whilst these error rates, particularly for Abbott, may still be low enough for these tests to have a useful role, they fall short of being game-changers, and certainly cannot be described as 100% accurate. Other tests exist which have similar performance. The two tests were evaluated on different samples in different laboratories, so caution is needed in making direct comparisons.
“It is also important to consider the margin of error in these estimates which arises because of sample size. The 95% confidence intervals for Roche figure ranges from 75% to 91%, an
Sign In
Follow the stocks
