Abstract

BACKGROUND/AIMS

Rapid diagnostic tests are designed to enable rapid diagnosis of infectious diseases, especially during outbreaks and pandemics. This study aims to retrospectively list the rapid antigen (Ag) tests used in the recent pandemic with the multi-criteria decision-making (MCDM) technique to evaluate the role of the MCDM technique in determining the most appropriate diagnostic techniques during a future outbreak.

MATERIALS AND METHODS

Twenty-eight Ag diagnostic tests authorized for emergency use by the Food and Drug Administration during the coronavirus disease-2019 pandemic were retrospectively included. Limit of detection, positive and negative percent agreement, point-of-care test, sample type, test technique, Ag target, and result time were evaluated. The overall performance of the 28 Ag diagnostic tests was investigated using the fuzzy preference ranking organization method (F-PROMETHEE) of the MCDM approaches.

RESULTS

According to the F-PROMETHEE analysis; clip coronavirus disease rapid ag test was ranked in first place, Sofia 2 Flu + severe acute respiratory syndrome (SARS) Ag fluorescent immunoassay, BD Veritor System, and VITROS Immunodiagnostic Products severe acute respiratory syndrome-coronavirus-2 (SARS-CoV-2) test kits were ranked second, and third, respectively. The VITROS Immunodiagnostic Products SARS-CoV-2 Ag Reagent Pack was ranked last, due to the selected parameters in the ranking.

CONCLUSION

The F-PROMETHEE method, one of the MCDM methods, can be applied to evaluate the tests used in the rapid diagnosis of pathogens, and can support clinicians and laboratories in choosing the most reliable and accurate diagnostic tests in future outbreaks and pandemics.

INTRODUCTION

Nucleic acid amplification with the real-time polymerase chain reaction technique is the gold standard in the identification of emerging microbial pathogens; however, antigen (Ag) testing, which detects virus-specific proteins, is widely used in the rapid diagnosis of pathogens, especially in extraordinary situations and disasters like pandemics.¹ Ag tests can be performed using both the nasopharyngeal swab and anterior nares. These tests are easy to use and are more suitable for point-of-care (POC) testing. As Ag diagnostic tests provide rapid and accurate results at a relatively low cost compared to the reference methods, they have made critical contributions in the control of the last pandemic.² During the pandemic, Ag tests were recommended for people with symptoms, asymptomatic individuals at high risk of infection, and for the purposes of contact tracing and screening of severe acute respiratory syndrome-coronavirus-2 (SARS-CoV-2) infected persons for epidemiologic investigations.³ Various Ag diagnostic tests were classified as “emergency-use authorized” for SARS-CoV-2 patient management during the global crisis.⁴ During the coronavirus disease-2019 (COVID-19) pandemic, many companies contributed to the management of the pandemic by specifically developing Ag tests in a short time frame. However, as the pandemic progressed, different measurements were needed to monitor the efficiency and clinical performance of these options. Multi-criteria decision making (MCDM) techniques are approaches that assist decision-makers (DMs) when faced with a selection problem involving multiple criteria.

Numerous issues, including the weights of the criteria, preference dependence, and conflicts between criteria, seem to exacerbate problems when DM evaluates the alternatives. This process requires the use of more advanced techniques to resolve them.^{5, 6} Hwang and K. Yoon⁷ proposed that MCDM problems can be divided into two major categories: multiple attribute DMs and multiple objective DMs, due to the various purposes and various types of data. Fuzzy logic theory, published by Lotfi A. Zadeh^{8, 9}, was the first to study the fuzzy logic process⁶ mathematically. Zadeh⁸ has brought many concepts to science, such as fuzzy sets, fuzzy logic, approximate reasoning, linguistic variables, and fuzzy if-then rules.

This study aimed to demonstrate the role and usability of the MCDM method in deciding the most appropriate diagnostic test during extraordinary circumstances by ranking the Ag tests that were used during the COVID-19 pandemic. The key advantage of the current study is that the fuzzy MCDM algorithms will serve as a model for evaluating diagnostic alternatives designed for future outbreaks and pandemics.

MATERIALS AND METHODS

One of the most successful MCDM techniques, the fuzzy preference ranking organization method (F-PROMETHEE) technique, was preferred to analyze Ag tests used for SARS-CoV-2 detection. The study was retrospectively performed, and the most frequently performed Ag testing during the COVID-19  pandemic was utilized to mimic the testing methodology of a similar global health problem. In the current study, linguistic triangular fuzzy sets were applied to determine the criteria and their weights numerically. As a preprocessing step for the data, the defuzzification process was then applied via Yager¹⁰ index to gather numerical information for the selected variables, preparing it for use in the PROMETHEE approach. The PROMETHEE approach enables DMs to rank the alternatives that contain various features.¹¹ The PROMETHEE approach, with fuzzy logic, is a recent MCDM technique that compares alternatives under an ambiguous environment. In this approach, the triangular fuzzy sets are used to determine the linguistic data numerically as fuzzy numbers.

Twenty-eight different SARS-CoV-2 Ag kits that had been authorized for emergency use by the Food and Drug Administration (FDA) during the pandemic were evaluated in this study. These criteria were included according to the information provided by the FDA.⁴ While predominantly lateral flow assay (72%) and chemiluminescent immunoassay (28%) were used in the study, paramagnetic micro-based immunoassays, bulk acoustic wave biosensors, and chromatographic digital immunoassay were also evaluated using mathematical tools. We preferred to use PROMETHEE for analysis since it provides different types of preference functions to determine the superiority of each decision option for each criterion. This distinguishes it from other MCDM models and shows the advantages and disadvantages of each decision option, allowing experts to control for result validation. The criteria used for analysis were: limit of detection (LoD); sensitivity/specificity; POC testing; specimen options (nasopharyngeal swab/anterior nasal swab); test techniques; Ag target [nucleocapsid (N)/spike (S)]; testing for SARS-CoV-2 and flu; time to result; first sampling after symptom onset; reagent storage conditions; requiring analyzer.

F-Promethee Analysis

These criteria were evaluated as linguistic triangular fuzzy sets: “very high (0.75, 1, 1)”, “high (0.5, 0.75, 1)”, “moderate/(0.25,0.5,0.75)”, “low/(0, 0.25, 0.75)”, and “very low/(0, 0, 0.25)” and the SARS-CoV-2 pandemic was considered during this scoring. After data collection, defuzzification was applied to convert fuzzy numbers to single numbers. Then the PROMETHEE method was applied with the Gaussian preference function. The detailed process of the PROMETHEE approach is given in.^{11, 12} In this study, the importance values of the selected criteria were obtained based on experts’ preferences as follows: very high: the LoD, positive predictive value/sensitivity, negative predictive value/specificity: high: the point of care testing, specimen option, target, sampling days after symptom onset, result time: moderate: the requiring analyzer, attributes-visual read, test for SARS-CoV-2/Flu, storage.

Ethical Approval

Ethical approval, including patient-informed consent, was not needed as the study involved publicly available data and did not involve human clinical samples.

RESULTS

Among 28 FDA emergency use authorization Ag diagnostic tests for SARS-CoV-2, clip COVID rapid Ag test [Luminostics, Inc., California, United States of America (USA)] was determined to be the most favorable one, followed by Sofia 2 Flu + SARS Ag fluorescent immunoassay (Quidel Corporation, San Diego, USA) and BD Veritor System for Rapid Detection of SARS-CoV-2 (Becton Dickinson and Company, New York, USA). Our finding showed that the clip COVID rapid Ag test (Luminostics, Inc, California, USA), which was the most feasible Ag testing kit according to the ranking, is a POC test that gives results with a portable clip analyzer. Due to its sensitivity and specificity of 96.9% and 100%, respectively, with a LoD of 0.88x10² TCID50 per milliliter, this method is highly reliable for diagnostic purposes. Furthermore, nasal swabs obtained within 5 days after the onset of symptoms were used, which still provided reliable results.⁴ The complete ranking of Ag tests against SARS-CoV-2 is given in
Table 1. This table includes more than 28 Ag tests because some tests can be performed either by nasopharyngeal swab or anterior nasal swab, which affects the sensitivity of the test results. Therefore, they were evaluated independently in this study, and different specimen options of the same test were differentiated with asterisks in
Table 1. Test kits with an asterisk indicate that both nasopharyngeal swabs and nasal swabs can be used as specimen collection options. Test kits without an asterisk indicate that a nasopharyngeal swab is the only specimen collection option. These results were obtained using the Decision Lab program.

DISCUSSION

Recently, disease outbreaks with unknown new pathogens have been reported.¹³ In the most recent COVID-19 pandemic, a fluctuating increase in cases was detected during the timeline of the pandemic.¹⁴ Therefore, measures have been taken to control the number of cases. However, implementations such as the closure of borders, schools, and workplaces have also had disparate effects.¹⁵ For this reason, control measurements were extended by continuing the screening tests during the later phases of the pandemic. Throughout the pandemic, Ag tests were useful in rapidly screening large populations. Although definitive detection of the virus is dependent on the detection of RNA gene targets such as S, envelope, N, RNA-dependent RNA polymerase, ORF1 by nucleic acid amplification testing, detection of virus-specific proteins by Ag testing has been preferred during the global crisis.¹⁶Many manufacturers developed diagnostic alternatives for SARS-CoV-2 to combat the global crisis. However, it was not easy to choose the most accurate test among many alternatives. This study revealed the effective role of the F-PROMETHEE technique in diagnostic microbiology. The applicability of the technique in making correct choices in similar public health concerns was demonstrated.

Throughout the world, preventing and controlling widespread outbreaks is a priority. Clemente-Suárez et al.¹⁷ provided information on the performance of fuzzy MCDM analysis of emergency systems regarding the applications of the technique in hospital settings during the pandemic, as currently applied. As the pandemic continue, the emergence of new variants may cause difficulties with the virus’s diagnosis, treatment, and prevention. Therefore, MCDM methods have been widely implemented to address this complexity. Sayan et al.¹¹ evaluated the capacity of SARS-CoV-2 diagnostic tests using MCDM.¹¹ The study on COVID-19 diagnostic tests was conducted at the beginning of the pandemic and focused on diagnostic testing techniques. Similarly, treatment alternatives for COVID-19 were evaluated by Yildirim et al.⁶ using a fuzzy MCDM approach, and they revealed the most appropriate diagnostic and treatment options to support healthcare professionals.

To our knowledge, this work presents a new study evaluating test kits for microbial diagnosis using the F-PROMETHEE technique. This study revealed that the performance of rapid diagnostic tests, widely used in the rapid diagnosis of infectious diseases, especially during disasters, can be evaluated with mathematical approaches before clinical use. Such approaches can guide diagnostic laboratories to choose the most appropriate rapid diagnostic tests to manage future outbreaks and pandemics effectively.

Study Limitations

To point out the study’s limitations, positive and negative control samples could not be used because the study was based on mathematical data analysis and was not conducted with clinical samples. Secondly, the results were obtained based on the nature of the analytical DMs process, selected parameters, and the experts’ preferences for determining the importance of the criteria; thus, these parameters can be updated according to the DMs’ priorities. Thirdly, the study was based on data from the test kit insert rather than clinical data. This data did not include positive and negative controls, which may have altered the results. Additionally, only the F-PROMETHEE technique was used in the study. Comparison of results with other MCDM techniques such as Technique for Order Preference by Similarity to Ideal Solution, Analytic Hierarchy Process, and ELimination and Choice Expressing REality may have affected the ranking.

CONCLUSION

In outbreaks, low-cost rapid diagnostic tests can be prioritized to reduce the workload of healthcare systems. By applying MCDM methods, DMs can systematically analyze multiple factors and prioritize tests according to their importance and necessity in outbreaks. The F-PROMETHEE method can be applied in this field, supporting DM in deciding on the most reliable and accurate rapid test for pathogen detection throughout outbreaks in the future. Additionally, each country can use appropriate diagnostic solutions according to its resources.