In the first comprehensive literature evaluation of electronic biosurveillance systems, half of 20 published evaluations of 11 electronic systems examined [Figure 1] reported at least one quantitative assessment from a list of 17 variables established by the study’s authors. None of the systems cited a specific example of public health decisions, actions, or outcomes resulting from their alerts.
Through a PubMed and Google Scholar Search of peer-reviewed publications evaluating the systems, quantitative and qualitative variables were determined for each of the electronic event-based biosurveillance systems examined. The variables were developed through review of surveillance system evaluation and outbreak detection system guidelines established by the U.S. Centers for Disease Control and Prevention. The variables selected to evaluate systems included accessibility, cost, data quality, population coverage, predictive value positive, timeliness and validity [Table 1].
“The internet has become a key resource for tracking infectious disease events, like the current Ebola epidemic. From local health agencies to global health organizations, public health practitioners use web-based applications to detect and assess epidemics,’’ said Naval Commander Jean-Paul Chretien, the study’s senior author. “We wanted to define what’s known about the performance of these systems. We brought together all of the published evaluations, and developed a common framework to determine whether there are significant gaps in our understanding of how and why they’re useful.” The use of electronic biosurveillance systems within the public health community has grown significantly as access to the Internet has fueled their popularity. Approximately 65 percent of the world’s first news about infectious disease events comes from informal sources, such as the Internet, and almost all major outbreaks investigated by the World Health Organization are first identified through these informal sources, the study noted.
The study’s authors evaluated electronic event-based biosurveillance systems that use publicly available Internet information sources, which monitor events that impact human health globally. These systems were identified through an inventory of systems and consultation with biosurveillance experts.
Older electronic biosurveillance systems had more evaluations than the newer systems, with the exception of HealthMap, which ranked second for the most evaluations despite being founded in 2006. The median number of key variables assessed per system was eight (range, 3-15), with six evaluations assessing seven or more key variables. Older systems were more likely to be reviewed in parallel with each other. There were two or fewer published evaluations onthe GODsN, EpiSpider, MiTAP and Geni-Db systems.
The study did find that in 10 of the 11 systems were evaluated for timeliness, a possible advantage compared to traditional methods such as local and state public health reporting.
Although there have been evaluations of individual electronic event-based biosurveillance systems, there has not been a structured evaluation or multiple ones, or a comprehensive assessment for them all, the study said. The authors added that more qualitative and quantitative evaluations need to be done on electronic event-event based systems to assess the usefulness of them in guiding public health action. These evaluations, the study noted, should compare the novel aspects of electronic systems to traditional surveillance approaches. For example, the evaluations should include variables of special importance to users, including policy readiness, number and geographic profiles of users, number of sources, system redundancy, and input/output geography.
“We need to understand better how medical and public health professionals, and the public, use these web-based tools to make decisions,” Commander Chretien said. “That would help developers create more useful applications for detecting and responding to epidemics.”
Click here to read entire study.
RSS