NBAF risk data of limited predictive value

Tom Manney

By A Contributor

I am responding to two NBAF-related articles in the Mar. 4 Mercury — the story by Brian Richardson and the editorial “Good news about NBAF risks.”

The editorial said, “It isn’t hard to imagine NBAF opponents questioning the newer assessment — even wondering if it’s legitimate.” This took little imagination since last Friday I said as much to Mr. Richardson.

So I wish to share my thoughts on the updated NBAF risk assessment for release of foot-and-mouth disease virus, or any of the NBAF germs that are actually deadly to humans.

After hours studying the roughly 1,000-page statistical modeling analysis, I do not question its findings; it seems to be a legitimate response to Congress’s mandate. Colleagues more authoritative in their knowledge of probability theory and statistical analysis agree.

But I do question and doubt the legitimacy of incomplete, uninformed and apparently disingenuous public interpretation of the report’s findings by the administrators and politicians who seem singularly focused on bringing this facility to Manhattan at any cost. Kansas State University officials and the Kansas congressional delegation, as reported nationwide, have dwelled almost exclusively on the risk of a foot-and-mouth disease outbreak during the NBAF’s 50-year operating life being down from 70 percent to 0.11 percent.

But these interpretations have ignored 1) the caveat stated repeatedly throughout the report that the calculated probabilities cannot be taken as absolute values; 2) that statistical models are only theories and are no more accurate in predicting actual events than the assumptions that are plugged into them, and 3) the uncertainty of many of the input assumptions, and therefore of the conclusion, are extremely large. These, too, are clearly stated in the report.

For example, on page 403: “it is fundamentally a modeling-based approach and therefore has limited ability to predict the absolute probability of an outbreak occurring and the corresponding consequences.”

Statistical analysis is highly technical, intensely mathematical and universally mysterious to ordinary people. I have yet to read, even in the Department of Homeland Security report itself, an intelligible statement of what a risk of 0.11 percent actually means in the real world. Yet Sen. Pat Roberts has declared that the risk is “de minimis,” which is even more obscure. One highly knowledgeable scientist explains that it means, for example, that if 10,000 identical NBAFs were built, then 10 of them would be certain to release foot-and-mouth disease viruses over their 50-year lifetimes. But as with all such probability estimates, it predicts absolutely nothing about any single case — a single NBAF. It is just a theory.

What is worse is that the problem of understanding what a probability of 0.11 percent means in real life carries over to understanding what the uncertainty of such a value means. The same computer model that calculated the value of 0.11 percent also calculated that there is a 95 percent chance that it could be as high as 2.5 percent.  What does that mean to you? De minimis?

Perhaps in an attempt to help ordinary people understand these obscure, theoretical concepts, the authors converted from percent to dollars based on the economic consequences of a foot-and-mouth disease release. In these units they conclude, “The uncertainty (standard deviation) in the 50-year cumulative risk was found to be approximately $15B, regardless of whether catastrophic events are included.” To me, this says the economic risk is somewhere between $0 and $15 billion. De minimis?

We hope that the National Research Council commission has a more realistic grasp of limits of a statistical model based on limited and uncertain input data.

Tom Manney, 5514 Anderson Ave., is a professor emeritus of physics and biology at Kansas State University.

Terms of Service | Privacy Policy | The Manhattan Mercury, 318 North 5th Street, Manhattan, Kansas, 66502 | Copyright 2016