Huh? COVID-19 science and the uncertainty dance

My friend Tim Caufield, professor of law at the University of Alberta, the Research Director of its Health Law Institute, and current Canada Research Chair in Health Law and Policy, writes in an op-ed for the Globe and Mail that COVID public health policies have been with us for a year. So has uncertainty. We’ve all lived through twelve months of “huh”? And this has added to the public’s frustration, fatigue, and stress.

In the early weeks and months of the pandemic, there was uncertainty about masks and asymptomatic spread. There was uncertainty about if and when we’d get a vaccine. There was uncertainty about what type of public health policies worked best and were most needed. We have all had to tolerate a lot of ambiguity. And as the vaccines roll out, we are being asked to tolerate even more. (When will I get a vaccine? Which one will I get? And what about the variants?)

For public health communications to be effective, the public must have confidence in the message. And, unfortunately, for some, that confidence isn’t there. A recent study from the University of Calgary explored pandemic communication and found, not surprisingly, that “participants felt that public health messaging to date has been conflicting and at times unclear.”

This perception is understandable. An atmosphere of seemingly relentless uncertainty and confusion has been created by a combination of scientific realities, media practices, some less-than-ideal communication from policy makers, and the spread of misinformation and conspiracy theories.

The science surrounding COVID was – and, for some topics, continues to be – highly uncertain. While a growing body of evidence has emerged around the most contested issues (such as the value of masks and physical distancing strategies), early in the pandemic there wasn’t much that was unequivocal. The science evolved and, as you would hope with any evidence-informed approach, the resulting science advice and recommendations evolved too. But for some, shifting policies, even if appropriate, just added to a sense, rightly or not, of chaos.

In addition, the media has been reporting on the research as it unfolds, including referencing studies that have not yet been peer reviewed. Often the preliminary or uncertain nature of the relevant research is not reported in the media, thus creating a false impression about the actual state of the science – as exemplified by the “hydroxychloroquine works!” debacle (PS, it doesn’t). Perhaps worse, relatively fringe perspectives – such as those pushing the value of “natural herd immunity” – have been given a relatively high profile in both the conventional press and on social media. This can create a false balance (fringe idea vs. broad scientific consensus) that we know can be detrimental to both public discourse and health behaviours.

Despite the frustration that uncertainty can create, the public has a demonstrated preference for honesty about the limits of our knowledge. A recent study from Germany found that “a majority of respondents indicated a preference for open communication of scientific uncertainty in the context of the COVID-19 pandemic.” This finding agrees with other research that has found that when uncertainty is relevant to their lives, the public wants to know about it.

People may want to hear about uncertainty, but will communicating it do more harm than good? Will it just add to an already confused information environment? The data on this point are actually fairly mixed, but recent research exploring the impact of communicating scientific uncertainty found that it either increased perceptions of trust in science or had almost no impact. This is good news. As the authors of one of the studies notes, “this should allow academics and science communicators to be more transparent about the limits of human knowledge.” Other studies have found that being honest about uncertainties in media reports about research can actually boost the perceived credibility of journalists.

And over the long term, honesty about the uncertainties of the evidence used to inform policy seems essential to the maintenance of public trust. For example, being overly dogmatic about a policy or predictive model could hurt the credibility of decision makers if new evidence requires a revision of a past positions.

When possible, public health authorities (or anyone seeking to communicate science) should start with well-defined and well-supported takeaway messages (e.g., please get vaccinated with whatever vaccine is available to and recommended for you!).But then be honest about what is not known (e.g., while vaccines are our best defense, we aren’t sure how long immunity will last).

Depending on the medium used (a social media post, for example, may not be the best venue for a long discourse on methodological challenges), it may also be wise to explain the limits of the research approach (e.g., observational studies can’t prove causation). If there are areas of scientific disagreement, be honest about that too – but be specific about what is being disputed. Often there is broad agreement about the big stuff (e.g., vaccines work!), but academic debate about some details. Often those trying to sow doubt – like those in the anti-vaccine community – will try to weaponize and over-emphasize small academic disagreements. Don’t give them that room.

When communicating about uncertainty it is also important to highlight what is being done to reduce it, such as forthcoming research or new data analysis. This provides a road map forward and invites the public to follow the science as it unfolds. It is also a way to stress that uncertainty is a natural part of the scientific process.

For the public, try not to let the uncertainty kerfuffle distract you from the big picture. Remember that there are many clear knowns. Vaccines, physical distancing, hand washing, masks, and being responsible when symptoms emerge will get us through this pandemic.

Finally, it is also important to take a break from all the uncertainty noise. Studies have shown that the constant consumption of conflicting COVID news can (no surprise here) add to our stress. Put down the phone, back away from the screen, and take ten from “huh?”

The only thing certain is more uncertainty: Europe tries new uncertainty approach

The European Food Safety Authority (EFSA) has developed a harmonised approach to assessing and taking account of uncertainties in food safety, and animal and plant health. This approach will boost the transparency of the resulting scientific advice and make it more robust for decision-making.


The EFSA Scientific Committee guidance on uncertainty in scientific assessments offers a diverse toolbox of scientific methods and technical tools for uncertainty analysis. It is sufficiently flexible to be implemented in such diverse areas as plant pests, microbiological hazards and chemical substances.

Prof Tony Hardy, Chair of the Scientific Committee said: “Since 2016, we have tested, refined and tailored our new approach to uncertainty analysis, benefiting from open consultations with EFSA’s partners and the wider public. Crucially, we learnt a great deal about how to apply the new approach by trialling it across all EFSA’s scientific areas of activity.

The approach is described in two separate documents: a short user-friendly (says who?) guidance with practical instructions and tips, and a supporting scientific opinion with all the detailed scientific reasoning and methods.

The long-term goal is that the new guidance on uncertainty will be an integral step in all EFSA’s scientific assessments.

Prof Hans Verhagen is head of EFSA’s department for risk assessment. He said: “The trial showed that in areas like plant health, an explicit uncertainty analysis is already being used, with positive feedback from risk managers who say this helps them with their decision-making. In other areas, where uncertainty analysis is not yet integrated in the assessment process, the testing phase has helped give a clearer idea how to develop tailored approaches.”

EFSA will implement the approach in two stages. In general scientific areas, the guidance will apply from autumn 2018 after the renewal of the Authority’s scientific panels.

In regulated products areas such as pesticides, food additives or food contact materials it will be phased in later on, in light of the experience gained in the ‘non-regulated’ areas.

In parallel, EFSA is developing practical guidance for communication specialists on how to communicate the results of uncertainty analysis to different target audiences, including the public. A public consultation will be held on a draft of the communication approach in 2018.

Others have been working on this for 40 years. When the goal is public health – so more people don’t barf – we already know it’s better to go public early and oftern.

Going public: Early disclosure of food risks for the benefit of public health


NEHA, Volume 79.7, Pages 8-14

Benjamin Chapman, Maria Sol Erdozaim, Douglas Powell

Often during an outbreak of foodborne illness, there are health officials who have data indicating that there is a risk prior to notifying the public. During the lag period between the first public health signal and some release of public information, there are decision makers who are weighing evidence with the impacts of going public. Multiple agencies and analysts have lamented that there is not a common playbook or decision tree for how public health agencies determine what information to release and when. Regularly, health authorities suggest that how and when public information is released is evaluated on a case-by-case basis without sharing the steps and criteria used to make decisions. Information provision on its own is not enough. Risk communication, to be effective and grounded in behavior theory, should provide control measure options for risk management decisions. There is no indication in the literature that consumers benefit from paternalistic protection decisions to guard against information overload. A review of the risk communication literature related to outbreaks, as well as case studies of actual incidents, are explored and a blueprint for health authorities to follow is provided.

Drivers of uncertainty in estimates of foodborne gastroenteritis incidence

Background: Estimates of the incidence of foodborne illness are increasingly used at national and international levels to quantify the burden of disease and advocate for improvements in food safety. The calculation of such estimates involves multiple datasets and several disease multipliers, applied to dozens of pathogens. Unsurprisingly, this process often produces wide interval estimates.

infinity.dogmaMaterials and Methods: Using a model of foodborne gastroenteritis in Australia, we calculate the contribution of both data and multipliers to the width of the interval. We then compare pathogen-specific estimates of the proportion of gastroenteritis that is foodborne from national-level studies conducted in Canada, Greece, France, the Netherlands, New Zealand, the United Kingdom, and the United States.

Results: Overall, we estimate that 74% (range 63–92%) of the interval width for foodborne gastroenteritis in Australia is a result of uncertainty in the proportion of gastroenteritis that is due to contaminated food. Across national studies, we find considerable variability in point estimates and the width of interval estimates for the foodborne proportion for relatively common pathogens such as Salmonella spp., Campylobacter spp., and norovirus.

Conclusions: While some uncertainty in estimates of gastroenteritis incidence is inevitable, an understanding of the drivers of this uncertainty can help to focus further research. In particular, this work highlights the value of studies quantifying the routes of transmission for common pathogens.

Foodborne Pathogens and Disease. -Not available-, ahead of print. doi:10.1089/fpd.2014.1816.

Glass Kathryn, Ford Laura, and Kirk Martyn D.

Anyone can armchair quarterback; dealing with uncertainty in real-time during foodborne outbreaks

It’s not easy being a food safety investigator in the face of deep uncertainty.

So says Ron Doering, a past president of the Canadian Food Inspection Agency who practices food law in the Ottawa offices of Gowling Lafleur Henderson, in his latest Food Law column.

In the U.S., the largest foodborne outbreak in the last decade involved a rare strain of Salmonella Saintpaul thought to originate from tomatoes. The Food and Drug Administration (FDA) acted quickly, providing a public warning to avoid eating tomatoes until its investigation was complete. After several weeks and hundreds of tests and interviews, the FDA concluded that the problem was likely not tomatoes but rather hot peppers. Politicians rushed to microphones attacking the FDA for “destroying” the tomato industry. Of course, these are the same people who would have been outraged if tomatoes had been the source of the Salmonella and the FDA had not acted quickly.

While the FDA probably did the right things in the face of so much uncertainty, they should have communicated better. Tracing the source of foodborne illnesses is very complicated, especially for produce like tomatoes where there are no bar codes, no packages, and they are quickly consumed, often with other produce.

In the EU, the largest foodborne illness outbreak in the last decade, and one of the developed world’s most severe in modern times, took place this summer when a very rare strain of E. coli (O104:H4) got into the European food supply.

The first death was reported on May 24. The next day, the Robert Koch Institute announced that the early epidemiology indicated that the likely culprit was cucumbers, tomatoes or green salads. And later that day German officials announced that the rare strain had been found in the stools of five of the sick patients. The following day a German state-level agency announced triumphantly that it had found E. coli on Spanish cucumbers, though it had not yet tested for the strain.

On May 31, after testing for the strain, it was announced that the cucumbers were not to blame, by which time, of course, the Spanish cucumber industry was destroyed and German vegetable growers were suffering losses of $2.8 million per day as consumers quit eating all salads. Finally, on June 5 it was reported that “initial tests” (it was not tests, it was the result of epidemiological tracing) revealed that sprouts grown on an organic farm in Germany were “likely” the source of the problem, even though they couldn’t find a smoking sprout on the farm. Then on June 12 several victims of the O104 strain fell ill in France. They had no connection to the German organic farm. Attention turned to seeds, with the French blaming the British (I’m not making this up). Finally, on June 29 tracing determined the source of the problem as sprout seeds imported from Egypt in 2009. By this time there had been more than 50 deaths and well over 4,300 people seriously ill, including approximately 900 cases of patients with permanent kidney damage.

In both cases, with the benefit of hindsight, academics and the media roundly criticized the regulators as incompetent.

In a detailed review of the EU sprouts case, Peter Sandman, an expert on risk communication, concluded that the main failure of the regulators was in not being more forceful in proclaiming in their risk messaging the level of uncertainty in the case.

To me it seems there is a more basic problem, and it flows from the use of the language of risk, because “risk” disguises the deep uncertainty inherent in complex cases like these. These regulators were dealing with uncertainty, not risk — they were engaged in crisis management, not risk management.

E. coli O157:H7 and cattle diet – it’s messy

“If only” is often how statements begin by food safety wannabes who are sure they have stumbled upon a vast conspiracy meant to subjugate society.

If only Monsanto didn’t genetically-engineer seeds …

If only products like milk were served raw and natural …

If only cattle were fed grass, there would be no E. coli O157:H7.

When someone makes such proclamations, or says they speak fact, usually with an air of authority, I immediately think that person is full of it. People who say “trust me” are immediately untrustworthy.

Megan Jacob, Todd Callaway, and T.G. Nagaraja of Kansas State University write about the dietary interactions and interventions affecting Escherichia coli O157 colonization and shedding in cattle in an upcoming issue of Foodborne Pathogens and Disease. It’s not a movie, not a blog, not a pulp fiction, so they sorta have to get it right. And they do, when they write:

“The specific mechanisms responsible for increased or decreased E. coli O157 shedding or survival are not known … results of studies are conflicting or not repeatable, which speaks to the complexity of the hindgut ecosystem, variation in animal feed utilization, and variation within feed products.”

The complete abstract is below.

Escherichia coli O157 is an important foodborne pathogen affecting human health and the beef cattle industry. Contamination of carcasses at slaughter is correlated to the prevalence of E. coli O157 in cattle feces. Many associations have been made between dietary factors and E. coli O157 prevalence in cattle feces. Preharvest interventions, such as diet management, could reduce the fecal prevalence and diminish the impact of this adulterant. Dietary influences, including grain type and processing method, forage quality, and distillers grains have all been associated with E. coli O157 prevalence. In addition, several plant compounds, including phenolic acids and essential oils, have been proposed as in-feed intervention strategies. The specific mechanisms responsible for increased or decreased E. coli O157 shedding or survival are not known but are often attributed to changes in hindgut ecology induced by diet types. Some interventions may have a direct bacterial effect. Frequently, results of studies are conflicting or not repeatable, which speaks to the complexity of the hindgut ecosystem, variation in animal feed utilization, and variation within feed products. Still, understanding specific mechanisms, driven by diet influences, responsible for E. coli O157 shedding will aid in the development and implementation of better and practical preharvest intervention strategies.