Just as war makes every citizen into an amateur geographer and tactician, a pandemic makes epidemiologists of us all. Instead of maps with colored pins, we have charts of exposure and death counts; people on the street argue about infection fatality rates and herd immunity the way they might have debated wartime strategies and alliances in the past. The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic has brought statistics and uncertainty assessment into public discourse to an extent rarely seen except in election season and the occasional billion-dollar lottery jackpot. In this paper, we reflect on our role as statisticians and epidemiologists and lay out some of the challenges that arise in measuring and communicating our uncertainty about the behavior of a never-before-seen infectious disease. We look at the problem from multiple directions, including the challenges of estimating the case fatality rate (i.e., proportion of individuals who will die from the disease), the rate of transmission from person to person, and even the number of cases circulating in the population at any time. We advocate for an approach that is more transparent about the limitations of statistical and mathematical models as representations of reality and suggest some ways to ensure better representation and communication of uncertainty in future public health emergencies.