BLS accuracy
From economist Justin Wolfers on BlueSky, the record of payroll employment revisions by the US Bureau of Labor Statistics
First: look at 2020!
Next, though, the purple and green lines are quite close together compared to the scale of year-to-year change even when there isn’t a pandemic.
On the other hand, people do actually care about differences of the size we see between the initial and revised estimates, as is demonstrated by the stock market reaction to the revisions. What this really shows is how difficult the estimation problem is. People care deeply about changes that are almost invisibly small on this graph, and that are right at the limit of what’s feasible statistically.
The ideal solution is probably for people to be more relaxed about small changes in estimated payroll employment, just as the ideal solution for political opinion polling discourse is for people to have a more realistic view of the limits of estimation. Alternatively, if people want to be unrelaxed about small differences, they need to be willing to pay more to get better estimates.
Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »