Monday, August 23, 2010

Regarding Government Numbers

One of the oldest arguments on the internet is the government -- or non-government numbers for that matter -- are deeply flawed and therefore useless in economic analysis. This argument falls short in one very important way: there is no proof.

Purveyors of this theory first point to the website Shadow Stats as evidence. However they are wrong. For example, according to their "methodology" the US inflation rate was over 5% for the entire decade of 2000-2010.


However during this same time the US 10-year Treasury crossed 5% three times:


Either Shadowstats is correct or one of the most liquid and heavily traded markets in the world is correct. Sorry Shadowstats, but you're way off.

Then there is the issue of "survivor bias." This theory states that surveys are biased in favor of firms that survive economic hard times, thereby skewing the numbers positively. Unfortunately, this theory is also wrong as we debunked here a while ago. Here is an example from the Census Bureau regarding retall sales.

"Births are added to the monthly survey in February, May, August, and November of each year. At the same time, deaths are removed from the survey. To minimize the effect of births and deaths on the month-to-month change estimates, we phase-in these changes by incrementally increasing the sampling weights of the births and decreasing the sampling weights of the deaths in a similar fashion. In the first month, we tabulate the births at one-third their sampling weight and tabulate the deaths at two-thirds their sampling weight. In the second month, we tabulate the births at two-thirds their sampling weight and tabulate the deaths at one-third their sampling weight. In the third month, we tabulate the births at their full sampling weight and the deaths are dropped (sampling weight equal zero)."

Then there's the issue that no one has issued any research on this topic. Here's a fun exercise. Go to SSRN -- the Social Science Research Network -- and start searching for papers from economists and statisticians on the failure of government numbers. If the problem were as widespread as some think, then there should be evidence as in an entire body of work indicting the government statistical system. However -- there is no such body of work; it does not exist.

The short version is people who have absolutely no training in statistics think they know more than the people who staff government and professional forecasting agencies -- who also happen to be trained in statistics.

The reason for this entry is to reiterate a central rule of the comments section:

2.) If you are going to challenge the veracity of government statistics you must provide a reference from a paper written by someone with at least a masters in a relevant discipline (statistics, mathematics, economics etc...). There has been a raging debate in the economic blogosphere about government statistics. The classic debate is about the birth/death model used by the Bureau of Labor Statistics. (To find out more, go to the Bureau of Labor Statistics and type in birth death in the search bar in the upper right hand corner). The BLS uses this to overcome sampling errors in their employment statistics. According to some bloggers this is a bogus adjustment which makes the numbers unreliable. However, go to www.ssrn.com -- the social science research network -- and type in birth/death in the search bar. You'll find 34 hits that center around health care systems. But there is nothing about the BLS' birth/death model. In other words, among academics in the economic and financial world, there isn't a debate (at least not yet). So, the people who should be calling bullshit -- and backing it up with data and information -- are not calling bullshit. When they are, I'll be happy to consider the information.

And please -- Shadowstats is crap.

And a second reason is people have a habit of saying, "the government statistics are wrong" when the statistics disagree with their assertions. But when the statistics confirm their assertions, the government numbers are sacrosanct.

Bottom line: these are the numbers economists use. When I see an economist with a Ph.D. say, "these numbers are flawed and I can prove it" then I'll listen. But when a guy on a blog with a political ax to grind says the same thing, well, let's just say credibility is an issue at that point.

So here's the deal. If you're going to argue that economic numbers are wrong on this blog, please prove it. And, no, your word isn't good enough. Find someone who can prove they know what they are talking about (that's what PhDs are for) to back-up your assertion. And make sure the entire paper is about the topic, not just a sentence you can mis-quote and mis-construe. Better yet, find a group of people to back-up your assertion. Until then, Mish and Daily Kos are open for business.