Krugman’s latest on Reinhart and Rogoff, which makes the rather obvious point that the meaning of the data is illuminated by looking at it on a country-by-country basis and by acknowledging country specific circumstances, raises a fundamental question for the social sciences: Whether the emphasis on data analysis is undermining the quality of social science research.
The biggest problem with data analysis is that it limits the discourse to things about which we have data series. Important questions are not addressed, because like the drunk looking for his keys, researchers are disincentivized from spending their time on them in the absence of good data.
The Reinhart Rogoff kerfluffle focuses attention on a secondary problem with data analysis: that it’s all about the underlying model, and typically represents a very narrow test built on very strong assumptions. Of course, the best data analysts understand this and some work genuinely addresses these concerns. However, the way data analysis is consumed and the approaches taken by average data analysts often fail to reflect how very little we learn from data analysis — in no small part because an argument is always more convincing if you deflect attention from its weaknesses rather than exposing those weaknesses to open debate.
For example, there’s nothing wrong with treating each high debt episode as a single event whether it comprises 19 years or 1, after all data analysis is all about heroic assumptions. There is, however, arguably an issue with doing this and not making it crystal clear — at least in a footnote or appendix — what your heroic assumptions are. (Reinhart and Rogoff’s critics allege that they failed to do this.) Afterall, understanding the heroic assumptions is essential to interpreting the research.
If it is the norm — and I don’t read enough of this literature carefully to know — to hide heroic assumptions and force other researchers to learn about them by reviewing your data set, then the idea that any conclusions drawn from any of this literature analyzing data are worth paying attention to is thrown into doubt.
Data analysis is a useful tool, but should be considered a secondary backup to broader, more nuanced historical analysis that goes far beyond the inherently simplistic nature of data analysis. Narrative historical approaches are currently undervalued, and it is often argued that their weakness that they are not disciplined by formal models. It is time to recognize, first, that this is also the strength of historical analysis and, second, that the fact that data analysis is disciplined by formal models is also a weakness.
Synthetic Assets 