Ökonomie zum Nachrechnen
In der Naturwissenschaft ist es ein Grundprinzip, dass Versuche wiederholbar sein müssen. Auch in der Ökonomie sollten alle Forschungsergebnisse zumindest nachvollziehbar sein. Das ist aber nicht immer der Fall.
“Economists treat replication the way teenagers treat chastity—as an ideal to be professed but not to be practiced.” This quote from a recent paper by Daniel Hamermesh (NBER working paper 13026) describes the current situation in the field of economic research very accurately. And this is wrong. Real scientific progress can only be made if empirical studies can be replicated. Unfortunately, editors of scientific journals are not very fond of papers replicating previous research as I recently experienced myself.
In a recent project with Richard Jong-A-Pin, we replicated a study by Ricardo Hausmann, Lant Pritchett and Dani Rodrik—abbreviated here as HPR—that was published in the Journal of Economic Growth (2005). In this very innovative paper, HPR examine whether political regime changes and economic reforms precede so-called growth accelerations. A growth acceleration is a sustained increase in economic growth. HPR identify more than 80 growth accelerations since the 1950s, which tend to be highly unpredictable. They find that a political regime change increases the probability of a growth acceleration by 5.3 percentage points while economic reforms are not related to growth accelerations.
Professor Rodrik kindly provided the data used by HPR. When we replicated this study, we discovered that HPR were led astray by a data-description error in the source they used to identify political regime changes. When we corrected this error, we found that political regime changes are not related to the probability that a growth acceleration occurs.
We submitted our paper to the Journal of Economic Growth. The paper was rejected on the basis of the argument that our note was a “welcome correction, however, of limited significance for the main contribution of the original paper.” However, in their abstract, HPR state that one of their main conclusions is that “Political regime changes are statistically significant predictors of growth accelerations.” As our paper was a comment on a previously published paper in the Journal of Economic Growth, it is unlikely to be accepted by another journal. However, a relatively new electronic journal called Econ Journal Watch, recognizes the importance of replication in economics. The editor of that journal, Dan Klein, was therefore happy to publish our paper. It will be published in the first issue of 2008. Of course, HPR get the opportunity to reply to our critique.
Even though I am very happy with this new outlet, I feel that editors of all scientific journals should pay much more attention to replication. A starting point is that authors of published empirical research should commit to make their data available to anyone interested. Unfortunately, even this is not common practice. As B.D. McCullough shows in a paper published in Econ Journal Watch in 2007, even those (few) journals that have a formal policy that authors should make their data available, in practice often fail to implement this policy. And even if the data were available, it turned out that often those studies often could not be replicated. McCullough concludes that all available evidence indicates that replicable economic research is the exception and not the rule. I am afraid he is right.
Jakob de Haan is Professor of Political Economy at the University of Groningen
Von Jakob de Haan