Upon rereading the classic 1976 Lucas macroeconomic article on policy evaluation, it
struck me that program evaluators as of 2018 still had not caught up. The essential point
of Lucas was that the very existence of a policy may change behaviour quite apart from
any real impacts. However upon rereading his examples, they seemed to be too exaggerated
to be applicable to the real world. Still that did not matter. The approach to macroeconomic
modelling that he critiqued produced forecasts that were wrong and he was seen as been right.
of Lucas was that the very existence of a policy may change behaviour quite apart from
any real impacts. However upon rereading his examples, they seemed to be too exaggerated
to be applicable to the real world. Still that did not matter. The approach to macroeconomic
modelling that he critiqued produced forecasts that were wrong and he was seen as been right.
The ability to predict has a huge impact on credibility. The proponents of big data appear to
be rewriting the rules of the game with their focus on prediction. In their world, model selection
is primarily driven by how well the model can predict the values of observations randomly
left out of the sample. R-square and T-test only play a very minor role. After all, if your
predictions are making money, nothing else matters.
be rewriting the rules of the game with their focus on prediction. In their world, model selection
is primarily driven by how well the model can predict the values of observations randomly
left out of the sample. R-square and T-test only play a very minor role. After all, if your
predictions are making money, nothing else matters.
So what does this mean for program evaluators? Perhaps if program evaluators were able
to use their corpus of knowledge to predict impacts resulting to changes in programs, we
could rewrite the rules of our game. I know that if inflation returns to a world where huge
budget deficits are occurring at the same time as very low unemployment rates, there will be
some macroeconomic modelers saying I told you so. Do evaluators know enough to go out
on a limb and start making predictions? If we do, then our role in policy formation could move
to the next level.
to use their corpus of knowledge to predict impacts resulting to changes in programs, we
could rewrite the rules of our game. I know that if inflation returns to a world where huge
budget deficits are occurring at the same time as very low unemployment rates, there will be
some macroeconomic modelers saying I told you so. Do evaluators know enough to go out
on a limb and start making predictions? If we do, then our role in policy formation could move
to the next level.
 |
| Two crystal balls provide two different views of the future |
