Can Program Evaluators do anything about data quality?

Since I started in evaluation in the early 90s, myself and my fellow evaluators have always complained about the quality of the data.  The only good that came from the complaints was that we were able to justify doing expensive surveys to replace the data that was not properly collected in the first place.  It seemed like this could go on forever but the response rates to surveys just kept getting lower and the computers that we used to analyse administrative data kept getting more powerful.

It is 2015, and all the years of complaining have not done anything.  This is even after the Canadian Auditor General found in 2009 that the bulk of the federal program evaluations suffered due to lack of data.  However, I would propose that there is something that can be done.  I was fortunate enough to be given some time to think about the challenge.  After a review of the literature, it became clear that many of the tools of program evaluation could be applied to data quality.

Will it be an easy sell?  I doubt it.  Everyone wants the quality of data to improve but they think that someone else should do it.  How many times have you heard about problems being identified but no one actually roles up their sleeves and replaces the wrong number with the right number. Fortunately, program evaluation on its good days can be an agent for organizational change.

The only downside that I can see to this is that we might overreach our expertise.  We can't forget that there are a lot of people who are trained in computer science and feel that they have something to say about data quality.  And to a large extent they are right.  For example, program evaluators may never have intelligent opinions on how pure an implementation of SQL should be.  However, it is evaluators that are well positioned to assess whether the data used by an organization is good enough to make strategic decisions

At the CES meeting in May 2015 in Montreal I will be floating my approach.  If I am crazy, at least it is for a noble cause.  Still I think that Program Evaluation has something to offer.  If you are interested, check out my paper at http://henskyconsulting.com/services/data-quality-evaluation.

Can you spot the outlier?