Let's Be Better Knowledge Managers!

At this time of year, towards the end of the December, we are always making resolutions.
Losing weight and getting more exercise are always popular.  I would propose that everyone should consider making an addition to their list.  We should all try to be better knowledge managers!

Why knowledge management?  Essentially, we are at a turning point in our history.  The cost of storing information has dropped to essentially zero. A five terabyte drive is sold for only two or three hundred dollars.  We are not at a point where there is no excuse to delete files.  However, one problem remains: it takes time to manage the files.

We all know that we exercise and diet less than we like to.  This is in in spite of our best intentions.  It is also obvious that we either let information accumulate and clutter our drives.  After a while it becomes impossible to find anything even with the best search engines. Basically, we have become buried in information.  It is now obvious that better at knowledge management is easier said than done.  It takes work to organize information and synthesize it into knowledge so that we make use of it at a later date.  It takes even more work for it to useful to others!

A big part of the cost of managing knowledge is the preparation and organization of material, so that it can be effectively used at a later date.  No report or study is produced in a vacuum.  The report can only be understood with an awareness of the context surrounding its production.  A key part of this is always the target readers.  Many reports are initially produced with a narrow target audience in mind, yet can become useful to a wider audience if the specialized jargon and context are broadened.

How will we know if we succeed?  The first test will be how long it takes to find your own reports. However, the ultimate proof will come if individuals are able find and using your reports without having to ask you for a copy.

So I know what one of my New Years Resolutions will be!

The Knowledge Management Question is whether to scan these documents in or throw them out

Social Media as a New Source of Data for Program Evaluation

In many fields, analysts are discovering the use and value of what is known as “Big Data”.  In some senses, program evaluators are ahead of the curve, as they have been making use of administrative data for decades.  However, when it comes to the use of data from Social Media, evaluators have something to learn from other fields.  Evaluators also have to forge their own path, as our passion for capturing causality is not shared by others.

Why is Social Media useful?  A substantial portion of the population shares their experiences with the rest of the world through this media.  Both fortunately and unfortunately, a small percentage of these experiences relate to government programs.  Unfortunately, this amounts to finding a needle in a haystack.  Fortunately, it is possible to harvest  a new data source with sufficient effort.

It is true that the identification of program effects will be a challenge.  At times it may seem as if the evaluators are trying to compare needles in a haystack of participants with the haystack of non-participants.  Further challenges come in finding ways to work with enormous volumes of low-quality data without introducing biases.  And this can only be done after a way has been found to turn human discourse into data.

Is it worth effort?  Many evaluators will treat Social Media as a fad and ignore it.  For others, it will be a truly new source data.  The immediacy of the client experiences will provide insights not possible with other data sources.  Plus, they may find it will become possible to compare the experience of citizens in different countries.

Privacy issues will also present a challenge in the use of this data.  Governments will need to be able to prove that they are not spying on individuals.  Still on the flipside, governments are open to charges of arrogance if they do not listen to citizens as they express themselves through this media.
These are exciting times for evaluators.  I will be making my own contribution to this discussion at the 2016 Meeting of the Canadian Evaluation Society with a paper using Twitter Data.  Have a look at the most recent draft here.  I am sure that other will have different opinions.  We are guaranteed many lively discussions.

Sometimes it can be hard to find that needle

Other times it is easy