Friday, 5 June 2015

SPSS 40 Years Later

SPSS was the first package that I used as a third-year student studying econometrics. This was in 1978 almost 10 years after the package first appeared. After almost 40 later, I found myself at a site where SPPS was the only package available. What had changed?

It is certainly true that there is a nice user interface on top of the old system.  In fact, it is probably the easiest package for anyone with no experience to do some very basic analysis.  However, at the core of the system there are still bugs.  It is hard to believe that after all this time basic operations can be unreliable.

What I ran up against was problems with the equation parser.  Some commands were not able to translate simple equations and would fail.  How to prevent this is something that one would study in an undergraduate computer science course.  I myself have written complex equation parsers in C++. It was difficult to do, but once it was done the parsers were robust and reliable.  It is this background that gives me the confidence to be surprised at this result.

However, initially, I thought that it must have be me that was at fault. After all it is a package that has been around for 40 years.  Fortunately, Google was my friend and there were others that had experienced the same.  Further research revealed that there even has been discussions of the reliability of the actual estimates coming from the statistical software.  These discussions date back to articles in reviews such as the Journal of Economic Literature in 1999.

Why does this situation exist?  Basically, people are too trusting and do not complain enough.  Now the decisions about statistical software are made by non-statisticians.  Unfortunately, they trust the men in suits from big companies when superior free software such as R exists.