Does anyone know of a good Perl unit test generator?

How broad/deep should you do unit-testing in building data analysis program?

• As you know unit-testing is critical in making sure your algorithms are correct. What I also believe is that when your requirement or purpose is fast-changing, there's no sense to write unit testing. Also, for data analysis program, it would take a great deal of time to construct test data for unit-testing. Anyone who could shed some light on how unit-testing is done in some rapid data analysis?

• Answer:

  I don't think anyone uses traditional unit tests in rapid prototype development for data anaylsis. There are multiple reasons for this: Very often, you will be using or building on standard tools or libraries, which have been tested already, and have intuitive APIs. So the complexity of your code will be limited, there aren't substantial algorithmic mistakes you can make, Very often data analysis tools do not give a precise answer. For example, if they can be based on stochastic gradient descent or Markov chain Monte Carlo, so in theory, running them twice can give numerically different results. The technique roughly equivalent to testing in predictive data analysis is cross validation. This involves splitting your data into distinct training and validation/test sets. You do your analysis (fit the model) on the training set, and then you evaluate the predictive performance of the model on the held-out test set. This helps avoiding the common problem of overfitting, and also highlights most bugs or algorithmic mistakes that may exist in your code. Strictly speaking this process is more analogous to integration testing. Very often, when researchers develop new techniques, they test their algorithms with synthetic toy datasets where they know roughly what to expect. You can think of this process as a form of testing, although in very few cases do researchers apply the level of rigour that software engineers would.