A few months ago, Paul Freet (VentureLab) reached out to me and asked if StarPound would be interested in being a beta tester for a new software testing platform (Tarantula) being developed by a team of researchers at Georgia Tech.  I forwarded the message on to our CTO, Wei Wang, who followed up with the team down there.  We are incredibly busy right now, and Wei didn’t think he would have enough spare time to dedicate to the task, despite the fact that we have wanted to bolster our QA processes for some time now.  I was a little disappointed (not at Wei, but at the situation), because I believe very strongly in QA, even in an agile environment.  Then, something wonderful happened.

The Tarantula team took the initiative, downloaded our open-source platform, and started running their own tests.  They are coming in next week (I think) to share their findings and get our feedback.  Kudos to the Tarantula team for thinking out of the box, and finding a way to get their product out there in the hands of users. There is most certainly a lesson there for other entrepreneurs - never take no for an answer - be creative - adapt, improvise, and overcome.  Be a real partner!

I am excited about the meeting, and I’ll post a followup here with the results of their findings.

If you aren’t familiar with Tarantula, here is a quick rundown:

Tarantula is a technique and tool for helping developers find bugs in source code.  The tool takes source coverage information from a set of test cases and produces a color-coded visual representation of the code. The color coding identifies areas of the code which are most likely to be buggy so that the developer may focus his/her attention in suspicious areas first.

For each statement in the source code, a suspiciousness and confidence value is computed. The suspiciousness is based on the relative numbers of passing and failing test cases that execute a statement. The confidence is based on the percentage of the total passing and failing test case sets that execute a statement. In the visualization, suspiciousness is represented by hue ranging from red to green; more suspicious statements appear more red and less suspicious statements appear more green. Confidence is represented by the brightness of the color; less confident statements appear dimmer and more confident statements appear brighter.

How does it work?

The Tarantula tool requires the source code, a set of test cases and per-test coverage information. While it has an independent data format that will support other tools in the future, the current implementation requires test cases written in JUnit and the use of Clover for coverage instrumentation. To use Tarantula, one instruments the source code with Clover and then runs a set of JUnit test cases. Assuming there are failing test cases to address, the data is imported from Clover reports into Tarantula. Then, the project is opened in Tarantula and the code is reviewed using the information it provides. The import functionality is available both as a wizard and a custom Ant task.

According to their FAQ, the original version worked only with code written in C, which makes it an prospective fit for Linux/UNIX system developers, embedded systems developers, etc.  StarPound is over one million lines of J2EE (not C), so it will be interesting to see how their approach and tools fare against a different code base.  Stay tuned!

Cheers.