Thoughts on Automated vs. Manual Quality Assurance

When programming it is always hard to get the code looking and working right the first time around. More often than not, we miss the littlest, most minute details in our program code and it fails because of it. Those are the times when it is ultimately useful to get an objective, outside view of your code. 

In addition to having another programmer look and check for errors in your code, there are many automated tools that scan your code for possible errors. Some of these tools like Checkstyle and PMD scan your source code for things like formatting and code efficiency errors. Other tools like FindBugs scan actual byte code for bugs. Using these sort of tools can make your code optimal and up to standard. But how do they compare to actual human quality assurance checks?

I ran Checkstyle, PMD, and FindBugs on my CodeRuler solution to see what sort of errors were found, and compared these results to what two of my other classmates found in my code. Here's what the automated tools found:

• Checkstyle : 24 errors in MyRuler.java (67 errors total including IBM classes)

1.         20 errors due to lines over 100 characters long
   
2.         3 Javadoc errors
   
3.         1 error due to using a import wildcard statement '*'
   

• PMD: 10 errors in MyRuler.java (36 errors total including IBM classes)

1.         All priority 3 errors with code suggestions
   

• FindBugs: 0 errors in MyRuler.java

As this was my first time using these automated tools I was surprised at how easy and simple they are to use. Instantaneously errors are found and presented in an extremely efficient way. Checkstyle proved to find many formatting errors and described them all well. With PMD I was surprised to find that they not only show the errors but links to full descriptions with examples. How great is that! FindBugs found no errors but I'm sure if the code was more substantial and complicated it would have.

CONCLUSION

Compared to Manual QA checks, I think the automated tools did an amazing job (at least with my CodeRuler classes). When Arthur Shum and Aric West did a check on my code they found many of the same problems - wildcard import statement, long lines of code, javadoc problems, etc. However, what they were able to find were more complex formatting problems such as redundant inline-comments, confusing code segments, and bracket placement. A lot of these problems would essentially make the code even more readable and Checkstyle did not find them. I think a Manual QA check is essential if your code needs to adhere to very specific standards. 

PMD found some errors that I think would take another programmer awhile to point out, especially with long code. It is possible that a programmer could find all these same errors. However, PMD finds them instantly and presents them so well. I actually learned a thing or two from looking at the errors in my code. The errors gave great suggestions into making the code more efficient and readable. 

Overall, what I think makes Automated QA so great is that is instant. These tools can be acquired and used in a matter of minutes. What's more is that using them can even teach you new tricks. If there are still problems in your code, such as logical errors, then manual checks can be done. From what I've seen by doing this, using a combination of both automated and manual quality assurance checks can make your code optimum.