Quality Analysis Evening with the Java User Group Lausanne

The Java User Group Lausanne (JUGL) organized last week an evening where Java quality analysis tools were compared side-by-side, analyzing the same open source project. More than 100 participants came to watch the presentations done by
* Coverity
* Headway Software
* Parasoft
* Sonar
* XDepend

Each tool was presented during 20 minutes, analyzing the same code base: the version 2 of the IceScrum open source project management tool. All tools have a different approach to quality, there are however a lot of common defects that are targeted by most of them like unit testing coverage, code duplication, dangerous code, code dependencies, coding standards respect.

The evening started with Sonar, the only open source tool of this presentation. Its aim is to produce quality dashboards (technical or management) of software development projects. The tool integrates existing commercial and open source specialized quality tools (PMD, Checkstyle, etc.) or plugins that are developed specially for Sonar to analyze the code under different perspectives. With its open architecture, every programming language could be covered by Sonar.

The next tool was JTest from Parasoft. It belongs to a wider quality framework and has both static and dynamic code analysis features. The static analysis comes with 1100 predefined rules and can also simulate code execution to produce a data flow analysis. The dynamic analysis is created by monitoring the code execution in the JVM.

XDepend was the third presentation. This tools focuses more on the design and architecture aspects of the projects. Inspired by the NDepend for .NET, the tool is strong on managing dependencies. Another interesting feature is the graphical representation of the relative importance of all components of the project where different metrics can be visualized. The tool bases its analysis on CQL that could be defined as a query language for code.

Structure 101 and Restructure 101 from Headway Software are also more focused on the architectural side of the application. Dependencies and complexity are analyzed by Structure 101. This provides the basis for refactoring the architecture with Restructure 10, improving the application quality without necessarily modifying the code.

Finally, Coverity Integrity Center offers a more classical code analysis vision. A particular feature is the capability to identify the same defect across the many branches of a product (current version, release n+1, etc.) that can exist in the configuration management system.

During the final discussion, questions were asked to all presenters on the customization options of their tools, the handling of “false positive” and where the analysis could and should be done (on the developer machine or on a shared team server). Two companies that couldn’t present during this session, Kalistick and Squoring, have analyzed the same code and put the results on the web.

Slides of the event
View the video of the event (mixing French and English) on the JUGL space of Parleys.com

Resources:

* Sonar presentation in Methods & Tools
* Delivering Working Code through Automation and Collaboration
* Improving on Unit Tests with Sonar