In the previous installments I created a repeatable Maven build from Hudson, and I set up static code analysis using SonarQube. It was revealed that there is considerable technical debt, and half of that is caused by 0% Coverage.
Getting Test Coverage in place has to be done before tackling any of the other issues like duplication, complexity and violations.
Project Setup
The Project setup for the Industrial SQL Connector for Mylyn is slightly different from other examples found on the internet.
The main abstract functionality is handled in the main two plugins:
com.industrialtsi.mylyn.core for headless stuff and connection handling, andcom.industrialtsi.mylyn.ui for the UI part.
Functionality for a specific kind of database is then "injected" using a fragment project. Three example projects are included:
com.industrialtsi.mylyn.demo.memory is a very simple in memory task list for demonstration and testing purposes.com.industrialtsi.mylyn.demo.jpa accesses a simple Derby database but using EclipseLink JPA annotationscom.industrialtsi.mylyn.demo.derby contains a link to a Derby database using the apache ibatis xml based SQL access language, that will allow quite complicated JOIN and UNION statements in its queries.org.apache.ibatis finally contains the ibatis stuff packaged in a separate plugin.
All the test code for all these projects finally is placed in a single plugin:
com.industrialtsi.mylyn.tests contains al test code.
Getting the test code to compile under Tycho
The first step is to add the test project to the main POM in
industrialtsi.mylyn.maven.
When we run a simple maven build compilation fails:
[ERROR] Failed to execute goal org.eclipse.tycho:tycho-compiler-plugin:0.19.0:compile (default-compile) on project com.industrialtsi.mylyn.tests: Compilation failure: Compilation failure:
[ERROR] /Users/maarten/Workspaces/workspace-industrial-google/com.industrialtsi.mylyn.tests/src/com/industrialtsi/mylyn/demo/derby/test/DerbyIbatisPersistorTest.java:[18]
[ERROR] import com.industrialtsi.mylyn.core.persistence.DerbyIbatisPersistor;
[ERROR] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ERROR] The import com.industrialtsi.mylyn.core.persistence.DerbyIbatisPersistor cannot be resolved
[ERROR] /Users/maarten/Workspaces/workspace-industrial-google/com.industrialtsi.mylyn.tests/src/com/industrialtsi/mylyn/demo/derby/test/DerbyIbatisPersistorTest.java:[43]
[ERROR] return new DerbyIbatisPersistor();
[ERROR] ^^^^^^^^^^^^^^^^^^^^
[ERROR] DerbyIbatisPersistor cannot be resolved to a type
[ERROR] 2 problems (2 errors)
When compiling in the workspace, eclipse is very helpful in finding all the needed dependencies.
As the missing import is located in a fragment of another plugin, normal dependency tricks via the MANIFEST.MF of
com.industrialtsi.mylyn.tests or as dependency in the POM file don't work.
The quick solution was to include the missing jar as
Extra Classpath Entry in the
build.properties of
com.industrialtsi.mylyn.tests. If anybody knows a better way, please let me know!
Adding tycho-surefire to pom.xml
Next we need to configure the tycho surefire plugin to run the tests and process the results.
I'm can rely on the
tycho-surefire plugin to load my tests plugin and its direct dependencies.
But I want to run a full workbench which I define with the
<application> and
<product> tags. I also want the
com.industrialtsi.mylyn.demo.memory and
com.industrialtsi.mylyn.demo.derby fragments to be loaded, which I do with the
<dependencies> section. Then I disable the more complicated test till later.
<plugin>
<groupId>org.eclipse.tycho</groupId>
<artifactId>tycho-surefire-plugin</artifactId>
<version>${tycho-version}</version>
<configuration>
<argLine>${ui.test.vmargs}</argLine>
<useUIHarness>true</useUIHarness>
<useUIThread>true</useUIThread>
<product>org.eclipse.platform.ide</product>
<application>org.eclipse.ui.ide.workbench</application>
<dependencies>
<dependency>
<type>eclipse-plugin</type>
<artifactId>com.industrialtsi.mylyn.demo.memory</artifactId>
<version>0.9.10</version>
</dependency>
<dependency>
<type>eclipse-plugin</type>
<artifactId>com.industrialtsi.mylyn.demo.derby</artifactId>
<version>0.9.10</version>
</dependency>
</dependencies>
<includes>
<include>**/*Test.java</include>
</includes>
<excludes>
<exclude>**/IbatisPersistorTest.*</exclude>
<exclude>**/PersistorsManagerTest.*</exclude>
<exclude>**/DemoDerbyTest.*</exclude>
<exclude>**/DerbyIbatisPersistorTest.*</exclude>
<exclude>**/IbatisCorePluginTest.*</exclude>
<exclude>**/TaskCreationTest.*</exclude>
</excludes>
<!-- Kill test JVM if tests take more than 10 minutes (600 seconds)
to finish -->
<forkedProcessTimeoutInSeconds>600</forkedProcessTimeoutInSeconds>
</configuration>
</plugin>
Controlling memory and start Thread on Mac
Running on a Mac requires different startup parameters passed with the
<argLine> tag.
This is handeld best in the profiles section of the POM:
<profiles>
<profile>
<id>macosx</id>
<activation>
<os>
<family>mac</family>
</os>
</activation>
<properties>
<ui.test.vmargs>-Xmx512m -XX:MaxPermSize=256m -XstartOnFirstThread</ui.test.vmargs>
</properties>
</profile>
<profile>
<id>other-os</id>
<activation>
<os>
<family>!mac</family>
</os>
</activation>
<properties>
<ui.test.vmargs>-Xmx512m -XX:MaxPermSize=256m</ui.test.vmargs>
</properties>
</profile>
</profiles>
ready to run the tests in the workspace
Then we need to run a Tycho build using the m2e tools. It is important to realize that Plug-in Unit Tests are run in the integration test phase, so we need to specify that as a target.
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running com.industrialtsi.mylyn.core.dto.IndustrialQueryParamsTest
Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.039 sec
Running com.industrialtsi.mylyn.test.db.core.GenericQueryParamsTest
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec
Results :
Tests run: 23, Failures: 0, Errors: 0, Skipped: 0
[INFO] All tests passed!
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] com.industrialtsi.mylyn.maven ..................... SUCCESS [0.073s]
[INFO] org.apache.ibatis ................................. SUCCESS [1.477s]
[INFO] com.industrialtsi.mylyn.core ...................... SUCCESS [1.605s]
[INFO] com.industrialtsi.mylyn.demo.memory ............... SUCCESS [0.386s]
[INFO] com.industrialtsi.mylyn.ui ........................ SUCCESS [0.997s]
[INFO] com.industrialtsi.mylyn.feature ................... SUCCESS [2.616s]
[INFO] com.industrialtsi.mylyn.demo.derby ................ SUCCESS [0.685s]
[INFO] org.apache.ibatis.feature ......................... SUCCESS [2.321s]
[INFO] com.industrialtsi.mylyn.demo.derby.feature ........ SUCCESS [2.329s]
[INFO] com.industrialtsi.mylyn.demo.jpa .................. SUCCESS [1.254s]
[INFO] com.industrialtsi.mylyn.demo.jpa.feature .......... SUCCESS [2.399s]
[INFO] com.industrialtsi.mylyn.site ...................... SUCCESS [3.129s]
[INFO] com.industrialtsi.mylyn.tests ..................... SUCCESS [7.804s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 57.488s
[INFO] Finished at: Tue Jan 21 11:33:31 CET 2014
[INFO] Final Memory: 59M/554M
[INFO] ------------------------------------------------------------------------
Adding Jacoco coverage to the pom.xml
Lets start with adding the Jacoco plugin to the POM file.
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.6.4.201312101107</version>
<executions>
<execution>
<id>prepare-integration-tests</id>
<phase>pre-integration-test</phase>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<include>com.industrialtsi.*</include>
<include>org.junit.*</include>
<!-- Where to put jacoco coverage report -->
<destFile>${sonar.jacoco.reportPath}</destFile>
<append>true</append>
</configuration>
</execution>
</executions>
</plugin>
Configuring a multi-module plugin project for coverage with Jacoco has some special hurdles which easily lead to frustration.
All coverage code must be placed in a single
*.exec file to analyzed later, this is handled by the
<destFile>${sonar.jacoco.reportPath}</destFile> line. This property is defined in the properties section:
<sonar.jacoco.reportPath>${project.build.directory}/../../com.industrialtsi.mylyn.tests/target/jacoco.exec</sonar.jacoco.reportPath>
Tycho has a special
argline variable to pass to the tycho-surefire plugin named
tycho.testArgLine that we must pass on in the tycho-surefire plugin:
<artifactId>tycho-surefire-plugin</artifactId>
<version>${tycho-version}</version>
<configuration>
<argLine>${ui.test.vmargs} ${tycho.testArgLine}</argLine>
Now we can run again and get to see that
jacoco.exec is actually generated in the workspace.
We can now run the unit tests from the test plugin and measure code coverage with jacoco from a maven build.
not to be forgotten: SonarQube setup
We are running SonarQube from Hudson using the
Run Standalone Sonar Analysis build step.
We want that analysis to reuse the
jacoco.exec file generated during the maven integration-test step.
The standalone step uses sonar-project.properties for configuration. So we add:
sonar.dynamicAnalysis=reuseReports
sonar.jacoco.reportPath=../com.industrialtsi.mylyn.tests/target/jacoco.exec
If we had run SonarQube from maven using the sonar:sonar or verify targets, we should configure sonar in the pom.xml.
<properties>
...
<sonar.java.coveragePlugin>jacoco</sonar.java.coveragePlugin>
<sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
...
</properties>
I do both know that I'm working on it so I have flexibility later.
Now commit to SCM and start a Hudson build
Start a Hudson build manually or wait for SCM polling to kick in, but the result looks promising:
3.63% Coverage is not much, but easy to expand now that this works.
Finally: results in SonarQube
I managed to reduce Technical Debt with $ 390 and one man day compared to the start of this blog post. Seems like a low yield for a half day work, but I guess the calculations do not allow for the ramp up cost: running the first coverage is hardest. Progress should be easier now that the build and static analysis infrastructure is in place.
Unit Test Coverage is up from 0% with only the two simplest tests executing.
Next steps are activating the tests disabled earlier and adding more test.
To be continued...
To make this happen with Workspace Mechanic install this file in your 
The Technical Debt widget tell us the technical debt in days, and also a percentage split of the main problems.
Lack of coverage explains half the debt, with design, comments and complexity as other issues. There are very few violations and duplications, the happy result of running with Checkstyle, Findbugs and PMD inside Eclipse during the development.
I always add a Most Violated Rules widget to the project dashboard. Exposing internal representation is the most common here as the very bad package cycles.
The "Most Violated Resources" widget instantly tells me that the objects used to ferry query parameters around are the main problem area. I have a tagged sonar-project.properties file so I will be able to see progress in the future on lines of code, technical debt and documented API.
Issues are mostly Critical and Major, so need to be fixed urgently. 1% duplications is not that serious, even though 0% is best of course. Test coverage is more serious as we've seen above that lack of Test Coverage accounts for half of technical debt. This maybe because the build is not yet configured to execute tests.
The complexity stats show mainly whether design is good, a method should do one thing, a class should have one responsibility. The LCOM4 measure of 1.0/class is quite positive.
The main problem in these is the startling 40.5% of package tangle index and more than 10 cycles!
This needs to be looked at with highest priority.