There are different notions of what an unit test actually is. While most will agree that it is a test that verifies a pretty small part of a software system, we don’t all agree on how big a part this is. Personally I want a unit test to cover as little as possible. This, I think, makes the tests more manageable and makes it more likely that we can execute them with a minimum of fuss and in the shortest time possible. The following sums it up:
A unit test is an automated piece of code that invokes the smallest testable part of a system and then checks a single assumption about the behavior of that part.
Because a these tests verifies such a small part of the system I think a unit test should be in the same bundle as the code it is testing. And since it is only testing code contained withing the same bundle, it won’t need any extra dependencies (except for Junit). Setting this up when using the traditional PDE Build cannot be easily done without including test code in deployed bundles. As a workaround you can create a plug-in fragment to host unit tests. That will work, but causes other complications. Apart from the hassle of managing an extra bundle, you will for example not be able to verify platform specific code hosted in a fragment using tests hosted in another fragment. You’ll want these tests to be in the platform specific fragment.
Hosting the tests in the same bundle as the tested code is not very straightforward when using Tycho and Maven either, but it is possible and well worth the effort. I’d like to propose a solution.
The first thing we’ll do is to set up the project in the Eclipse IDE. As you may know, Maven by default expects a root src folder with main and test subfolders. This layout is not commonly found in Eclipse bundles, nor is it default when creating new plug-ins. Also we don’t want the eclipse-plugin packaged bundles to include the tests. So that’s why I elected to add another src-test folder instead. Do as follows:
- Add a new source folder for tests, src-test
- Exclude this in build.properties by adding
src.excludes = src-test/ - Add Junit libraries to the build path. Go to Project properties > Java Build Path > Libraries. Add “Junit 4”.
Unit tests now go in the src-test folder of the bundle. And because the dependecy to Junit 4 is added to project properties and not the MANIFEST.MF, we can compile and run the test code without the org.junit bundle ending up in the product. To hone you TDD skills you may now consider installing a tool for continusly running your unit tests. Infinitest is a good candidate.
Tycho will not automatically pick up these tests so we need to add some configuration in the pom.xml to let Maven know that we have these:
- The test source folder must be specified (src-test).
- The Maven compiler plug-in must be bound to the test-compile phase so the tests gets compiled.
- The Maven Surefire plug-in must be activated
${project.basedir}/src-test
...
org.apache.maven.plugins
maven-surefire-plugin
2.12.4
test
test
**/*Test.java
test
org.apache.maven.plugins
maven-compiler-plugin
2.5.1
compiletests
test-compile
testCompile
And that’s all there’s to it. Now, integration tests and UI tests should go into separate bundles packaged as eclipse-test-plugin as before. These will be executed in the Maven integration-test phase. Well after the test phase in which these tests will execute. So if there are any issues uncovered by the unit tests, the build will fail as soon as possible.
References:
Hi Torkild,
Interesting post, but would not take the size of a unit test as a very strong argument for putting it into the same bundle. We usually create fragments for testing bundle code, because we prefer to have – besides a clean separation of the project’s responsibilities – different compiler warnings/errors settings in the test fragments.
One way to avoid at least the hassle of creating the extra fragment project is to let a particular maven archetype do this for you. This way one can create a hole host-fragment-feature structure within seconds.
I also doubt that the JUnit dependency would be sufficient for me, as in general AssertJ, Mockito, JUnitParams and the like has proven themself as very useful. And in this case I assume there is no advantage in maintaining these dependencies in the project’s build path and pom rather than in the fragment’s manifest.
I do not have much experience with platform specific fragments, so this might be some kind of awkward and I will not argue here.
In general the decision of where to put the test code is probably based on a good deal of team conventions and personal taste. And as I am personally very happy about all people who are serious about unit testing I would not go over a war on this topic… 😉
Thanks Frank,
I agree – the size of test is not the best of arguments for putting it into the same bundle. I do find it very convenient though. In particular when running the unit tests more or less continuously and often switching between production code and tests. Also you bring up a good point with regards to different compiler settings. This can be handled in the pom.xml, but that will obviously not work for the UI.
However the most important point (which I failed to emphasize in the post); is that using a setup like this will allow you to run the unit tests in the Maven test phase. If your Unit tests are small they should fail early and fast, which I think is a good thing. If they are not small and take time to execute, you might as well put them in separate bundles alongside the integration tests.
Actually, the reason I started digging into this was a particular platform specific bundle which I could not get tested in the ordinary fashion. It was simply not possible.
And yes, where to put the test code is very much up to team convention. But it can be a factor in your builds performance, so one should be aware of the consequences.
I see what you are striving at. Indeed it also seems a question of different working patterns. We usually have different generic test suites for unit and integration tests. Both test types reside in the fragments, but they are separated by naming conventions. So it is easy to execute the fast running unit tests separately from the slower integration suite while working in the IDE.
Hi Torkild and Frank,
Do you have more feedback to provide two years later?
Are you still using the same patterns?
In Eclipse world, seems not to be so common to have “real” unit tests. I mean wihout launching an OSGi platform.
In my previous project, we were doing an Eclipse RCP application for which we didn’t provide any API and which was not intended to be integrated by externals. In this case we put the unit tests in the same bundle and using “optional” options for “test required plugins”.
I’m now on a project which can be integrated. So having these “optional” things is not acceptable.
Are you managing several “Libraries” for AssertJ, Mockito or other things? is it easy?
Regards,
Hi Aurélien. Yes, I pretty much do the same thing. When it comes to situations where Mockito and other libraries are required for testing I like to create fragments or at least separate bundles since these tests would typically be integration tests. However it should be possible to use i.e. Mockito in the same bundle as the code tested but you will have to do some adjustments to keep the test code out of the packaged bundle.