There has been a discussion recently, on the JUnit group, about doing unit testing with Spring. The arguments provided in the answers to that question reminded me of a problem that we faced in the project that I am currently working on: the usage of Spring for unit tests was a bad idea. Let me tell you how we started, and why we got to realise that it is not good to mix the well-known framework in the unit tests.When we started the project we thought that Spring is nice to add to the unit tests. After all, with just a few annotations, you have an instance of the class under test ready for testing. Spring takes care of creating and configuring all the dependencies required for our target instance, injects those dependencies, and we get an object that is ready for testing. However, were we writing unit tests or integration tests? I think the latter is the correct answer.
1. Testing half of the system at once
Therefore, the first problem with using Spring is that you don’t test just one class (that is, one unit). No sir, you are integrating and testing a part of your system, composed of your class under test but also the other services that this class is making use of. Even if you strive to only test your class (which is what you should do, since that is the unit test for a particular class, not for the entire system or for a particular functionality of the system) you will be forced to rely on other services, and hope that the other services will work correctly, and if your test fails, then the problem is in your class. Which brings us to the second problem.
2. When something breaks, you better be mr. Holmes
The problem nr 2 arises from the fact that your class is mixed with other services. Since it DOES use other services, this is the normal way for the class under test to work, but it is not ok for unit testing. For unit testing you need isolation. Why? Because only then you know that if a test fails, then you have a problem in either the class under test or the actual test that fails. You don’t have to search on 5 other services to make sure that the data retrieved from those services was correct and that the tests of the other services are correct (if they are correct, then at least you know that your class is the best candidate for the flaw owner). By making a change in a class and getting several tests to fail is a clear sign that your unit tests are not done in complete isolation and are depending one to another.
3. Working with test data
Preparing the test data for an unit test that is using Spring can be a pain in the ass. Since your services are processing and returning the data to the class under test, you need to know very well that kind of processing those services are doing, how should the input data look like, and you must tailor the input data so that after it gets through the processing of 5 services, it reaches your class under test in the shape that you want it. This, in most cases, involves custom test files and custom database records. A good approach to solve this problem is creating a fixed testing database content, which is used by all the tests. While this works for most cases, you will quickly find the need to throw some wild data to your class under test, just to make sure that you are also testing the limits and the special cases for the class methods (not only the happy paths). Since the setup for this wild data breaks the initial contract with static test data inserted in the database, you will have to resort to workarounds in order to reach your goal.
4. It’s only time…
With a good architecture and sound testing practices, you can make your live easier, even if you have to deal with the first 3 problems. However, what bothered us enough to decide that testing with Spring is bad was the time. Running a test was taking, in some cases, even 2 minutes to start. That is because Spring has a lot of dependencies to create, a lot of things to load and to start up, and it might take 2 minutes to setup the application context and create the instance of the class under test, until you get to see the first method executing. All in all, we found that half of the time the build system required to build our project was spent on creating and destroying Spring contexts.
Removing Spring from tests
Therefore, we decided to remove Spring from tests. We are using Mockito (we started with EasyMock but decided that Mockito’s methods and approach look better in code) to create mocks for the services that the class under test needs. Since Mockito allows you to make all the mocking and wiring through annotations, in some cases we don’t even need the @Before setup method. Moreover, with such mocks much more easier to prepare the test data. Since you only need to specify what data should be delivered to the class under test, you can focus on the class your are testing, and remove from your mind all the other services. You are effectively and efficiently testing your class in isolation.
Another nice thing with this approach is that you are forced to create your classes easy to test. Your classes will not be too coupled with other classes, because if they are then you will have a bad time writing the unit tests and you will like more and more to work with interfaces instead of concrete classes (even though Mockito and also EasyMock can create mocks for concrete classes).
Your tests will be blazingly fast, because all you have to wait for is the JVM to start. When something will break, it will surely be a problem in the class under test or the actual unit tests. You know this for sure, because everything that the class is using are mocks, and mocks do what they are told to do. They don’t have bugs outside the unit test.
Epilogue
To finish my story, we are currently in the process of removing Spring from the unit tests. Whenever we have to work on a test that has Spring, we first remove Spring from it, then we do the update that we had to do in the first place. In time, this showed that the tests are becoming simpler to read and understand (for someone looking at that test for the first time), the build time gets smaller and you also get the nice feeling of working with small unit test boxes that are completely under your control.
E
March 21, 2011 at 5:04 pm
IN my experience a spring application context should only be started for integration tests where you need the data sources and pools set up. For all true unit tests I always initialize the classes under test myself and manually inject collaborators(real classes or mocks) as needed (which should be easy since those classes are already being used in a spring context and will have setters to allow injection.)
Bruce
March 21, 2011 at 9:33 pm
I usually use Spring in my JUnit test to inject test properties and stub or mock classes
Bogdan
March 21, 2011 at 9:42 pm
In almost all cases I found to be more clear and easy for maintenance to not include external data in my tests. I prefer to create the data manually in the test rather than fetch it from some external source (however I am ok with creating dummy factory classes that are providing me with ready-to-use test data). The same stands for the external configuration of tests.
I think that scattering the pieces of a test in several locations (Java class, Spring context file, test property file) leads to complex unit tests. The simplicity and small quantity of code for the Java class file (which you get with this scattering) is just an illusion, in my opinion.
Jason
March 22, 2011 at 1:27 am
Yes, you shouldn’t be using spring in your unit tests. Otherwise they’re not unit test anymore, they’re integration tests.
Now when it comes to integration testing, you shouldn’t be re-using one huge spring config file for all the tests. You can come up with smaller config files that just contain the beans the test uses.
Spring provides annotation support to make this easy in some testing frameworks. And remember your tests can be spring beans too.
If you’re like me you’ll end up with just a few spring config files that you re-use between multiple tests. They describe logical subsets of the whole application.
The large single spring config file should have been split up into these subset to begin with. Its just another good refactor that is spured by writing the tests properly.