Post date: Oct 27, 2013 11:51:5 AM
The other day we were discussing at work how acceptance testing is getting closer and closer to development rather than being something that QA does after the developers are "done" writing the code (and hopefully their own unit tests). This is a requirement of today's software development modern practices such as Continuous Deployment and Devops.
Some people have even claimed that this is a step towards a world where testers do not exist! All that's left is testing coaches and developers that can integrate all levels of testing into development.
But forgetting these more speculative, IT-world-changing claims, everyone agrees that testing is being done very early in the development process in most organisations. That means that tools such as SoapUI, an API-testing tool by SmartBear, my employer, need ways to get integrated into the application's build so that it can be part of the Continuous Integration process. SoapUI does have a Maven plugin which allows people to integrate their SoapUI tests with their Maven builds. But that's pretty much all SoapUI has to offer in this respect. In a world where Maven no longer has the dominance it once had (at least in the Java world) and alternative build systems such as Gradle are rapidly gaining in popularity, I think that SoapUI users deserve better!
That's the first reason why I decided to spend a weekend trying to write a SoapUI Gradle plugin.
The other reason is that I have been using Gradle in my own hobby-projects to substitute Maven and so far I have been quite happy with it. One of the things I really wanted to try was to write a plugin for it, so to write a SoapUI plugin seemed like a really great idea.
The first thing I did was to have a look at the SoapUI Maven plugin (which is open-source). It seemed quite simple, so I felt more confident this was something I could actually go ahead with.
Then, I tried to find out if it was possible to convert a Maven pom.xml to a build.gradle (the build script used by Gradle to build a project). It turned out to be extremely easy!
After cloning the Maven plugin with git, all I had to do was to type the following in the root folder:
gradle setupBuild
# or for Gradle 1.1
gradle init
This gave me a basic Gradle build script reflecting the Maven pom.
Next, I had a look at the Gradle page which shows how to create a Gradle plugin. You can write a Gradle plugin in any JVM language you want, as long as it compiles to Java bytecode (that includes Java itself, of course, but also Groovy, Clojure and Scala, for example). Looking at the code for the Maven plugin, though, it was obvious that using a dynamic language would be really benefitial (most code was dedicated to transferring properties between objects, something easy-peasy to do in dynamic languages), so choosing Groovy was an easy choice.
To write a Gradle plugin, all you have to do (well, nearly all) is to create a class implementing Plugin<Project>. This is an interface with a single method: void apply(Project project).
From there, you just need to write the actual code and learn how to use the Gradle API, which is very well designed.
But before we start writing code, let's see what a plugin looks like in Gradle, as compared to Maven.
Here's the Maven plugin basic example found in the SoapUI documentation (in both examples, I am ommiting the declaration of the repository where Maven/Gradle will get the plugin from.. in the case of Gradle, the plugin version is defined in that declaration, see the real example here):
<plugins>
<plugin>
<groupId>com.smartbear.soapui</groupId>
<artifactId>soapui-maven-plugin</artifactId>
<version>4.6.1</version>
<configuration>
<projectFile>simple-test-soapui-project.xml</projectFile>
<projectProperties>
<value>message=Hello World!</value>
</projectProperties>
</configuration>
</plugin>
</plugins>
Being somewhat familiar with Gradle plugins, I knew that, for the example above, the script should look something like this:
apply plugin: 'soapui'
soapui {
projectFile = 'simple-test-soapui-project.xml'
projectProperties = [ message: 'Hello World!' ]
}
Much nicer, in my humble opinion. So that's what I set out to build.
This was the easy part. The Maven plugin contains a few different goals (eg. test, loadtest, mock), each implemented in a different Mojo. I decided to start with the most important one: test.
So the Maven plugin TestMojo is reflected by my own TestPlugin (notice that in Gradle, you talk about tasks rather than goals, so test will be a Gradle task).
The first thing to do is to create the test task. To avoid having to use qualified task references, I decided to call the test task soapuiTest. Here's one way to declare this task with Gradle:
project.task( 'soapuiTest' ) << {
// do something
}
Once this plugin is applied to the project (as shown in the gradle script above), the user will be able to call this task by running the following command:
gradle soapuiTest
You can easily test you've gotten your basic code right by using the Gradle testing API to write something like this (example uses the Spock framework for testing)
def "Project with soapui plugin applied gain the 'soapuiTest' task"( ) {
given:
def project = ProjectBuilder.builder().withName( 'my-proj' ).build()
when:
project.apply plugin: 'soapui'
then:
project.tasks.soapuiTest
}
This confirms that the soapuiTest task has been added to a test project after the soapui plugin was applied to it. Pretty neat.
To use the syntax shown in the build script we're trying to build (where we can call soapui { ... }, we need to create a project extension which will make soapui available in the build script. The following code does just that:
project.extensions.create( 'soapui', SoapUIConfig )
Here, SoapUIConfig is a Groovy class I created containing all the properties the user might set to configure SoapUI. This class contains exactly the same fields as the Maven plugin's TestMojo.
A small part of the SoapUIConfig class is shown (with comments removed for brevity) below:
class SoapUIConfig {
String projectFile
String testSuite
boolean printReport
....
}
Supposing only the above properties, we could already write a build script containing the following SoapUI config:
soapui {
projectFile = 'soapui-projects/simple-project.xml'
testSuite = 'Suite A'
printReport = true
}
Without any further work, inside the testSoapuiTask closure, we would get an instance of SoapUIConfig with all the above properties set.
As in the Maven plugin, the only mandatory property is projectFile. So the minimum configuration requires the user to specify only this property.
Not coincidentally, all of the properties in SoapUIConfig are also settable properties of SoapUITestCaseRunner, which is what actually runs the SoapUI tests.
So the next step is to set all non-null properties of the SoapUIConfig instance we receive from Gradle into an instance of SoapUITestCaseRunner. I wrote a generic class which can do that easily using the power of Groovy:
// provider is an instance of SoapUIConfig
// destination is an instance of SoapUITestCaseRunner
def providerGetters = findMethods( provider.class, 'g' )
def destinationSetters = findMethods( destination.class, 's' )
providerGetters.each { getter ->
def matchingSetter = destinationSetters.find { setter ->
referToSameProperty( getter, setter ) }
if ( matchingSetter && provider."${getter}"() != null ) {
try {
destination."${matchingSetter}"( provider."${getter}"() )
} catch ( e ) {
println "Unable to use ${matchingSetter} due to ${e}"
}
}
}
Now we've got a SoapUI test case runner ready to go! Now, let's try to get it going.
With the Gradle plugin basically ready to go, I wrote a separate project (using Gradle, naturally) containing basically a SoapUI project file and a Gradle build script. When trying to run it, however, I found out that SoapUI, being a mature product which supports an incredible number of different technologies, depends on half of the Maven repository libraries (just joking, but it does depend on more than 100 different libraries, not counting duplicates, of course).
So, building the Gradle plugin, which depends on SoapUI and all of its dependencies, can be entertaining if you like watching Gradle download jars and metadata for some 20 minutes!
If that had been the only problem, I would have been, really, quite glad with everything... it took just a few hours to learn how to write a Gradle plugin and acually write it based on an existing Maven plugin. If only I knew how hard it would be to sort all the conflicting SoapUI dependencies just to be able to get anything running at all, I would probably have given up before even granting it much thought. But having built nearly the whole thing, of course I could not give up now, before seeing something actually running, and getting a test result from SoapUI!
First of all, two dependencies had broken metadata which Maven seems to be happy ignoring, but which Gradle does not permit!
Those were jtidy (SoapUI depends on version r872-jdk15, but the metadata for this version declares its version as only r872 - God knows why Maven does not complain about that) and cajo (the groupId is gnu.cajo in the SoapUI's pom, but in the metadata, the groupId is just cajo).
I could have fixed those in my local repository, but then the build would not be portable... so I decided to simply ignore these dependencies for now (which can be a problem if the code which relies on these libraries is ever executed):
compile( group: 'com.smartbear.soapui', name: 'soapui', version: '4.6.1' ) {
....
exclude( module: 'jtidy' )
exclude( module: 'cajo' )
}
The next issue: the commons-codec:1.4 module (SoapUI depends on version 1.3, but 1.4 is a transitive dependency, unfortunately) in my local repo did not contain a jar. I deleted it and got Maven to re-download the module, but it still did not fetch the jar! Looking at Maven Central, I found the jar and downloaded it manually, after which Gradle stopped complaining about that. Why, why???????
Finally, the project compiled and actually started running. But, as expected by now, there was a problem with a class having been compiled with a different, incompatible version of an external library (log4j, as it turned out) which gave this Error:
java.lang.IncompatibleClassChangeError: Implementing class
Following the stack-trace lead me to AbstractSoapUIRunner.java:74, which is where an instance of log4j's ConsoleAppender is created.
I decided to take more drastic measures by actually studying the SoapUI's dependency tree (which can be created with mvn dependency:tree). Unfortunately, this does not show duplicate dependencies.
However, I found out that Gradle allows users to take control of the dependency resolution mechanism and choose what to do when conflicting dependencies exist!
So, I decided to forbid the existence of conflicting dependencies (Gradle by default chooses the most recent version of any library present in the class path more than once) and pick the ones I wanted to use manually. I ended up with this being added to my build script:
configurations.all {
resolutionStrategy {
failOnVersionConflict()
force 'log4j:log4j:1.2.12',
'commons-codec:commons-codec:1.4',
'commons-io:commons-io:1.4',
'org.apache.httpcomponents:httpcore:4.1.1',
'commons-logging:commons-logging:1.1.1',
'commons-lang:commons-lang:2.4',
'xalan:xalan:2.7.1',
'xml-apis:xml-apis:1.3.04',
'junit:junit:4.10',
'xom:xom:1.1',
'org.apache.santuario:xmlsec:1.4.5'
}
}
This means that every library listed above is referred to more than once in SoapUI's (including transitive) dependencies! Wow, not easy to manage this!
Finally, I was able to compile with certainty about each version of which library was being used by Gradle. I even found out how to print all of the runtime dependencies of a project! Just add this to your build script:
task( deps ) << {
project.configurations.runtime.collect { it.name }.sort().each {
println it
}
}
and run with
gradle deps
To my astonishment, the error java.lang.IncompatibleClassChangeError still did not go away! :(
The only solution I found was to actually comment out the offending code in the SoapUI code, recompile the whole thing, and then run it again!
Finally, I got the plugin working!
WATCH OUT FOR THE CLASSPATH HELL!
It is a real problem in larger projects in the Java language (use OSGi if you can).
On a positive note, I loved Gradle's capabilities, flexibility and ease-of-use (even if sometimes it's a little difficult to find the right information).
This plugin is not ready to be used just yet, but I hope to be able to put some more time to make it work very soon, so please keep an eye on the project's GitHub page!