Automation and more: March 2017

Monday 27 March 2017

Creating a simple Maven Project

When I was first introduced to maven, I was introduced as a build tool, an alternative to ant build which I was using for my SoapUI project builds. However, once I dwell deeper into maven and built my interest because of the wide range of features supported by Maven, only then I realized that maven is much more than just a build tool. It is a complete project management tool which allows the developers to have synergy in the project structure. It provides a defined structure to the project which allows developers to easily understand and help in case there is any transition on the project. The new developers who are familiar with maven structure can easily determine the class file, resources etc on the project.

In this post I am going to start with creating a simple maven project and then in later part I will explain how you can create a complex project in maven with multiple modules.

You can create a simple project with below command (here I am assuming that you already have configured JDK and Maven).

mvn archetype:generate -DgroupId=myProject.myCompany.com

   -DartifactId=myProject

     -DarchetypeArtifactId=maven-archetype-quickstart

   -DinteractiveMode=false

the above command will create a maven project structure where you will have

pom.xml : This is the main and single configuration file that contains majority of the information used by maven to build the project.

Directory structure as :

src -

main -

java -

myProject -

myCompany -

com -

App.java : It is the class file

test -

java -

myProject -

myCompany -

com -

App.java : It is the JUnit test class file for testing the class created by maven automatically.

To run this maven project file you need to run a simple command as below :

mvn install

Above command will compile your source code, perform test and then create a jar package in the directory as below :

target

myProject-1.0-SNAPSHOT.jar

classes

myProject

myCompany

App.class

maven-archiver

pom.properties

maven-status

maven-compiler-plugin

compile

default-compile

createdFiles.lst
inputFiles.lst

testCompile

default-testCompile

createdFiles.lst
inputFiles.lst

surefire-reports

myProject.myCompany.com.AppTest.txt
TEST-myProject.myCompany.com.AppTest.xml

test-classes

myProject

myCompany

AppTest.class

As you can see that maven has compiled and packaged the project into a jar file (if it is web project then a war will be generated). You can run your package with following command from your project directory

java -cp target/myProject-1.0-SNAPSHOT.jar myProject.myCompany.com.App
Hello world!

So, as you can see in the above example that it is very easy to create a maven project. Now we look into the pom.xml file that has been generated by maven

Following element in the pom file together are known as co-ordinates of the projct

groupId
artifactId
packaging
version

Creating a complex project with multiple child elements or modules is even easier. Use the same command as above to create the maven master project. Now go to the project directory and remove the "src" directory created inside and edit the "pom.xml" and change the value of element "packaging" to "pom"

<packaging>pom</packaging>

Now your master project is almost ready and you can create the modules inside using the similar command as you used for creating the master project file.

mvn archetype:generate -DgroupId=myProject.myCompany.com

   -DartifactId=module1

     -DarchetypeArtifactId=maven-archetype-quickstart

   -DinteractiveMode=false

mvn archetype:generate -DgroupId=myProject.myCompany.com

   -DartifactId=module2

     -DarchetypeArtifactId=maven-archetype-quickstart

   -DinteractiveMode=false

The above two commands will create two modules or child projects within the master project and you will realize that the pom file inside the master project is updated to have below elements

And the child project will have reference to the master project as below

For a complex project you only need to compile the master project and child projects will be launched on their own.

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] myProject .......................................... SUCCESS [ 0.757 s]
[INFO] module1 ............................................ SUCCESS [ 5.211 s]
[INFO] module2 ............................................ SUCCESS [ 0.748 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6.909 s
[INFO] Finished at: 2017-03-26T19:26:54+05:30
[INFO] Final Memory: 15M/37M
[INFO] -----------------------------------------------------------------------

Thursday 23 March 2017

Maven : Compilation of Groovy Class files

There are various ways to build and use your groovy class files, you can use Gradle, or Goovyc Ant or using maven. In maven we have two main possibilities, one with GMaven and the other with Groovy eclipse.

I will elaborate an example with GMaven. GMaven Plugin definition requires the groovy dependency (groovy-all).

Below is an example of a pom file which compiles all the groovy class files located in a specific location.

<?xml version="1.0"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
 <modelversion>4.0.0</modelVersion>
 <parent>
  <groupid>app.myGroup.com</groupId>
  <artifactid>app.bu.master</artifactId>
  <version>1.0-SNAPSHOT</version>
 </parent>
 <groupid>app.myGroup.com</groupId>
 <artifactid>app-utils</artifactId>
 <version>1.0-SNAPSHOT</version>
 <name>app-utils</name>
 <url>http://maven.apache.org</url>
 <properties>
  <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
 </properties>
 <dependencies>
  <dependency>
   <groupid>org.codehaus.groovy</groupId>
   <artifactid>groovy-all</artifactId>
   <version>2.4.4</version>
   <scope>compile</scope>
  </dependency>

<dependency>
   <groupid>com.oracle</groupId>
   <artifactid>ojdbc7</artifactId>
   <version>12.1.0.2</version>
   <scope>compile</scope>
  </dependency>
  
  <dependency>
   <groupid>com.oracle</groupId>
   <artifactid>xmlparserv2</artifactId>
   <version>12.1.0.2</version>
  </dependency> 
 </dependencies>

<build>
  <plugins>
   <plugin>
    <groupid>org.codehaus.gmavenplus</groupId>
    <artifactid>gmavenplus-plugin</artifactId>
    <version>1.5</version>
    <executions>
     <execution>
      <id>app-utils</id>
      <phase>process-classes</phase>
      <goals>
       <goal>compile</goal>
      </goals>
     </execution>
    </executions>
    <configuration>
     <sources>
      <source>
       <directory>${basedir}/src/main/java/app/myGroup/com</directory>
       <includes>
        <include>**/*.groovy</include>
       </includes>
      </source>
     </sources>
    </configuration>
   </plugin>
  </plugins>
 </build>
</project>  <br>

In this example, my scripts are making connection to an oracle 12g database, hence I had to add dependency for ojdbc and since the DB is in 12G I was facing further errors related to xdb.war. So I downloaded the war file and installed it on maven using command below.

mvn install:install-file -Dfile=xdb6.jar -DgroupId=com.oracle -DartifactId=xmlparserv2 -Dversion=12.1.0.2 -Dpackaging=jar

After installing the war to my maven repository and added the same to dependencies as shown in example above.

Now I can use the above maven pom in my master file and other child pom build can use the classes compiled from it.

Dependency Error : SoapUI 5.2.1

In our project we observed a strange error because of which the maven build was failing. Further analysis led to a conclusion that the problem was specific to the version of Soap-UI plugin we were using in our project.

Here below is the error that we observed while executing the build

...
07:11:42 [INFO] Finished at: 2017-03-23T07:11:42+01:00
07:11:42 [INFO] Final Memory: 29M/71M
07:11:42 [INFO] ------------------------------------------------------------------------
07:11:42 [ERROR] Failed to execute goal com.smartbear.soapui:soapui-maven-plugin:5.2.1:test (myProject_v1) on project myProject_v1.0: Execution myProject_v1 of goal com.smartbear.soapui:soapui-maven-plugin:5.2.1:test failed: A required class was missing while executing com.smartbear.soapui:soapui-maven-plugin:5.2.1:test: com/jgoodies/forms/layout/CellConstraints
07:11:42 [ERROR] -----------------------------------------------------
07:11:42 [ERROR] realm =    plugin>com.smartbear.soapui:soapui-maven-plugin:5.2.1
07:11:42 [ERROR] strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy
07:11:42 [ERROR] urls[0] = file:/home/jenkins/.m2/repository/com/smartbear/soapui/soapui-maven-plugin/5.2.1/soapui-maven-plugin-5.2.1.jar
07:11:42 [ERROR] urls[1] = file:/home/jenkins/.m2/repository/com/oracle/ojdbc7/12.1.0.2/ojdbc7-12.1.0.2.jar
07:11:42 [ERROR] urls[2] = file:/home/jenkins/.m2/repository/org/codehaus/groovy/groovy-eclipse-compiler/2.9.1-01/groovy-eclipse-compiler-2.9.1-01.jar
07:11:42 [ERROR] urls[3] = file:/home/jenkins/.m2/repository/org/codehaus/plexus/plexus-compiler-api/2.2/plexus-compiler-api-2.2.jar
07:11:42 [ERROR] urls[4] = file:/home/jenkins/.m2/repository/org/codehaus/plexus/plexus-utils/3.0.8/plexus-utils-3.0.8.jar
07:11:42 [ERROR] urls[5] = file:/home/jenkins/.m2/repository/org/apache/xbean/xbean-reflect/3.4/xbean-reflect-3.4.jar
...

There are two possible ways to manage this error, one is to degrade the plugin version for Soap-UI. We saw that this error did not exist with Soap-UI plugin 4.6.? however with 5.?.? we have the problem.

Another way to fix the problem was to add "com.jgoodies:forms:1.0.7" dependency to your pom file.

Tuesday 7 March 2017

Maven : Integration of tests

In the changing environment, the way tests are executed are also changing. Once we moved to agile mode from a traditional waterfall or v-Cycle mode of development we saw the need of automation. However, there is always a next step to be done, and in this case the obvious next step is to integrate them with development builds. There could be various ways to achive the same among which the most common one is deploying your tests on Jenkins. It provides a common platform where the development builds are done and at the same time the test builds can be done. In this article we will explore the possibility to integrate the various automated tests.

Maven : Integration of SoapUI automated Tests

SoapUI includes a Maven plugin for running SoapUI tests and mocks during Maven build. To use the correct version of SoapUI, you will have to see under the Soap Maven Repository. Maven should know about the soap repository and dependencies related to your project and it can be achieved by updating the maven settings which would be available at "conf" directory under maven binary location.
Edit the settings.xml and update the below elements as needed.

Local Repository : Location of the directory where dependency repository would be stored locally for compilation
Mirror : Location of maven mirror repository.
Repository : maven repositories and other repository where dependent tools and configuration can be found

PluginRepository : maven repositories and other repository where dependency plugins can be found

Attached an example settings.xml which is being used for one of the project.

<settings xmlns="http://maven.apache.org/POM/4.0.0"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
                      http://maven.apache.org/xsd/settings-1.0.0.xsd">
                      
 <localrepository>C:\work\m2-repository</localRepository>
  
 <mirrors>
 
  <mirror>
   <id>myrepos_central</id>
   <mirrorof>central</mirrorOf>
   <name>Maven Proxy mirroring central (maven 2) repository</name>
   <url>http://maven2.myrepos.com/proxy/repository/public</url>
  </mirror>   
 </mirrors>

<repositories>
    <repository>
     <id>myRepos.central</id>
     <name>myReposCentral Repository</name>
     <url>http://maven.myRepos.com/m2-repository/</url>
     <snapshots><enabled>true</enabled></snapshots>
     <releases><enabled>true</enabled></releases>
    </repository> 
    <repository>
       <id>myRepos.repository</id>
       <name>myRepos Repository</name>
       <url>http://maven.myRepos.com/m2-repository/</url>
       <releases><enabled>true</enabled></releases>
       <snapshots><enabled>false</enabled></snapshots>
     </repository>
     <repository>
       <id>myRepos.snapshot.repository</id>
       <name>myReposSnapshot Repository</name>
       <url>http://maven.snapshot.myRepos.com/m2-snapshot-repository/</url>
       <releases><enabled>false</enabled></releases>
       <snapshots><enabled>true</enabled></snapshots>
     </repository>
     <repository>
     <id>myRepos.ext.non.free.repository</id>
     <name>myRepos Ext-Non-Free Repository</name>
     <url>http://maven2.myRepos.com/proxy/content/repositories/ext-non-free/</url>
     <releases><enabled>true</enabled></releases>
     <snapshots><enabled>false</enabled></snapshots>
    </repository>
    <repository>
     <id>EviwarePluginRepository</id>
     <name>myRepos Inhouse Releases</name>
     <url>http://maven2.myRepos.myCompany.com/proxy/content/repositories/eviwarePluginRepository/</url>
     <snapshots><enabled>true</enabled></snapshots>
     <releases><enabled>true</enabled></releases>
    </repository>
    <repository>
     <id>soap.ui.repository</id>
     <name>Soap UI</name>
     <url>http://www.soapui.org/repository/maven2/</url>
     <snapshots><enabled>true</enabled></snapshots>
     <releases><enabled>true</enabled></releases>
    </repository>

</repositories>
   <pluginrepositories>
    <pluginrepository>
     <id>myRepos.plugins</id>
     <name>myRepos Central Repository</name>
     <url>http://maven.myRepos.com/m2-repository</url>
     <snapshots><enabled>true</enabled></snapshots>
     <releases><enabled>true</enabled></releases>
    </pluginRepository>
    <pluginrepository>
      <id>myReposInhouseReleases</id>
      <name>myRepos Inhouse Releases</name>
      <url>http://maven2.myRepos.mycompany.com/proxy/content/repositories/inhouse/</url>
      <snapshots><enabled>false</enabled></snapshots>
      <releases><enabled>true</enabled></releases>
    </pluginRepository> 
    <pluginrepository>
     <id>soap.ui.repository</id>
     <name>Soap UI</name>
     <url>http://www.soapui.org/repository/maven2/</url>
     <snapshots><enabled>false</enabled></snapshots>
     <releases><enabled>true</enabled></releases>
    </pluginRepository>

</pluginRepositories>
  
</settings>

Once the settings are configured your maven can execute soapUI tests.
The second step is to create pom files for the execution of the SoapUI tests. You may have two scenario here, one where you only have one project file and the other where you have multiple project files. For single project file based execution it is easier as you have to create only one pom file which can be located at the same place as the SoapUI project file. For multiple project file however, we need to organize our project to be in the way below :

Soap-Project

Project-1

Soap-Project-file.xml (Soap UI project file)
pom.xml (Child pom file configured for this project)

Project-2

Soap-Project-file.xml (Soap UI project file)
pom.xml (Child pom file configured for this project)

Project-n

Soap-Project-file.xml (Soap UI project file)
pom.xml (Child pom file configured for this project)

pom.xml (master pom file)

Prefer to use names of the files or directories without spaces or special characters to avoid referencing issues.

The master pom file would look as below :

A typical child pom file would look as below :

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelversion>4.0.0</modelVersion>
    <groupid>myApp.myOrg.com</groupId>
    <artifactid>SoapUI_ProjectName</artifactId>
    <version>1.0-SNAPSHOT</version>
    <parent>
        <groupid>myApp.myOrg.com</groupId>
        <artifactid>master-pom</artifactId>
        <version>2.1.4-SNAPSHOT</version>
    </parent>
    <properties>
        <oracle.jdbc.driver.class.name>oracle.jdbc.OracleDriver</oracle.jdbc.driver.class.name>
        <nuodb.jdbc.driver.class.name>com.nuodb.jdbc.Driver</nuodb.jdbc.driver.class.name>
        <db.connection.url>jdbc:com.nuodb://192.168.0.101:48004/MYDATABASE?schema=SCHEMAU1</db.connection.url>
        <db.driver.class.name>${nuodb.jdbc.driver.class.name}</db.driver.class.name>
        <db.connection.username>USERNAME</db.connection.username>
        <db.connection.password>KARMAONEPL</db.connection.password>
        <application.url>http://xxx.xxx.xxx.xxx/applicationURL</application.url>
        <application.url>http://xxx.xxx.xxx.xxx/applicatioURL</application.url>
        
        <proxy.host>xxx.xxx.xxx.xxx</proxy.host>
        <proxy.port>proxy-port</proxy.port>
    </properties>
    <profiles>
        <profile>
            <id>profile-1</id>
            <properties>
                <application.url>${application.url}</application.url>
                <use.proxy>false</use.proxy>
                <proxy.host></proxy.host>
            </properties>
        </profile>
        <profile>
            <id>profile-2</id>
            <properties>
                <application.url>${application.url}</application.url>
                <use.proxy>false</use.proxy>
                <proxy.host></proxy.host>
            </properties>
        </profile>
    </profiles>
    
    <dependencies>
        <dependency>
            <groupid>com.oracle</groupId>
            <artifactid>ojdbc6</artifactId>
            <version>11.2.0.4.0</version>
        </dependency>
        <dependency>
            <groupid>com.nuodb.jdbc</groupId>
            <artifactid>nuodb-jdbc</artifactId>
            <version>2.3.1</version>
        </dependency>
         
        <dependency>
            <groupid>com.myOrg</groupId>
            <artifactid>linux</artifactId>
            <version>1.0</version>
            <scope>system</scope>
            <systempath>${project.basedir}/../dependencies/Linux.jar</systemPath>
        </dependency>
    </dependencies>
    <pluginrepositories>
        <pluginrepository>
            <id>smartbear-sweden-plugin-repository</id>
            <url>http://www.soapui.org/repository/maven2/</url>
        </pluginRepository>
    </pluginRepositories>
   <build>
        <plugins>
            <plugin>
                <groupid>com.smartbear.soapui</groupId>
                <artifactid>soapui-maven-plugin</artifactId>
                <version>5.2.1</version>
                <executions>
                    <execution>
                        <id>Project-1</id>
                        <phase>test</phase>
                        <goals>
                            <goal>test</goal>
                        </goals>
                        <configuration>
                            <projectfile>Project-1/Soap-Project-file.xml</projectFile>
                            <outputfolder>${project.basedir}/target-result</outputFolder>
                            <junitreport>true</junitReport>
                            <exportall>true</exportAll>
                            <testfailignore>true</testFailIgnore>
                            
                            <projectproperties>
                                <value>variable1=value1</value>
                                <value>variable2=value2</value>
                                <value>variablen=valuen</value>
                            </projectProperties>
                        </configuration>
                    </execution>
                </executions>
                <dependencies>
                    <dependency>
                        <groupid>com.oracle</groupId>
                        <artifactid>ojdbc6</artifactId>
                        <version>11.2.0.4.0</version>
                    </dependency>
                    <dependency>
                        <groupid>com.nuodb.jdbc</groupId>
                        <artifactid>nuodb-jdbc</artifactId>
                        <version>2.3.1</version>
                    </dependency>
                </dependencies>
            </plugin>
        </plugins>
    </build>
</project>

You can configure additional details for the project execution by further addition following parameters

projectFile : Specified the name of the SoapUI project file to use, default value is ${pom.artifactId}-soapui-project.xml
testSuite : Specifies the name of the TestSuite to run
testCase : Specifies the name of the TestCase to run
endpoint : Overrides the service endpoint to be invoked by any TestRequests
host : Overrides the target host:port to be invoked by any TestRequests
username : Overrides the username used by any TestRequests run
password : Overrides the password used by any TestRequests run
domain : Overrides the domain used by any TestRequests run
printReport : Controls if a small test report should be printed to the console (true/false)
outputFolder : Set which folder results/reports are saved to
junitReport : Turns on creation of JUnit-reports, (true/false)
exportAll : Controls if all test requests should be exported (default only exports errors), (true/false)
settingsFile : Specifies SoapUI settings file to use
wssPasswordType : Specifies WSS password type
project.password : Specifies password for encrypted project
settingsFile.password : Specifies password for encrypted settings file
globalProperties : Sets global properties
projectProperties : Sets project properties
saveAfterRun : Saves project file after run
testFailIgnore : Ignore failed tests.
reportFormat : Sets formats for created report. Pro only.
reportName : Sets which report to create. Pro only.
coverage : Sets the output to include Coverage HTML reports. Pro only.
environment : Sets the active environment. Pro only.

You will have to replicate the above configuration for all the child projects. Child project names in the pom, the directory name and the name mentioned under the master pom file should be same. Once the pom configuration is done you can test your execution with below command.

> mvn -U clean install

In case of integration of mock service you will have to do some minor modification in the phase and goal of the execution section. The execution should appear as below :

You can also define the following properties in order to customize your mock service on run time

projectFile : Specified the name of the SoapUI project file to use, default value is ${pom.artifactId}-soapui-project.xml
mockService : Specified the MockService to run
port : The local port to listen on, overrides the port configured for the MockService
path : The local path to listen on, overrides the path configured for the MockService
noBlock : Turns off blocking when MockRunner has started
settingsFile : Specifies SoapUI settings file to use, will contain path to integrated tool to run
project.password : Specifies password for encrypted project file
settingsFile.password : Specifies password for encrypted soapui settings file
saveAfterRun : Saves project file after run.

For more details following links can be referenced

https://www.soapui.org/test-automation/maven/maven-2-x.html

http://blog.karit.geek.nz/2009/08/using-soapui-to-do-testing-as-part-of.html

https://cduu.wordpress.com/2012/10/17/execute-specific-maven-goal-using-profiles/