Using groups with TestNG

This post demonstrates a simple example of using groups with TestNG.

For this exercise I’m using a very simple file layout. The jar file was copied directly from the TestNG package. SampleTest.java and testng.xml are my test and configuration files, respectively.

[23:31 20090914 crashing@desktop ~/simpletestng]
$ ls
SampleTest.java testng-5.10-jdk15.jar testng.xml

SampleTest.java has two test methods, annotated with @Test, and with an assigned group name.

import org.testng.annotations.*;
public class SampleTest {
@Test(groups = { “groupA” })
public void methodA() {
System.out.println(“groupA”);
assert true;
}

@Test(groups = { “groupB” })
public void methodB() {
System.out.println(“groupB”);
assert true;
}
}

testng.xml defines these two groups as belonging to a test set.

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd"&gt;
<suite thread-count="5" verbose="1" name="TrialTest" annotations="JDK">

<test name="TestSet" enabled="true">
<groups>
<run>
<include name="groupA"/>
<include name="groupB"/>
</run>
</groups>

<classes>
<class name="SampleTest"/>
</classes>
</test>

</suite>

I compile the SampleTest class,

$ javac -classpath testng-5.10-jdk15.jar SampleTest.java

and then run all the test methods as configured in testng.xml

$ java -classpath .:testng-5.10-jdk15.jar org.testng.TestNG testng.xml

[Parser] Running:
/Users/crashing/simpletestng/testng.xml

groupA
groupB

===============================================
TrialTest
Total tests run: 2, Failures: 0, Skips: 0
===============================================

Both test methods ran.

Alternatively I can specify specific groups to run. For this I use the -groups and -testclass option. I do not use the testng.xml file.

java -classpath .:testng-5.10-jdk15.jar org.testng.TestNG -groups groupB -testclass SampleTest

[Parser] Running:
Command line suite

groupB

===============================================
Command line suite
Total tests run: 1, Failures: 0, Skips: 0
===============================================

Specifying both the testng.xml and a -groups option will cause both configurations to be processed.

[22:58 20090914 crashing@desktop ~/simpletestng]
$ java -classpath .:testng-5.10-jdk15.jar org.testng.TestNG testng.xml -groups groupA -testclass SampleTest
[Parser] Running:
/Users/crashing/simpletestng/testng.xml
Command line suite

groupA
groupB

===============================================
TrialTest
Total tests run: 2, Failures: 0, Skips: 0
===============================================

groupA

===============================================
Command line suite
Total tests run: 1, Failures: 0, Skips: 0
===============================================

This is probably not what you want. So, use either the testng.xml or the -groups and -testclass option, not both.

Cesium for automated YSlow reporting

Hot on the heels of yesterday’s posting about my brief experience with YSlow in a Selenium framework, comes the release of Cesium 0.1, a tool specifically designed for automating YSlow runs.

I don’t care enough about YSlow tests to setup and maintain another test harness just for that purpose, so I don’t think I’ll be trying it myself. For others who get in to this sort of thing, here’s something to get in to.

Loving Hudson

Hudson rocks. I mean, holy shit, this is one nice software package. I’m just starting my first foray into continuous integration (CI) and haven’t compared Hudson to other CI software but so far Hudson is so feature rich and easy to use that I don’t see a need to evaluate others.

Hudson has a clean user administration interface but happily obliges me if I want to manually tinker with the XML-format configuration files – something I had to do several times after repeatedly locking myself out trying to get the security settings correct. Editing the configuration files also comes in handy for making bulk changes across a dozen of job settings – for example, adding Jabber notification or renaming projects.

The formal documentation is a little on the light side but the GUI interface and layout of the configuration files is intuitive enough that I usually didn’t need to RTFM. The active support forum helped fill in many knowledge gaps and a little experimentation on my own has so far solved the rest.

Overall, I’m really amazed and thrilled how seamlessly it works with our existing build and testing system which consists of hobbled together shell scripts wrapping other Perl scripts and Ant tasks. I’ll be adding Selenium tests for our web components in the future.

I’ve hit a few, small stumbling blocks – nothing insurmountable – incorporating Hudson into my project and I’ll be posting about some of those. Still, I’m very excited about the prospects of having Hudson help me solve some long standing frustrations on this project, even those that aren’t really a component of continuous integration.