Repeatability has quite a few definitions, however, in the context of developer testing it essentially means to do, make, or perform again. By their nature, xUnit frameworks facilitate repeatability through fixtures; however, what tends to be underestimated is fixture complexity when it comes to higher level testing.

For example, the following system test is repeatable as far as JUnit and JWebUnit are concerned, but this test isn’t technically repeatable.

import net.sourceforge.jwebunit.WebTester;
import junit.framework.TestCase;

public class WidgetCreationTest extends TestCase {
 private WebTester tester;

 protected void setUp() throws Exception {
  this.tester = new WebTester();
  this.tester.getTestContext().
   setBaseUrl("http://localhost:8080/acme/");
 }

 public void testWidgetCreation() {
  this.tester.beginAt("/CreateWidget.jsp");
  this.tester.setFormElement("widget-id", "55589");
  this.tester.setFormElement("part-num", "rt676");

  this.tester.submit();
  this.tester.assertTextPresent("55589");
  this.tester.assertTextPresent("successfully created.");
 }
}  

A logical analysis of this test case reveals a subtle issue: a widget is created. It’s apparent that the widget is unique too. What do you think will happen if this test is run twice? Hopefully, the second run will fail due to integrity constraints in the underlying database.

To achieve logical repeatability, additional steps need to be taken. Two that come to mind are:

• Create an additional test case which deletes the test case after the widget’s creation
• Coerce the architecture into repeatability through back-door access frameworks like DbUnit

By employing DbUnit to place the database into a known state before and after the test case, I can logically assume that there are no widgets in the database with the same id before I run the test case and that my newly created widget won’t be there the next time I run this test case.

Taking the code from above, I can now add DbUnit logic via delegation and thus facilitate database management within the test case below.

package test.come.acme.widget.Web;

import java.io.File;
import java.io.IOException;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;

import org.dbunit.database.DatabaseConnection;
import org.dbunit.database.IDatabaseConnection;
import org.dbunit.dataset.DataSetException;
import org.dbunit.dataset.IDataSet;
import org.dbunit.dataset.xml.FlatXmlDataSet;
import org.dbunit.operation.DatabaseOperation;

import net.sourceforge.jwebunit.WebTester;
import junit.framework.TestCase;

public class RepeatableWidgetCreationTest extends TestCase {
 private WebTester tester;

 protected void setUp() throws Exception {
  this.handleSetUpOperation();
  this.tester = new WebTester();
  this.tester.getTestContext().
   setBaseUrl("http://localhost:8080/widget/");
 }

 public void testWidgetCreation() {
  this.tester.beginAt("/CreateWidget.jsp");
  this.tester.setFormElement("widget-id", "55589");
  this.tester.setFormElement("part-num", "rt676");

  this.tester.submit();
  this.tester.assertTextPresent("55589");
  this.tester.assertTextPresent("successfully created.");
 }

 private void handleSetUpOperation() throws Exception{
  final IDatabaseConnection conn = this.getConnection();
  final IDataSet data = this.getDataSet();
  try{
   DatabaseOperation.CLEAN_INSERT.execute(conn, data);
  }finally{
   conn.close();
  }
 }

 private IDataSet getDataSet() throws IOException, DataSetException {
  return new FlatXmlDataSet(new File("test/conf/seed.xml"));
 }

 private IDatabaseConnection getConnection() throws
   ClassNotFoundException, SQLException {
    Class.forName("org.hsqldb.jdbcDriver");
    final Connection jdbcConnection =
     DriverManager.getConnection("jdbc:hsqldb:hsql://127.0.0.1",
	  "sa", "");
    return new DatabaseConnection(jdbcConnection);
 }
}

Now this test case can be run twice in a row without any concern that the underlying architecture will cause the second run to fail.

Interestingly enough, however, this test case still isn’t 100% repeatable– there is one other major assumption this test case makes– do you know what it is?

Once complexity metrics, like Cyclomatic complexity, have flagged a method, three proactive steps can be taken to reduce the risk of defects:

1. Write tests for that method
2. Refactor the method into smaller, more isolatable methods
3. Write tests for those newly created methods

This flash demo walks you through the steps of applying the Extract Method refactoring pattern within Eclipse and writing a corresponding test case utilizing the JUnit-addons framework.

Watch the demo.

I’ll be speaking about Unit Testing Best Practices at the Southeast Virginia Java Users Group on September 19th– if you are in the Virginia Beach area come on out!

I’ll also be giving a modified version of this presentation at SD Best Practices conference in Boston on September 13. This will be a hands on presentation, which should provide some good entertainment.

Annotations have given JUnit 4 a whole new look, but is it flexible enough to compete with TestNG?

Andrew Glover recently published his eighth article, “JUnit 4 vs. TestNG,” in the IBM developerWorks’ series “In Pursuit of Code Quality.” This month he compares JUnit 4 with the features of TestNG, considers what’s unique about each framework and reveals three high-level testing features you’ll only find in TestNG.