I was just perusing a conference catalog for a major conference and it all of a sudden dawned on me that a majority of conference sessions were about creating this application with that featured technology. What’s more, this phenomena isn’t unique to this conference– just take a look at any development portal and look at the article titles– they’ll most likely reflect the same pattern. In fact, this pattern of touting the latest and greatest framework or technology for building software applications isn’t new. We’ve been chasing technology silver bullets since the days of mainframes.

Don’t get me wrong– I love learning about the newest framework out there and trying to ascertain if I’ll have an opportunity to utilize it at some point, but why is it that we as an industry appear to focus the vast majority of our attentions on creating applications when we actually spend a far greater time maintaining software applications?

It turns out, as Robert Glass wrote in “Facts and Fallacies of Software Engineering” that:

Maintenance typically consumes 40 to 80 percent (average, 60 percent) of software costs. Therefore, it is probably the most important life cycle phase of software.

Therefore, wouldn’t it seem that we’d see a lot more conference presentations on how to make that large maintenance cost as low as possible? Or when someone builds the next greatest framework for building these, that, and those types of applications, why is it that they tout how fast you’ll be able to do it. I, for one, would love for someone to tell me how easy it is to maintain that particular tool or framework, because ultimately, that’s where I’ll end up spending the most money.

If the lion’s share of money is spent after the initial application is built, it seems to me that the criteria for success then isn’t how quickly a high quality application was delivered to a customer. A more logical standard for success, it seems, would be that an application was indeed delivered quickly, but that it also is relative inexpensive to enhance and therefore will prove to be an economically friendly long-lived application.

Instead, it appears that the vast majority of us don’t want to admit that software is aeternum and thus go about learning how to build interesting applications with equally interesting new frameworks and technologies. I’ll be doing the same, but with the added realization that many of those interesting applications will indeed be built and will undoubtedly be spending the majority of development costs in maintenance.

If you are planning on attending JavaOne, you’ll undoubtedly want to:

• Catch up with old friends
• Learn something new
• Take home a testearly.com ruler

Yes, that’s right ladies and gentlemen– testearly.com will be giving away, what will most likely be, the most useful trinket of 2007.

These limited edition devices that enable you to measure lengths (not to exceed 12 inches) are fashioned in wooden form and are enhanced with, what will unquestionably be, the seminal phrase of 2007: “Friends Don’t Let Friends Ignore Code Metrics.”

Keep your eyes open as the testearly.com rulers are sure to go quickly.

developerWorks is in the process of creating a community of spaces (think MySpace) each with a focused theme. Stay tuned to the Accelerate development space, which is functioning as a preview at this point; nevertheless, this space promises to be a one stop portal for all things related to developer testing, continuous integration, code metrics, and refactoring to name a few.

One of the nice aspects of Google’s Web Tool kit is its built-in developer testing functionality à la JUnit. Not only does GWT give you the ability to develop Ajax components within Java (as opposed to JavaScript) but by standardizing the development platform on Java, they’ve facilitated the use of testing Ajax related code with what’s arguably the standard testing platform for Java developers. One key thing to keep in mind, however, is that Google’s GWTTestCase isn’t meant to test UI related code– it’s meant to facilitate testing asynchronous aspects that can be triggered by UI interactions.

For example, the image to the right demonstrates an Ajax application that retrieves definitions for a given word. It is a rather simple problem domain– once a word is entered into the web from and a user presses the submit button, an asynchronous call is made to retrieve the word’s definition from a server side database.

Once the definition is retrieved from the database, it is inserted below the form as follows:

Because the brunt of the work related to this simple web flow is essentially the heart of Ajax, the meat of the code is executed when the submit button is clicked. Consequently, one must approach writing this particular code in such a manner so as to facilitate testing it programmatically via JUnit. Sure, you could indirectly test the code via functionally executing the flow either manually or in an automated fashion with Selenium, but by using JUnit, you can essentially test the server side communication work more easily.

For example, the submit button is configured with a ClickListener that essentially defers logic to a UI specific method.

btn.addClickListener(new ClickListener() {
 public void onClick(Widget sender) {
  submitWord();
 }
});

The submitWord method is UI specific because it uses UI components.

protected void submitWord() {
 String word = this.getTextBox().getText().trim();
 this.getTextBox().setEnabled(false);
 this.getDefinition(word);
}

Note how the submitWord method then calls the getDefinition method, which takes a String. This method tries not to be UI specific (as much as possible) and accordingly can be tested effectively with JUnit.

protected void getDefinition(String word) {
 final Label output = this.getOutputLabel();
 WordServiceAsync instance = WordService.Util.getInstance();
 instance.getDefinition(word, new AsyncCallback() {
   public void onFailure(Throwable error) {
    Window.alert("Error occured:" + error.toString());
   }
   public void onSuccess(Object retValue) {
    output.setText(retValue.toString());
   }
  });
 this.getTextBox().setEnabled(true);
}

Note, this method creates an anonymous class of type AsyncCallback, which limits how much you can do inside of it; hence, the definition is actually set as a Label’s text. This means that in order to validate things work, one must grab an instance of the Label and assert that its text is the correct definition.

Finally, because this code is essentially Ajax, which has an ‘A’ for asynchronous, you must force JUnit to wait using a Timer object, which is provided in the GWT.

public void testDefinitionValue() throws Exception {
 final WordModule module = new WordModule();
 module.getDefinition("pugnacious");
 Timer timer = new Timer() {
   public void run() {
    String value = module.getOutputLabel().getText();
    assertEquals("should be...", "...", value);
    finishTest();
   }
 };
 timer.schedule(200);
 delayTestFinish(500);
}

Note the call within the Timer to obtain the definition from the Label (getOutputLabel().getText()).

As you can see, GWT facilitates testing of server side interaction code quite nicely– that’s a huge feature if you plan on building Ajax enabled web applications. Plus, UI related code can (and should) be tested by Selenium as this is what Selenium was designed for.

For all its benefits, Continuous Integration (CI) does have its share of flaws. For one, CI is somewhat reactionary when integration builds break because the code has already been committed to the mainline of the version control system.

Broken Builds
The main problem of reactionary CI is that you don’t learn that your full integration build has failed until the code is committed to the version control system and other developers are affected by it, which can decrease team velocity.

Current workarounds
In Software Configuration Management Patterns, Berczuk and Appleton describe the Private System Build as a pattern for preventing a broken build from occurring on the integration build machine. The private system build includes integrating changes from other developers on your project and running a successful integration build before you commit changes to your version control system. However, there are certain errors such as those that may be configuration-related or you may have simply forgotten to commit new files to the repository. Another workaround to preventing broken builds is performing Manual Integration Builds as James Shore covers in his blog. In this workaround, developers are put into a queue where only one person (or pair) integrates at a time. This was the original concept of CI with XP teams. This technique should be coupled with the private system build. While this approach can take some discipline and can be a little more difficult for distributed teams, it can prevent builds from being broken too long. However, you still have the same problem of having broken code (compile, test or otherwise) in the repository — even if it isn’t as long as an asynchronous build when using a CI server.

Possible Solutions
There is a new breed of tools entering the marketplace that hold some promise. While they probably don’t yet solve all the “problems” of CI, they are addressing some of CI’s flaws. There are servers that provide CI capabilities and can prevent “bad” code from entering the version control system. Borland’s Gauntlet (full disclosure: my company is a partner of Borland) and JetBrains’ TeamCity. Gauntlet uses a feature called “sandboxing” which creates a temporary repository branch for each integration build and promotes the code to the mainline of the repository only if the integration build is successful (including compilation, tests, inspections and so on).

What’s Next?
Don’t get me wrong, I use and evangelize CI regularly (even wrote a book on it) and I think its practices provide significant benefits…especially when compared to “big-bang” integration when teams wait until the end of a significant milestone (e.g. the day before they were supposed to release) to integrate all of their changes. However, like anything, things can be improved and we are continuing to see improvements in the evolution of the practice and I expect to see more in the future. As I eluded, there are other issues with CI and I will cover these in another blog entry…

In his seminal article on Continuous Integration, Martin Fowler covers the practice of “staged builds” when discussing fast builds. He addresses what he calls “commit builds” and “secondary builds”. I like to think of them as lightweight builds and heavyweight builds. In any case, the purpose in having two different types of builds run from your Continuous Integration server is so that your primary (commit) build is super fast to provide rapid feedback. In particular, some teams will not move on to their next task until their integration build is successful, so if they must wait more than say 10 minutes or so (impatient, aren’t we ), they’ll get discouraged with the build process “taking too long”.

In Extreme Programming Explained, Second Edition, Kent Beck recommends that (integration) builds take no longer than 10 minutes to execute. This can be difficult if you are running your compilation, unit tests, component tests (that rely on a database), system tests, and functional or acceptance tests, a host of automated inspections and web deployments with every build. By segmenting your integration builds, you can use your commit build to run your compilation, execute unit tests (that don’t have heavy external dependencies) and perform a deployment. Then, your secondary build can run your component and system tests that have heavier dependencies along with inspections tools such as code coverage, coding standard checks, and so on. Typically, you want to use your commit build success as an indicator to move on to your next development task, while your secondary build is running.

Let’s look at an example in CruiseControl. I’ll use my brewery example. I create two CruiseControl projects in the config.xml. The first is named brewery-commit and the second is named brewery-secondary. The brewery-commit project uses the modification set to poll the mainline of my Subversion repository like so:

<project name="brewery-commit">
...
<modificationset quietperiod="120">
<svn RepositoryLocation="http://build.integratebutton.com/svn-repo/main/brewery/trunk"
username="patrick.henry"
password="L!berty"/>
</modificationset>
...

Note how the commit build is polling the repository for the latest commit to Subversion. Now, let’s see the secondary build:

<project name="brewery-secondary">
...
<modificationset quietperiod="120">
<buildstatus logdir="logs/brewery-commit" />
</modificationset>
...

Instead of monitoring the Subversion repository, it looks for a successful build in the brewery-commit CruiseControl project using the buildstatus element. Essentially, you are establishing a project dependency. The other trick is that you must call a different delegating build. For instance, the brewery-secondary project will call a delegating build called build-brewery-secondary.xml which calls the same project build file, but a different target. This target is responsible for executing the longer-running tests and the inspections. Make sense?

Code coverage statistics are often heavily relied upon in an effort to gauge testing efforts and infer relative code quality. But, as I’ve written about before , code coverage reports can often fool you. As such, I highly recommend you use these reports not as a means for judging what’s tested but in judging what’s not tested.

It turns out that code conditionals are a bit tricky when it comes to measuring code coverage and in certain scenarios, tests can trigger high line and branch coverage rates but fail to uncover nefarious defects. In fact, I recently posted a code example that demonstrates short-circuiting code coverage via a simple OR condition .

In my posting’s code example, the second clause in an if conditional throws an exception; however, because one can easily write a test that only logically executes the first clause of the conditional, the second part of the conditional is skipped, yet a coverage tool may end up reporting 100% branch coverage as it did for me. The question then arises– how can one infer the quality of tests if code coverage reports are often liberal in their calculations?

One highly effective technique that yields much more effective testing (and therefore more precise coverage) is the notion of path testing. By measuring paths through a method (which, in essence, is the Cyclomatic complexity value of the method) one can quickly gauge how many tests one would need to achieve adequate coverage.

For example, running the Eclipse Metrics plug-in against this method yields a CC of 3. This means there are 3 logical paths through this method and accordingly, I’d need to write at least 3 tests to accurately test the branchIt method.

The branchIt method isn’t terribly complex, but look closely at its logic and see if you can spot the 3 paths.

public void branchIt(int value){
 if((value > 100) || (HiddenObject.doWork() == 0)){
  this.dontDoIt();
 }else{
  this.doIt();
 }
}  

It seems the 3 paths are divided as follows:

• The first conditional has two clauses, hence there’s 2 paths
• The later else conditional

The good news is that you’d find the defect rather quickly in this case– the mere act of trying to trigger the else clause will almost force the defect to show up, won’t it? What’s also interesting is that if you look at the Cobertura report for this code, you’ll notice that it correctly notes that the else conditional wasn’t covered in my first test. If I were trying to achieve 100% line coverage, I’d quickly uncover the defect lurking in the first clause by trying to force the else.

Robert Keith of DigitalTester.com has written a great entry on some techniques for testing multithreaded applications with JUnit using Java’s java.util.concurrent package. Check out his entry entitled “Homegrown Multithreaded JUnit Tests” for an informative read.

One of the most handy features of JUnit 4 is its ability to run parameterized tests, which essentially means you can create a generic test and run it multiple times with various test parameters. In previous versions of JUnit, if you wanted to simulate this same behavior, you’d have to either:

• loop over a collection of values
  • which means if there was a failure, the loop wouldn’t terminate
• write unique test cases for each test data combination
  • which could prove to be a lot of coding

With parameterized tests, you can create a highly flexible testing scenario in five easy steps:

1. Create a generic test and decorate it with the @Test annotation
2. Create a static feeder method that returns a Collection type and decorate it with the @Parameters annotation
3. Create class members for the parameter types required in the generic method defined in Step 1
4. Create a constructor that takes these parameter types and correspondingly links them to the class members defined in Step 3
5. Specify the test case be run with the Parameterized class via the @RunWith annotation

For example, the following test defines a generic test dubbed verifyGoodZipCode that tests four different combinations of zip codes defined in the feeder method named regExValues. Note too, the use of the @Parameters annotation and the @RunWith(Parameterized.class) class level decoration.

@RunWith(Parameterized.class)
public class ParametricRegularExpressionTest {

 private String phrase;
 private boolean match;

 private static String zipRegEx = "^\d{5}([\-]\d{4})?$";
 private static Pattern pattern;

 public ParametricRegularExpressionTest(String phrase, boolean match) {
  super();
  this.phrase = phrase;
  this.match = match;
 }

 @Parameters
 public static Collection regExValues() {
  return Arrays.asList(new Object[][] {
   {"22101", true },
   {"221x1", false },
   {"22101-5150", true },
   {"221015150", false }});
 }

 @Test
 public void verifyGoodZipCode() throws Exception{
  Matcher mtcher = this.pattern.matcher(phrase);
  boolean isValid = mtcher.matches();
  assertEquals("Pattern did not validate zip code", isValid, match);
 }

 @BeforeClass
 public static void setUpBeforeClass() throws Exception {
  pattern = Pattern.compile(zipRegEx);
 }
}

The astute reader will notice a subtle pattern relating to parameterized classes in JUnit 4– because of the member variable mapping with constructors, it becomes obvious that JUnit creates a new instance of the enclosing class for each test run. So in the example above, four instances will be created and the test method will be run four times with unique values relating to the Collection defined in the feeder method.

What this essentially means is that if you put a non-parametric test case in a parameterized class, it will also be run as many times as JUnit creates a new instance of your parameterized class. This isn’t a terrible thing, but it isn’t exactly elegant either. Hence, if you create parameterized test classes with JUnit 4, it probably makes sense to keep non-parameterized tests out of them. Simply put those methods in another test class.

Oh yeah, by the way, TestNG’s parameterized test classes don’t display this behavior. A non-parameterized test in a parameterized class will only be executed once. Nevertheless, JUnit 4’s parameterized test classes are an excellent way to achieve a high degree of code confidence with fewer test methods.