So you want to make MSTest work on a .Net 2.0 projects and have CruiseControl.NET monitor everything for you. Well maybe “want” is a strong word. Maybe someone (your employer) told you to do so. It is possible, actually not even that hard. So here’s what I did.

My tools of choice here are MSBuild and MSTest. I agree Microsoft should distribute MSTest separately from the Visual Studios Team Edition (VSTS) for Testers. However Microsoft does not, so for now, the path of least resistance is to give Bill his pound of flesh and install the IDE on the build server. I actually installed the Visual Studios Team Suite on the build server that way I had all my favorite tools handy. The build server didn’t even whine.

I started off with an MSBuild file Build.xml that:

1. Compiles two solution files
2. Cleans out the TestResults directory
3. Runs the tests
4. Converts the code coverage to a XML file
5. Deploys the out put to a MSI file

The CCNet.config file:

1. Pulls source from SVN
2. Labels source
3. Runs a MSBuild task using the Build.xml from above
4. Merges the XML output from the MSBuild task
5. Emails the results to the appropriate people

The Dashboard displays:

1. The MSTest results
2. The Code Coverage results

Lets look at the build.xml first.

<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets ="Deploy" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" >
	<ItemGroup>
    <Project Include="$(MSBuildProjectDirectory)\csla20cs\cslacs.sln" />
    <Project Include="$(MSBuildProjectDirectory)\SourceMonitorSummary.sln" />
  </ItemGroup>
	<Target Name="BuildAll">
		<MSBuild Projects="@(Project)" Targets="Build"/>
	</Target>
  <ItemGroup>
    <TestFiles Include="$(MSBuildProjectDirectory)\**\*.*"/>
  </ItemGroup>
  <Target Name ='DeletetestResults' DependsOnTargets ='BuildAll'>
    <Delete Files="@TestFiles"  />
    <RemoveDir Directories="$(MSBuildProjectDirectory)\TestResults"  />
  </Target>
  <Target Name="RunTests" DependsOnTargets ="DeletetestResults">
    <RemoveDir Directories="$(MSBuildProjectDirectory)\TestResults" />
   <Exec Command='"C:\Program Files\Microsoft Visual Studio 8\Common7\IDE\MSTest.exe" /testmetadata:SourceMonitorSummary1.vsmdi /testlist:"BuildTests" /runconfig:localtestrun.testrunconfig' />
	</Target>
  <Target Name ='TestCoverageToXML' DependsOnTargets ='RunTests'>
    <Exec Command='"C:\Program Files\MSTestCoverageToXML\bin\MSTestCoverageToXML.exe" $(MSBuildProjectDirectory)' WorkingDirectory ='$(MSBuildProjectDirectory)\TestResults'/>
  </Target>
  <Target Name ='Deploy' DependsOnTargets ='TestCoverageToXML'>
    <Exec Command='"$(MSBuildProjectDirectory)\CreateSetup.bat"'/>
  </Target>
</Project>

I’ll break it down step by step. First I build up an item group of solutions to compile. I have two solutions to deal with, SourceMonitorSummary.sln which depends on cslacs.sln. Next I execute a MSBuild task named “BuildAll” to compile the solutions listed in @(Project). That will compile cslacs.sln first and then SourceMonitorSummary.sln.

Next I delete all the files and directories in the TestResults folder. That guarantees that when the tests run I will have one and only one folder inside TestResults directory. This is to support the code coverage report. John Cunningham has a wonderful blog The Visual Studio Team System “off-road” code coverage experience on converting the code coverage data from a test run into an XML file. I have a simple version of John’s code running that navigates the TestResult directory tree to find and move the data.coverage file to the out directory and then creates a Coverage.xml file that is later merged into the CruiseControl.net build log for the project.

Now for the good part. I execute the Exec Command task named “RunTests” that runs MSTest.exe from the command line. The key here is to setup the test meta data ahead of time. You need a test list, in my case I created BuildTests. Use the “Test/Create New Test List…” menu options in VSTE for Testers. When MSTest runs it will create a test results file in the TestResults directory. The file has a .trx extension. It is an XML file that can be merged directly into the build log.

You can also enable code coverage found under “Test/Edit Test Run Configurations/Local Test Run”. Pick the artifacts that you want code coverage against.

After that I execute the MSTestCoverageToXML.exe which converts the code coverage data to an XML file. If you have not enable code coverage for the test configuartion skip this task.

Finally I run the CreateSetup.bat for deployment. I’m using a Microsoft WIX file that gathers the output from the compile and packages it into an MSI. We have a repeatable build that can be executed from a command line “MSBuild Build.xml”.

Lets look at the CCNet.config file now, and see what is needed there.

<cruisecontrol>
  <project name="SourceMonitorSummary">
     <workingDirectory>C:\CruiseControl\WorkingDirectory\dev\SourceMonitorSummary</workingDirectory>
     <artifactDirectory>C:\CruiseControl\ArtifactDirectory\devSourceMonitorSummary</artifactDirectory>
     <category>Category 1</category>
     <webURL>http://localhost/ccnet/Projects/dev/SourceMonitorSummary</webURL>
     <modificationDelaySeconds>2</modificationDelaySeconds>
     <triggers>
        <intervalTrigger name="continuous" seconds="60" buildCondition="IfModificationExists"/>
     </triggers>
     <state type="state" directory="C:\CruiseControlState" />
     <sourcecontrol type="svn">
        <trunkUrl>svn://localhost/svn/dev/SourceMonitorSummary</trunkUrl>
        <workingDirectory>C:\CruiseControl\WorkingDirectory\dev\SourceMonitorSummary</workingDirectory>
     </sourcecontrol>
     <labeller type="defaultlabeller">
        <prefix>SourceMonitorSummary</prefix>
        <incrementOnFailure>true</incrementOnFailure>
     </labeller>
     <tasks>
        <msbuild>
           <executable>C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727MSBuild.exe</executable>
           <workingDirectory>C:\CruiseControlWorkingDirectory\dev\SourceMonitorSummary</workingDirectory>
           <projectFile>Build.xml</projectFile>
           <buildArgs>/noconsolelogger /p:Configuration=Debug /v:diag</buildArgs>
           <logger>ThoughtWorks.CruiseControl.MsBuild.XmlLogger,C:\Program Files\CruiseControl.NET\webdashboard\bin\ThoughtWorks.CruiseControl.MsBuild.dll</logger>
        </msbuild>
     </tasks>
     <publishers>
        <merge>
           <files>
              <file>C:\CruiseControl\WorkingDirectory\dev\SourceMonitorSummary\TestResults\*.trx</file>
              <file>Coverage.xml</file>
           </files>
        </merge>
        <xmllogger logDir="C:\CruiseControl\buildlogs" />
       <email from="BuildMaster@MailServer.com" mailhost="MailServer" mailhostUsername="BuildMaster" mailhostPassword="pwd" includeDetails="TRUE">
       <users>
          <user name="Matt.Roswurm" group="Developers" address="Matt.Roswurm@MailServer.com"/>
          <user name="BuildMaster" group="Build" address="BuildMaster@MailServer.com"/>
          <user name="ProjectManager" group="Developers" address="ProjectManager@MailServer.com"/>
       </users>
       <groups>
          <group name="Developers" notification="failed"/>
          <group name="Build" notification="always"/>
       </groups>
       </email>
     </publishers>
   </project>
</cruisecontrol>

The interesting stuff happens in the MSBuild task. CruiseControl is executing exactly the same build that the developer executes locally. Note the custom logger in the <buildArgs>, this is required to convert the MSBuild output in to XML. Next CruiseControl will merge the output files that we created in the automated build into the build log, this will make the test results and code coverage output available to the build report. The *.trx file contains the test results and the Coverage.xml file holds the code coverage output.

Finally we have the dashboard.config changes that will display the MSTest output.

<?xml version="1.0" encoding="utf-8" ?>
<dashboard>
  <remoteServices>
    <servers>
      
      <server name="local" url="tcp://localhost:21234/CruiseManager.rem" allowForceBuild="true" allowStartStopBuild="true" />
    </servers>
  </remoteServices>
  <plugins>
    <farmPlugins>
      <farmReportFarmPlugin />
      <cctrayDownloadPlugin />
    </farmPlugins>
    <serverPlugins>
      <serverReportServerPlugin />
      <serverLogServerPlugin />
      <serverInformationServerPlugin />
    </serverPlugins>
    <projectPlugins>
      <projectReportProjectPlugin />
      <latestBuildReportProjectPlugin />
      <viewAllBuildsProjectPlugin />
      <projectStatisticsPlugin xslFileName="xslstatistics.xsl" />
      <serverLogProjectPlugin />
      <viewConfigurationProjectPlugin />
    </projectPlugins>
    <buildPlugins>
      <buildReportBuildPlugin>
        <xslFileNames>
          <xslFile>xslheader.xsl</xslFile>
          <xslFile>xslmodifications.xsl</xslFile>
          <xslFile>xslcompile.xsl</xslFile>
         
          <xslFile>xslMsTestSummary.xsl</xslFile>
          <xslFile>xslfxcop-summary.xsl</xslFile>
          <xslFile>xslNCoverSummary.xsl</xslFile>
          <xslFile>xslSimianSummary.xsl</xslFile>
          <xslFile>xslfitnesse.xsl</xslFile>
          <xslFile>xslMSTestCoverage.xsl</xslFile>
        </xslFileNames>
      </buildReportBuildPlugin>
      <buildLogBuildPlugin />
      <xslReportBuildPlugin description="NUnit Details" actionName="NUnitDetailsBuildReport" xslFileName="xsltests.xsl" />
      <xslReportBuildPlugin description="NUnit Timings" actionName="NUnitTimingsBuildReport" xslFileName="xsltiming.xsl" />
      <xslReportBuildPlugin description="NAnt Output" actionName="NAntOutputBuildReport" xslFileName="xslNAnt.xsl" />
      <xslReportBuildPlugin description="NAnt Timings" actionName="NAntTimingsBuildReport" xslFileName="xslNAntTiming.xsl" />
      <xslReportBuildPlugin description="FxCop Report" actionName="FxCopBuildReport" xslFileName="xslFxCopReport.xsl" />
      <xslReportBuildPlugin description="NCover Report" actionName="NCoverBuildReport" xslFileName="xslNCover.xsl" />
      <xslReportBuildPlugin description="Simian Report" actionName="SimianBuildReport" xslFileName="xslSimianReport.xsl"/>
      <xslReportBuildPlugin description="Fitnesse Report" actionName="FitnesseBuildReport" xslFileName="xslFitnesseReport.xsl"/>
      <xslReportBuildPlugin description="MSTest Report" actionName="MSTESTReport" xslFileName="xslMsTestSummary.xsl"/>
      <xslReportBuildPlugin description="MSTest Coverage Report" actionName="MSTestCoverageReport" xslFileName="xslMSTestCoverage.xsl"/>
    </buildPlugins>
  </plugins>
</dashboard>

There is another great blog entry covering MSTest and CruiseControl .NET at BM Bloggers. In that blog Richard talks about MSTest and displaying the test results in the build report. I have added his MSTestCoverage.xsl to the <xslFileNames> and commented out the unitttest.xsl as I’m not using NUnit. For code coverage report, John Cunningham to the rescue, Team System “off-road” code coverage analysis and reporting experience has a great description of how to format the code coverage data using XSLT transformation.

I don’t think that I have done anything really new here, I’ve just pulled together information from several sources that allow MSTest to co-exist with CruiseControl.net. I’d would love to here about other peoples experences with MSBuild/MSTest in their environments.