Showing posts with label MSTest. Show all posts
Showing posts with label MSTest. Show all posts

Friday, January 28, 2011

xUnit.Net – Running the tests (TestCategory)

In my previous post I showed an example about converting a MSTest class to xUnit.Net, and now I want to provide a solution for converting MSTest TestCategory attribute to an equivalent implementation in xUnit.Net.

MSTest allowed us to run the test that belongs to an specific category, let’s see a solution on how this can be accomplished in xUnit.Net.

using Xunit;
using Xunit.Extensions;

namespace xUnitCustomizations
{
    [CustomTestClassCommand]
    public class TestClass
    {
        [Fact, TestCategory("Unit")]
        public void FastTest()
        {
            Debug.WriteLine("fast test executed");
            Assert.True(true);
        }

        [Fact, TestCategory("Integration")]
        public void SlowTest()
        {
            Thread.Sleep(5000);
            Debug.WriteLine("slow test executed");
            Assert.True(true);
        }
    }
}

Create TestCategory attribute
 
namespace Xunit.Extensions
{
    public class TestCategoryAttribute : TraitAttribute
    {
        public TestCategoryAttribute(string category)
        : base("TestCategory", category) { }
    }
    ...
}

CustomTestClassCommandAttribute attribute is used to indicate that a custom test runner will be used

public class CustomTestClassCommandAttribute : RunWithAttribute
{
    public CustomTestClassCommandAttribute() : base(typeof(CustomTestClassCommand)) { }
}

CustomTestClassCommand  is the class that implements ITestClassCommand and acts as the runner for the test fixture

public class CustomTestClassCommand : ITestClassCommand { // Delegate most of the work to the existing TestClassCommand class so that we // can preserve any existing behavior (like supporting IUseFixture<T>). readonly TestClassCommand cmd = new TestClassCommand(); #region ITestClassCommand Members public object ObjectUnderTest { get { return cmd.ObjectUnderTest; } } public ITypeInfo TypeUnderTest { get { return cmd.TypeUnderTest; } set { cmd.TypeUnderTest = value; } } public int ChooseNextTest(ICollection<IMethodInfo> testsLeftToRun) { return cmd.ChooseNextTest(testsLeftToRun); } public Exception ClassFinish() { return cmd.ClassFinish(); } public Exception ClassStart() { return cmd.ClassStart(); } public IEnumerable<ITestCommand> EnumerateTestCommands(IMethodInfo testMethod) { return cmd.EnumerateTestCommands(testMethod); } public bool IsTestMethod(IMethodInfo testMethod) { return cmd.IsTestMethod(testMethod); } public IEnumerable<IMethodInfo> EnumerateTestMethods() { string category; foreach (IMethodInfo method in cmd.EnumerateTestMethods()) { category = string.Empty; foreach (IAttributeInfo attr in method.GetCustomAttributes(typeof(TestCategoryAttribute))) category = attr.GetPropertyValue<string>("Value"); if (category.ToLower().Contains("unit")) // We can make this configurable yield return method; } } #endregion }


The Method public IEnumerable<IMethodInfo> EnumerateTestMethods() filters the tests methods by TestCategory's attribute Value, note that we can make this configurable so we can configure our CI server to run unit test as soon as there are changes in the repository to provide quick feedback and schedule (e.g. once a day) the execution slower test like integration or Web UI test.

Saturday, October 9, 2010

CruiseControl.Net and MSTest

I wanted to put together some notes about CruiseControl.Net and MSTest to have it as reference:

First, to be able to generate reports the following element is required on ccnet.config file.

<xmllogger />     

this element is used to create the log files read by the CruiseControl.NET web page, if it isn't defined the web page will throw the following error:

Exception Message
Request processing has failed on the remote server: Unable to find Log Publisher for project so can't find log file.

To view the MSTest report, we need to add the result file in ccnet.config to merge the content in the build report, for example :

..................
<exec>
<executable>C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\MSTest.exe</executable>
<baseDirectory>.\</baseDirectory>
<buildArgs> /testcontainer:SeleniumSearch\bin\Debug\SeleniumSearch.dll /resultsfile:Results\results1.trx</buildArgs>
</exec>
tasks>
<publishers>
<merge>
<files>
<file>.\Results\results1.trx</file>
</files>
</merge>
<xmllogger />
</publishers>
..................
Include the MSTest plugin to generate the report on the web page, for example dashboard.config file

..................
<buildPlugins>
<buildReportBuildPlugin>
<xslFileNames>
<xslFile>xsl\header.xsl</xslFile>
<xslFile>xsl\modifications.xsl</xslFile>
<xslFile>xsl\MsTestSummary2008.xsl</xslFile>
</xslFileNames>
</buildReportBuildPlugin>
<buildLogBuildPlugin />
<xslReportBuildPlugin description="MSTest2008 Report" actionName="MSTESTReport" xslFileName="xsl\MsTestReport2008.xsl"/>
</buildPlugins>
..................
And include the xls file for MSTest report on ccservice.exe.config/ccnet.exe.config

..................
<!-- Specifies the stylesheets that are used to transform the build results when using the EmailPublisher -->
<xslFiles>
<file name="xsl\header.xsl"/>
<file name="xsl\compile.xsl"/>
<file name="xsl\unittests.xsl"/>
<file name="xsl\MsTestSummary2008.xsl"/>
<file name="xsl\fit.xsl"/>
<file name="xsl\modifications.xsl"/>
<file name="xsl\fxcop-summary_1_36.xsl"/>
</xslFiles>
..................
The xls used on the above example is the one included in ccnet installation. I tested the report generation running tests on VS 2010 and VS 2008 and both reports are generated correctly. The only thing that caused issues was having MSTest on the path windows environment variables on a computer that has both VS 2010 an VS 2008

Also, if MSTest needs to connect to external resources, like accessing remote files or connecting to SQL Server, make sure the account that runs CruiseControl.Net Server windows service (by default is Local System Account) has access to those resources.

Friday, October 1, 2010

CruiseControl.Net driving Selenium Tests

This is how you can quickly get CruiseControl.Net running and driving your selenium test – in 6 steps:

  1. Donwload & Install CruiseControl.Net
  2. Start “CruiseControl.NET Server“ Windows Service
  3. Download SeleniumTest sample project and extract it on a folder C:\Projects\.
  4. Edit CruiseControl.NET configuration file (C:\Program Files (x86)\CruiseControl.NET\server\ccnet.config) and replace it with the information in C:\Projects\SeleniumTest\CruiseControl.Net\ccnet.config. Modify it with proper locations of MSBuild.exe, MsBuild.dll and MSTest.exe 
  5. Start selenium server by running C:\Projects\SeleniumTest\SeleniumServerPort4441.bat.
  6. Open http://localhost/ccnet and you will se the project SeleniumSearchTest1 click on Force. It should build the project and start the tests.

The tests will run using chrome, you can change this parameter in C:\Projects\SeleniumTest\SeleniumSearch\SeleniumSearchTest.cs and use "*iehta" for Internet Explorer or  "*firefoxproxy" for Firefox.

// "*iehta" IE -  "*firefoxproxy" FF - *googlechrome
selenium = new DefaultSelenium("localhost", 4441, "*googlechrome", "http://seleniumhq.org/");

This is how the ccnet.config file looks

<cruisecontrol xmlns:cb="urn:ccnet.config.builder">
<project>
<name>SeleniumSearchTest1</name>
<triggers>
<scheduleTrigger time="23:30" buildCondition="ForceBuild">
<weekDays>
<weekDay>Monday</weekDay>
<weekDay>Tuesday</weekDay>
<weekDay>Wednesday</weekDay>
<weekDay>Thursday</weekDay>
<weekDay>Friday</weekDay>
</weekDays>
</scheduleTrigger>
</triggers>
<workingDirectory>C:\Projects\SeleniumTest</workingDirectory>
<tasks>
<msbuild>
<executable>C:\Windows\Microsoft.NET\Framework\v3.5\MSBuild.exe</executable>
<workingDirectory>C:\Projects\SeleniumTest</workingDirectory>
<projectFile>SeleniumTest.sln</projectFile>
<buildArgs>/noconsolelogger /p:Configuration=Debug /v:diag</buildArgs>
<targets>Build</targets>
<timeout>900</timeout>
<logger>C:\Program Files (x86)\CruiseControl.NET\server\ThoughtWorks.CruiseControl.MsBuild.dll</logger>
</msbuild>
<exec>
<executable>DelResults.bat</executable>
<baseDirectory>.\</baseDirectory>
</exec>
<exec>
<executable>C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\MSTest.exe</executable>
<baseDirectory>.\</baseDirectory>
<buildArgs> /testcontainer:SeleniumSearch\bin\Debug\SeleniumSearch.dll /resultsfile:Results\results1.xml</buildArgs>
</exec>
</tasks>
<publishers>
<merge>
<files>
<file>.\Results\results1.xml</file>
</files>
</merge>
<xmllogger />
</publishers>
</project>
</cruisecontrol>

This example applies for VS 2008 Professional, ccnet 1.5.7256.1 and Selenium RC 1.0.3

Links: