Setting up BDD javascript testing in Visual Studio Team Services

21. April 2017 20:54 by martijn in javascript

Visual Studio Team Service

Lately I have been very impressed by Visual Studio Team Services. Working in this environment is really a breeze and having build agents in the cloud makes development and setting up a build or release pipeline really easy :) This blogpost however will dive into using VSTS build with Javascript BDD testing. The amount of options and frameworks surrounding this can be a bit overwhelming and so this blogpost might give some guidance on how you could set it up. 

What do you mean BDD?

By BDD testing I mean specifying your tests in a Gherking style like this

 

Scenario: A bottle falls from the wall
   Given 100 green bottles are standing on the wall
   When 1 green bottle accidentally falls
   Then there are 99 green bottles standing on the wall

Using a Gherkin syntax provides an exta abstraction layer over more direct style unit-testing frameworks like Mocha or Jasmine. Here a non-developer could provide extra scenario's and the scenario's are pretty readable.

What are we building?

We want to be able to run our javascript BDD tests on Visual Studio Team Services and integrate the test results in our build. We want to see the results in the test tab of our build results.

 What do you need?

NodeJS

First of all you need to install Node.js for running javascript code and downloading javascript packages. Download it here. Once you have Node.js installed you should be able to run npm (Node Package Manager) from the command line. NPM provides an easy way to install javascript modules and dependencies of these modules automatically. If you have never used npm you should read up here.

Mocha 

Now you have installed npm the next step is creating a folder that will be the base folder for our development. In this folder we will initialize npm so npm creates a config file called package.json keeping track of our required modules. So open a command prompt to create a folder and initialize npm.

 

mkdir bddjavascript
cd bddjavascript
npm init (press enter a few times on all the questions)

 Next we need to install Mocha. Mocha is a javascript framework that will let us write tests. To install Mocha we use npm again.

npm install --save-dev mocha

 

As you can see we now have a "devDependency" on Mocha. This means that if run "npm install" in this folder without parameters Mocha will be installed again. Mocha itself was installed in the node_modules folder along with it's depencies.

 

 

Running our first test

Now we have Mocha installed we should verify that it works correct. Create a folder called test in the root of our project (next to package.json) and create a file in that folder called test.js with the following contents:

var assert = require('assert');

describe('Array', function() { 

  describe('#indexOf()', function() { 

    it('should return -1 when the value is not present', function() { 

      assert.equal(-1, [1,2,3].indexOf(4)); 

    }); 

  });

});

To run this test we use mocha from the command line.

node_modules\.bin\mocha

And you should see the test pass.

Yadda Yadda Yadda

Next we can add the BDD layer by installing Yadda. In the console we use npm again

npm install yadda --save-dev

Now we have Yadda we need a few more files to get it all working. Yadda requires a feature file with our scenario('s), a file implementing the steps and normally an object under test. For the object under test we'll use a regular javascript array so we don't need a file for that. In our test folder we create a features folder.

mkdir features

In this folder create a file called "array.feature" containing:

Feature: Javascript array

Scenario: An array should contain more elements after adding an element

Given I have an array of 2 elements

When I add an element

Then my array contains 3 elements

Next to features folder create a steps folder with a array-steps.js file containing

 var assert = require('assert');

var English = require('yadda').localisation.English;

 

module.exports = (function() {

  return English.library()

    .given("I have an array of $NUM elements", function(number, next) {

       array = new Array(parseInt(number));

       next();

    })

    .when("I add an element", function(next) {

       array.push("element");

       next();

    })

    .then("my array contains $NUM elements", function(number, next) {

       assert.equal(parseInt(number), array.length);

       next();

    });

})();

Now we only need a main entry point for running the feature file. Create a file called yadda-test.js in the root of your project

var Yadda = require('yadda');

Yadda.plugins.mocha.StepLevelPlugin.init();

 

new Yadda.FeatureFileSearch('./test/features').each(function(file) {

 

  featureFile(file, function(feature) {

 

    var library = require('./test/steps/array-steps');

    var yadda = Yadda.createInstance(library);

 

    scenarios(feature.scenarios, function(scenario) {

      steps(scenario.steps, function(step, done) {

        yadda.run(step, done);

      });

    });

  });

});

Your project structure should resemble this

Running yadda can be done from the command line using

node_modules\.bin\mocha yadda-test.js

And you should see the tests pass

Running Mocha in the build proces

Finally we need to run the tests as part of our build proces. We are going to npm for this again and redefine the test script in our package.json file. Change the package.json file so mocha is called in the "test" script. And run "npm test" to verify it runs our test.

 

{
  "name": "bddjavascript",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "mocha yadda-test.js"
  },
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "mocha": "^3.2.0",
    "yadda": "^1.3.0"
  }
}

Next we should check in our code and start working on the build. Create a new empty build definition in VSTS and add a npm task. Set the working folder of the task so it runs in the folder where your package.json is. 

Verify your build works by saving and queueing it. You should see npm installing the modules. Next add another npm task in the same folder but this time run the "test" command

 

Save and queue again and you should see the tests are run in the log.

Getting the test results

Now we have the tests running in VSTS it would be great if we could also get the test result in test tab. This can be done by letting mocha report in JUnit format and let VSTS pick up the result. First install the mocha-junit-reporter

npm install mocha-junit-reporter --save-dev

Next make sure npm test uses this test reporter by editing the "test" script

"scripts": {
"test""mocha yadda-test.js --reporter mocha-junit-reporter"
  },

In VSTS we can then publish the results by adding a "Publish test results" task to our pipeline

 

Finally make sure the "npm test" task continues on error so the results are always published.

  

Getting templateID of glass generated interface

26. January 2017 08:18 by martijn in

If you are using glass with the t4 template genaration you can generate partial interfaces for your Sitecore datasource templates. These interfaces look a something like this: 

I create a simple helper to get the ID like this

  

And here's the implemenation:

public static class GlassHelper

{

        public static Guid GetTemplateId<T>() where T : ISitecoreBase

        {

           var attribute = typeof(T).GetCustomAttributes(typeof(SitecoreTypeAttribute), true).FirstOrDefault() as SitecoreTypeAttribute;

           return !string.IsNullOrEmpty(attribute?.TemplateId) ? 

               new Guid(attribute.TemplateId) : 

              Guid.Empty;

        }

}

Deploying unicorn yml files using web deploy.

25. January 2017 15:30 by martijn in sitecore

Where's my unicorn?

 

There has been al lot of attentention for Sitecore Helix and Habitat lately and it has indeed been a great inspiration for project structure and general Sitecore ideas. The Sitecore Habitat example however does not provide any guidance in handling the unicorn files in a build environment. This blog post shows a way in which the yml files are included in the Feature zip web-deploy files automatically. This means you don't have to add the yml files to TFS and your project. Futhermore Habitats folder structure is maintained so the templates are in a "serialiazation" folder next to the "code" folder containing the project. 

 Habitat folder structure

 The "code" folder contains the .csproj of the website project folder and "serialization" is the physicalRootPath configured in your unicorn file for the feature.

Auto-including the yml files

So how to get the files in "serialization" folder in the zip created by web deploy? This can be done by modifying the csproj file and including a custom targets file (unicorn.targets). In the targets file the CopyAllFilesToSingleFolderForMsdeployDependsOn target is extended. In this target the yml files in serialization folder are added FilesForPackagingFromProject collection. The files are included at the App_Data/<ProjectName> folder. That way when all the features are deployed no files are overwritten since each feature has its yml in its own folder within App_data. 

 

Once we build a project zip with this we get the following structure in the zip. 

 

The target files can be imported by adding the following line to the bottom of your csproj:

<Import Project="../../../unicorn.targets" Label="Unicorn" />

Download the targets file here : unicorn.targets (1.11 kb)

Hope this is of some use to you!

 

Compiling cshtml from visual studio tools menu

10. May 2016 15:46 by martijn in

We all know and love the MvcBuildViews settings you can add to your project to enable compilation of views during build. But sometimes this does not give you the desired result and it slows down your build considerably. An alternative is to invoke the view compilation on demand by configuring it as a external tool.

This is actually quite easy to do and integrates nicely with VS. In the screenshot below you see how to configure an external tool for this. The only thing that is not in the screenie is the path to the aspnet_compiler.exe which is invoked. The path i have is C:\Windows\Microsoft.NET\Framework64\v4.0.30319\aspnet_compiler.exe but this obviously depends on the version of .net your are using. 

 

To invoke it just select your web project and click Tools -> Compile views. The cool thing is that VS opens the file with the error once you doubleclick a line.  Sweet :)

 

using jQuery File Ajax Upload plugin with ASP.NET

25. March 2015 21:37 by martijn in jQuery, jquery.fileuploader

Looking at the jquery fileupload plugin I noticed there wasn't a decent example of how to get it working in a asp.net webforms application. So I decided to share my little simple sample project with you. 

 

The project features:

  1. Selection of a file
  2. A separate upload button
  3. A progress bar

  

The project uses:

  1. Jquery file uploader https://blueimp.github.io/jQuery-File-Upload/
  2. Bootstrap
  3. Jquery 1.9.1
  4. Jquery UI for the widget (this seems a bit heavy but the demo breaks i
  5. A simple ashx handler for the server side

The ashx handler is very basic and you should probably expand it. I found some nice resources here but the project was not included for download. 

Download it here

UploadSample.zip (2.27 mb)

Debugging log4net

14. March 2015 19:03 by martijn in log4net

Log4net is a great tool but it can be a bit tricky to get working. Adding a debug appender that logs to the visual studio output can be of great help. You can debug log4net by attaching the log4net DebugAppender:

Add a log4net app setting in your app.config file:

<appSettings>

<!-- log4net configuration when running in debug mode. -->

<add key="log4net.Internal.Debug" value="true"/>

</appSettings>

Add a debug appender in the log4net config:

<appendername="DebugAppender" type="log4net.Appender.DebugAppender">

<immediateFlush value="true"/>

<layout type="log4net.Layout.SimpleLayout"/>

</appender>

Finally add the appender to the log4net config root:

<root>
<level value="ALL"/>
<appender-ref ref="DebugAppender"/>
</root>

Running SOAP UI tests from MsTest

13. March 2015 13:37 by martijn in .net, mstest, soapui

Recently I have been working in a project in which the tester used a tool called Soap UI. SoapUI makes it easy to create xml based test for services. Being a big fan of Continuous Integration I thought it would be cool to include SoapUI in our CI build cycle. Here's what you need to do to make it working

 

1. Install Soap UI

You can get a free version here. Take notice of the location in which it installs. For me this was : "C:\Program Files (x86)\SmartBear\SoapUI-4.6.4". I put this in the app.config of my test project.

2. Create a SoapUI project. 

Start up your SoapUI and go File->New Project. Point it to the wsdl of the service you want to test and let it create a testsuite. 

 

 

3. Add the Soap UI project to your MsTest project

In SoapUI right click save-project as on the project node. Save your xml file into a unit test project and include the xml in your project. Set Copy to Output Directory to "Copy if newer"

  

4. Running soap UI from MsTest

This is the tricky part. How can we get SoapUI to run our test from a UnitTest?? It turns out SoapUI has something called a Testrunner.bat which makes it relatively straightforward. Basically I create a testrunner proces in a unit test and redirect the output of SoapUI to the test output. Here's what the code looks like: (a zip of the project is included below). 

        /// <summary>

       /// Runs soapUI test named testName

        /// </summary>

        private void RunSoapUItest(string soapProject, string testSuiteName, string testName)

        {

            const string fileName = "cmd.exe";

            var soapProjectFileName = Path.GetFullPath(soapProject);

            var arguments = string.Format("/C testrunner.bat -s\"{0}\" -c\"{1}\" \"{2}\" ", testSuiteName, testName, soapProjectFileName);

 

            //for me this is C:\Program Files (x86)\SmartBear\SoapUI-4.6.4\bin

            var soapHome = System.Configuration.ConfigurationManager.AppSettings["SoapUIHome"];

            //start a process and hook up the in/output

            var proces = new Process

            {

                StartInfo = new ProcessStartInfo

                {

                    FileName = fileName,

                    Arguments = arguments,

                    WorkingDirectory = soapHome,

                    CreateNoWindow = true,

                    ErrorDialog = false,

                    RedirectStandardError = true,

                    RedirectStandardOutput = true,

                    UseShellExecute = false

                },

                EnableRaisingEvents = true

            };

            //pipe the output to console.writeline

            proces.OutputDataReceived += (sender, args) => Console.WriteLine(args.Data);

 

            //store the errors in a stringbuilder

            var errorBuilder = new StringBuilder();

            proces.ErrorDataReceived += (sender, args) =>

            {

                if (args != null && args.Data != null)

                {

                    errorBuilder.AppendLine(args.Data);

                }

            };

            proces.Start();

            proces.BeginOutputReadLine();

            proces.BeginErrorReadLine();

 

            proces.WaitForExit();//wait for SoapUI to finish

 

            //fail the test if anything fails

            var errorMessage = errorBuilder.ToString();

            if (!string.IsNullOrEmpty(errorMessage))

            {

                Assert.Fail("Test with name '{0}' failed. {1} {2}", testName, Environment.NewLine, errorMessage);

            }

        }

 5. Running a test per SoapUI testCase

A typical SoapUI project includes multiple testcases. To make each testcase run as a separate test we can use something called a DataSource attribute for the test. This attribute cause the test to run for every 'Row' in the XML.

[DataSource("Microsoft.VisualStudio.TestTools.DataSource.XML", "TestData\\test-service.xml", "testCase", DataAccessMethod.Random)]

Since every testCase is a row the makes every testCase run separately. You have to make sure you get xml deployed (I used a deploymentItem for that).

 

        /// <summary>

        /// Test Soap Stuff

        /// </summary>

        [TestMethod]

        [DeploymentItem(@"test-service.xml", "TestData")]

        [DataSource("Microsoft.VisualStudio.TestTools.DataSource.XML", "TestData\\test-service.xml", "testCase", DataAccessMethod.Random)]

        public void RunSoapServiceTest()

        {

            var testCaseName = TestContext.DataRow["name"].ToString();

            RunSoapUItest(@"TestData\test-service.xml", "BasicHttpBinding_ILicenseService TestSuite", testCaseName);

        }

I used the deploymentitems because I don't want absolute paths etc (I need the buildserver to handle this as well). Run the test and we get SoapUI output in Visual Studio / Build. Sweet :) To run this in a CI build you only need to install SoapUI on the buildserver and all should be well.

Here's the project:

MsTestSoapUI.zip (12.08 kb)

 

Happy Testing!

Post deployment Integration testing using TFS 2012 builds

16. December 2014 17:06 by martijn in tfs2012, testing

Why should I do integration testing?

Although I very much like unit-testing and frameworks like Moq they sometimes miss the big picture. You might have 99% code coverage and a green test suite only to find out later you messed up your environment because you did not register your new code in your dependency configuration or forgot to check-in some other configuration. Every little part of your code was tested but the sum did not work out. What if you could deploy to a test server and run your tests against this server? This is what post-deployment integration testing is about. The process below describes setting up a TFS build definition that does compile, test, deploy and integration-test. If any step fails the build fails. You can combine this with a gated check-in to make sure your environment never brakes. I would recommend to use the lab environment for this if you have set this up already : http://msdn.microsoft.com/en-us/library/hh191495.aspx. The method described here uses just a build controller and build agent.

Describing the process

The proces is a small modification of the normal gated-checkin process of TFS 2012. Only a integration-tests step is added.

Modifying the build template

To get integration testing we need to modify the build in two ways.

  1. Add a integration testing step to the build process. This is basically a copy of the normal unit-testing step with a slight modification. Instead of using the TestSpecs and DisableTest variabeles we use IntegrationTestSpecs and DisableIntegrationTest. I included the definition file for download below.
  2. Make sure your integration test are only run in the integration step and not in the regular unit-testing step. You can accomplish this by using categories for your integration tests. 

 

BuildTestDeployAndTestTemplate.xaml (117.36 kb)

Logging all WCF messages

16. September 2014 10:38 by martijn in

Just as a reminder. Here's how you log all messages for a wcf service. Copy & Paste into web.config.

<system.diagnostics>

    <sources>

      <source name="System.ServiceModel.MessageLogging" switchValue="Verbose">

        <listeners>

          <add name="messages"

          type="System.Diagnostics.XmlWriterTraceListener"

          initializeData="c:\logs\messages.svclog" />

        </listeners>

      </source>

    </sources>

  </system.diagnostics>

  <system.serviceModel>

    <diagnostics>

      <messageLogging

           logEntireMessage="true"

           logMalformedMessages="true"

           logMessagesAtServiceLevel="true"

           logMessagesAtTransportLevel="true"

           maxMessagesToLog="300000"

           maxSizeOfMessageToLog="200000"/>

    </diagnostics>

  </system.serviceModel>

Synching FileZilla settings using Dropbox

23. December 2013 11:28 by martijn in

I love Dropbox. One of the things it allows me to do is have setting files (keys/docs) synched accross my computers without any hassle. Combined with the sitemap.xml of FileZilla and I have easy access to my ftp accounts on all my machines. This howto describes how you can set up FileZilla / Dropbox so your sitemanager is stored on your DropBox.

So how can you set it up?

  1. First copy your FileZilla settings to your Dropbox. For me the settings files were stored in C:\Users\Martijn\AppData\Roaming\FileZilla. Copy the whole folder to your dropbox eg: C:\Users\Martijn\Dropbox\Filezilla
  2. Next we need to change the FileZilla defaults. There is a file called fzdefaults.xml.example in your FileZilla program files 'docs' folder (C:\Program Files (x86)\FileZilla FTP Client\docs). Copy this file to the folder above (C:\Program Files (x86)\FileZilla FTP Client\) and rename it to fzdefaults.xml. FileZilla will pick up this file and use it to find the other configuration files.
  3. Change the Config Location key to point to your Dropbox FileZilla folder of step 1

 

That's it! Happy synching!