Testing Umbraco backend extensions with Jasmine

Heads Up!

This article is several years old now, and much has happened since then, so please keep that in mind while reading it.

Introduction

This article is the second part in a series of two articles about extending the Umbraco 7 backend. Our previous article covered some vital aspects about using AngularJS in combination with TypeScript. The current article introduces testing the client-side application using Jasmine unit tests. We assume some basic AngularJS knowledge, there is no room to explain basic AngularJS concepts. Of course, we are more than happy to answer (almost) all of your questions.

Tooling: Visual Studio, Jasmine and Chutzpah

At Seven Stars HQ, we are used to have Visual Studio as our IDE. Besides developing front-end code, we also write lots of .NET code. Although Visual Studio might not be the most efficient IDE to use for client-side development, being accustomed to an IDE and assisting development tools does also count for us. Combined with some extensions, the client-side testing experience using Visual Studio is very impressive.

We are using Jasmine (version 2.0) as our unit test framework for client-side development. Jasmine (http://b.ldmn.nl/jasmine-20) is a very powerful behavior-driven unit test framework for testing JavaScript code. Although we will cover some basics about Jasmine, we suggest you to follow the link to the official documentation to discover all the possibilities Jasmine has to offer.

Visual Studio does not offer 'out-of-the-box' support for Jasmine. Therefore, we use a tool called Chutzpah (http://b.ldmn.nl/chutzpah-20). You enable Chutzpah by installing two small Visual Studio extensions.

Chutzpah extensions
Chutzpah extensions

After installation, you can use the Visual Studio Test Explorer or right-click any test file to fire your tests or get code coverage results for your tests.
Currently, we are still writing most Jasmine unit tests in plain old JavaScript. It would be very nice to have them written in TypeScript. At the moment, the added complexity (learning curve, since almost no examples exist using Jasmine and TypeScript) and the fact that test code is no production code has until now resulted in our decision not to focus much on TypeScript for Jasmine unit tests.

In this article, we will cover some Jasmine basics first. After that, we combine those basics with some Jasmine basics for testing an AngularJS application. Since the proof of the pudding is in the eating, we finally show a Jasmine test suite testing the AppointmentsService class from our previous article.

Testing with Jasmine

Suites (describe, beforeEach, afterEach)

To start unit testing with Jasmine, you always start creating a suite. A suite is declared by the describe function, which has two parameters: a name and a body function. The suite combines tests which belong together.

A suite contains a set of test specifications, but is primarily responsible for test environment setup (using the beforeEach function) and teardown (using the afterEach function). As their function names imply, those functions are respectively called before and after each test specification. The beforeEach function is the right place to mock the boundaries of your system under test.
For better granularity, you can nest suites to compose trees of tests, for instance grouped by methods under test. Setup and teardown is performed in order, which makes it very convenient to prepare and clean up the test environment on different levels with their own concerns.

Test specifications (it)

When testing code, the best practice is to test all public functions for your TypeScript classes. Of course, it is possible to test private functions as well, but it should not be necessary, because public functions should call all private functions. If reaching 100% code coverage on your file is not possible when testing all possible code paths, you probably developed some unused code.
A test specification is declared by the it function, looking pretty much the same as the describe function with a name and body function parameter. Inside a test specification, expectations are crucial to verify the code behaves like you expect it to.

At Seven Stars, we like to keep our test specifications as predictable as possible. Therefore, we always follow the Arrange (set up some test prerequisites you cannot setup in the suite's beforeEach function), Act (the actual call to the system under test), Assert (verify expactations) structure inside each test specification.

Expectations

Roughly divided, there are two types of expectations:

  • expectations on the results from the system under test
  • expectations on the behavior of the system under test

It is quite easy to assert results using the expect function. Jasmine provides us with lots of asserting function calls like toBe, toEqual and toBeNull.
The real power of Jasmine, however, lies in testing the behavior of our system under test. For instance, if your service calls a resource, you probably like to assert the resource is called exactly one time and using the expected parameters. Jasmine provides us with spies to assert behavior. Spies are described using the spyOn function (when spying on existing objects/functions) or the createSpy function (to create a new spy, especially handy when mocking system boundaries).

Testing an AngularJS application

System under test

By default, Jasmine provides us with a lot of test supporting functions. In order to get a running test, we have to reference the system under test. This is taken care of by /// <reference> comments, which Jasmine uses to load test dependencies.

The system under test is the Javascript file you are going to test. When using a complex environment like AngularJS, only referencing the system under test will not suffice. You should also reference every component the system under test depends on (which cannot be mocked). Sometimes, you could reach success by just referencing AngularJS and the ngMock module (more about that one below). If you use extension libraries to make life easier (we often use libraries like underscore.js and moment.js), you have to reference those as well.

AngularJS dependency injection

AngularJS heavily depends on dependency injection and asynchronous function calls (using promises). Therefore, you would expect testing AngularJS code to be difficult. Fortunately, it is not, because AngularJS is built with testing in mind. AngularJS provides the ngMock module (https://docs.angularjs.org/api/ngMock), which provides support to inject and mock Angular services into your Jasmine tests. A lot of AngularJS's core modules are covered by default. For example, think of the root scope ($rootScope) and the queuing service ($q).

If you need AngularJS dependency injection for your system under test, you have to use the inject function (provided by the ngMock module) in the beforeEach function. The beforeEach function only takes one parameter, the body function. The inject function also takes a body function and provides a chaining mechanism to ensure proper test operation. The inject body function can take any number of parameters. The real power of the inject function is that those parameters are injected with AngularJS module mocks (if recognized by ngMock). Thus, specifying the $rootScope parameter actually gives access to a working AngularJS $rootScope object.

Of course, you need the injected objects (and the other mocks you create in beforeEach) in your test specifications. You can reach this goal in two ways: either by specifying global variables in your describe function or placing variables in the 'this' scope. We prefer the latter, because this way there is no need to specify variables and assigning them later on. Of course, this is a matter of taste.

Testing

Testing AngularJS applications is pretty straightforward. There are a few tricky parts in testing, though:

Testing promises requires some extra mocking skills. Luckily, the ngMock module takes care of AngularJS's asynchronous nature. When testing, it is important to keep in mind that ngMock does not always do all its magic by itself: we found out that it is necessary to understand AngularJS heavily uses digestion cycles. In order to get your promises to resolve, you have to kick off the digestion process. This is done by the following command: $rootScope.$apply().

Another thing to keep in mind is that Jasmine is not aware of asynchronicity: when the test reaches the last line of the test specification, success is celebrated, although your promise is not reached yet (and so aren't the expectations inside). Luckily, Jasmine provides us with a done callback function which can be used to notify Jasmine when testing is done. When using the done callback function, Jasmine will wait until the function is called (or the test times out).

Testing the Umbraco backend plugin

Setup

Alright, are you feeling ready for the real deal? We are almost there, after we cover some setup basics first.

We consider separating test and production code to be crucial. Therefore, our Jasmine unit tests are in a separate (web) project. This project should include the Jasmine test framework. You can download the source code from the Jasmine website, or use the 'Jasmine Test Framework' NuGet package. When testing the DOM interaction (preferably in directives), you probably face test exceptions because JQuery is not recognized by default. Referencing JQuery and the Jasmine-JQuery (http://b.ldmn.nl/jasmine-jq) libraries helps.

Finally, you need a Chutzpah.json file when using Chutzpah inside Visual Studio. This file should be somewhere in the folder tree between the tests and the project's root (suggestion: place one in root, provide differentiation in subfolders if needed). Here is an example of the Chutzpah.json file

{
    "Framework": "jasmine",
    
    "TestFileTimeout": 10000,
    
    "TestHarnessLocationMode": "TestFileAdjacent",
    
    "TestHarnessDirectory": "Tests/",

    "TypeScriptCodeGenTarget" : "ES5",
    
    "RootReferencePathMode":"DriveRoot",
    
    "CodeCoverageIncludes": ["*.js"],
    
    "CodeCoverageExcludes": ["*\\<Your Jasmine Test Project Name>\\*"]
}

Chutzpah.json

Testing AppointmentsService.ts

For this article, showing off a fully tested solution would be a bridge too far. Below you can see the test suite for the appointments service. Let's walk through some particularities:

  • Jasmine (and the browser) is a JavaScript test framework. We are not testing our TypeScript class, but the resulting Javascript file. Notice that we therefore include a reference to the Javascript file.
  • Notice the arrange, act, assert structure in each test.
  • It is not very easy to equal objects which are no real memory references. Instead of walking all properties by hand, we use a so-called custom matcher, toEqualData. This custom matcher is in the utils.js file and is registered in the Jasmine test framework in beforeEach.
  • Notice dependency injection is used in beforeEach for $rootScope, $location, $filter and $q.
  • Since we are testing the appointments service, we mock the boundaries of our system under test. One of those boundaries is the appointmentsResource. As you can see, we create all public resource functions using Jasmine spies, which call a fake method returning data like the actual resource would do normally. This mocking structure is a typical example of mocking calls which result in promises.
  • Inside the overall test suite (the describe function named 'appointmentsService') we use nested test suites for each public function of the appointments service. This results in a clearly organized test suite.
  • Notice that for the AppointmentsService.getAppointments function both the result (using toEqualData) and the service behavior is asserted. Regarding asserting the behavior: appointmentsResource.get is one of the spies we created. A spy offers several tracking capabilities, including parameter tracking and tracking how many times the spy has been called.
  • The call to this.$rootScope.$apply makes sure the promise is resolved by AngularJS, causing the then() callback function to be called. In order to be able to test the results of the service function call, expectations have to be inside the then() callback function.
  • Since all tests are dealing with asynchronous behavior, all tests do specify and use the done callback function.
  • The AppointmentsService.saveAppointment function has some business logic inside to either create or update an appointment based on the id value being more than zero (in that case, we are dealing with an existing appointment). Since we are using a REST API which uses POST for creation en PUT for updates, we have to assert if both routes are covered by our function. This is why four (!) tests exist to ensure proper test coverage is reached.

Here is the appointments-service-specs.js test suite file:

/// <reference path="../../../Scripts/typings/jasmine/jasmine.d.ts" />
/// <reference path="../../../Scripts/utils.js" />

/// <reference path="../../../scripts/angular/1.1.5/angular.min.js" />
/// <reference path="../../../Scripts/angular/1.1.5/angular-mocks.js" />
/// <reference path="../../../Scripts/underscore.min.js" />

/// <reference path="../../../../GerGemEmmeloord.Web.UI/App_Plugins/CalendarSection/Backoffice/AngularJS/Services/AppointmentsService.js" />

'use strict';

describe('appointmentsService', function () {

    beforeEach(inject(function ($rootScope, $location, $filter, $q) {
        utils.registerCustomMatchers(this);
        var that = this;

        this.$rootScope = $rootScope;
        this.$location = $location;
        this.$filter = $filter;
        this.$q = $q;

        this.dateMock = new Date();
        this.appointmentsMock = [{ id: 1, start: this.dateMock, end: this.dateMock, title: 'Test Appointment' }];

        this.appointmentsResourceMockMock = {
            get: jasmine.createSpy('get').and.callFake(function (userType, start, end) {
                return { then: function (callback) { return callback(that.appointmentsMock); } };
            }),
            post: jasmine.createSpy('post').and.callFake(function (event) {
                return { then: function (callback) { return callback(event); } };
            }),
            put: jasmine.createSpy('put').and.callFake(function (event) {
                return { then: function (callback) { return callback(event); } };
            })
        };

        this.appointmentsService = new CalendarModule.Services.AppointmentsService(this.$location, this.$filter, this.$q, this.appointmentsResourceMock);
    }));

    describe('getAppointments', function () {
        it('Should get the expected items', function (done) {
            // arrange
            var expected = { events: [{ id: 1, start: this.dateMock, end: this.dateMock, title: 'Test Appointment', location: '' }] };

            // act
            this.appointmentsService.getAppointments('test').then(function (response) {
                // assert
                expect(response).toEqualData(expected);
                done();
            });

            this.$rootScope.$apply();
        });

        it('Should call appointmentsResourceMock.get once', function (done) {
            // arrange
            var that = this;

            // act
            this.appointmentsService.getAppointments('test').then(function (response) {
                // assert
                expect(that.appointmentsResourceMock.get).toHaveBeenCalledWith('test', undefined, undefined);
                expect(that.appointmentsResourceMock.get.calls.count()).toBe(1);
                done();
            });

            this.$rootScope.$apply();
        });
    });

    describe('saveAppointments', function() {
        it('Should save a new appointment (no id) and return the saved appointment', function (done) {
            // arrange
            var saved = { id: undefined, start: this.dateMock, end: this.dateMock, title: 'Test Appointment', location: '' };

            // act
            this.appointmentsService.saveAppointment(saved).then(function (response) {
                // assert
                expect(response).toEqualData(saved);
                done();
            });

            this.$rootScope.$apply();
        });

        it('Should save a new appointment (no id) using appointmentsResourceMock.post (once)', function (done) {
            // arrange
            var that = this;
            var saved = { id: undefined, start: this.dateMock, end: this.dateMock, title: 'Test Appointment', location: '' };

            // act
            this.appointmentsService.saveAppointment(saved).then(function (response) {
                // assert
                expect(that.appointmentsResourceMock.post).toHaveBeenCalled();
                expect(that.appointmentsResourceMock.post.calls.count()).toBe(1);
                done();
            });

            this.$rootScope.$apply();
        });

        it('Should save an existing appointment (id > 0) and return the saved appointment', function (done) {
            // arrange
            var saved = { id: 1, start: this.dateMock, end: this.dateMock, title: 'Test Appointment', location: '' };

            // act
            this.appointmentsService.saveAppointment(saved).then(function (response) {
                // assert
                expect(response).toEqualData(saved);
                done();
            });

            this.$rootScope.$apply();
        });

        it('Should save an existing appointment (id > 0) using appointmentsResourceMock.put (once)', function (done) {
            // arrange
            var that = this;
            var saved = { id: 1, start: this.dateMock, end: this.dateMock, title: 'Test Appointment', location: '' };

            // act
            this.appointmentsService.saveAppointment(saved).then(function (response) {
                // assert
                expect(that.appointmentsResourceMock.put).toHaveBeenCalled();
                expect(that.appointmentsResourceMock.put.calls.count()).toBe(1);
                done();
            });

            this.$rootScope.$apply();
        });
    });
});

appointments-service-specs.js

Considerations

Testing AngularJS code using Jasmine is relatively easy and fun. However, there are some tricky parts you should know about:

  • Testing DOM integration is doable, but difficult. You should never mix business logic and DOM logic (for instance, JQuery) in controllers, services, etc. This quickly makes testing an impossible operation. If you look at the CalendarController.ts file showed in our previous article, you probably see the bad thing happen ;) ... Instead, move DOM integration you cannot mitigate entirely into a directive. This takes care of as much clean, testable classes as possible. Please notice that you can even split a directive into the directive itself and an accompanying directive controller.
  • When possible, try not to use $scope inside controllers. Testing $scope is a bit hairy. Instead, try to use the 'controller as' syntax. One caveat: the current version of Umbraco 7 uses AngularJS v1.1.x, which means the 'controller as' syntax is not in there (it has seen birth since AngularJS v1.2.x). Luckily, the Umbraco core team is considering to upgrade AngularJS someday soon, so stay informed ;) .

Wrapping up

We hope we have got you excited about AngularJS and Jasmine unit testing. As said before, we love to get feedback. You can react below or send us an email. Should you have an opinion on how to get even better unit test code, we would love to learn from you as well ;) ! Happy coding!

Bert Loedeman

Bert is on Twitter as

Marcel Verweij

Marcel is on Twitter as