Building from Blueprints

Heads Up!

This article is several years old now, and much has happened since then, so please keep that in mind while reading it.

The birth of "content templates"

I had had this idea brewing for a long time that some kind of templating was needed to ease editors into using the grid.  I came up with a "templatable grid" that was floating around as open source (and a beta-package) for a while. It served some of the purpose. In some discussion, probably on Twitter, Lee Kelleher commented that templates would be useful for all properties. After all, there was a complete lack of default values for property editors or properties unless they had their own implementation.
This summer I was lucky enough to be invited to the Umbraco retreat. I brought this luggage and proposed we killed the default value problem once and for all. I'd also figured that the simplest way would really be to allow for templated content. And to do that we could just add
one more NodeObjectType GUID to the core, then re-use all the content apis. It was that simple!
To my joy, it was accepted as a good idea. It took Jaevon Leopold, Shannon Deminick and me around 20 minutes to flesh out the requirements  and technical how-to. Then we spent 20-30 minutes figuring a nice name. After all "template" was taken, so we couldn't just go ahead and call it "content templates", could we? After scanning hundreds of synonyms for "template" we stumbled upon "blueprint" and finally agreed that was a good term. We made the feature and it was actually complete in two (long) days of work. I was even more happy when it was released nice and polished in version 7.7. But guess what? 
It's named "content templates".

How does it work

When Jan Skovgaard asked me to write for 21 days in Umbraco, he proposed I'd write about content templates. I was a bit unsure it was a good idea since it's so bloody simple. And besides, Kevin Giszewski has already made a brilliant video tutorial about it.
It can be summed up in a few simple points:
  • Editors (with permission) can save a document node as a content template
  • All editors get an option when creating contnet to base it on existing content templates
  • Users with access to settings can manage content templates
    • Templates are organized by document type
    • Templates can be edited as with content
So what else is this article about then? Well, I hope you got a nice story. But what I've really wanted to write about for the community for a while is unit testing JavaScript. And the thing is - most of the code I did for content templates was the UX for creating and selecting them with dialogs. Shannon did the bulk of the work to re-use the existing content APIs. But we made sure to compete about the most test coverage. The result on my part was that Umbraco's client-side test count grew from 128 to 136 tests. I pray to the binary gods that this number _will_ increase. At least my tests still pass with the latest commit in the core.
So if you want to, tag along for a tour of the JavaScript tests I wrote, otherwise, I hope you enjoyed the story. But wait! There's is a small hidden feature of content templates at the end you might want to check out.
For the next part, we'll be writing "blueprint" descriptions of the features we want and then proving them with tests afterwards. Hence the title: building from blueprints. It's TDD, really! :) (TDD is short for "Test Driven Development")

The prerequisites

To be effective with JavaScript testing; and all those hot-shot ninja JavaScript techiques these days; you need to be able to fire off NPM tasks in Visual Studio (NPM stands for Node.js Package Manager). Granted, you can be a real ninja and just use the CLI (Command Line Interface), but installing Mads Kristensen's NPM Task Runner is a lifesaver. It enables you to run frequent tasks by doubleclicking within a window in VS and seeing the output right there.
With that out of the way we can have a look at what's required to test JavaScript. Given we're testing stuff for the backoffice in Umbraco (property editors, grid editors, dashboards and whatnot) we're stuck with Angular 1.1.5 for now. So that's a requirement. Angular 1.1.5 was pretty tight with the Jasmine testing framework, and it's one of the few frameworks that support our setup. Jasmine is also a requirement.
Finally, we need a test runner. Previously, I've been using Resharper and PhantomJS to run tests ad hoc while writing them. But it's quite quirky, and it's not very useful with build servers and sharing runnable code. The team at Google produced a test runner several years ago called "Testacular". It later evolved into the Karma test runner. Karma is what Umbraco use and what we're going to use here. It runs on node, so it fits well with the toolset we've set up.
When writing pull-requests for Umbraco, there is already a task you'll find in the NPM  task runner called "test:unit". Karma is set up to look for *.spec.js files, so any file in the Umbraco.Web.UI.Client project named like that will be executed. You can find Umbracos own JavaScript tests under Umbraco.Web.UI.Client\test\unit.
When you roll your own JavaScript projects, there might be newer, fancier, better ways. But if you want to stick to the basics, here's the gist of it:

Node packages

Any proper JavaScript cowboy has a package.json file these days. If you don't, I think you'll have an "add item" option in Visual Studio after installing the NPM task runner. You'll need a couple of "devDependencies". Here's a simple setup:
{
    "version": "1.0.0",
    "name": "my.plugin",
    "private": true,
    "scripts": {
        "test": "karma start"
    },
    "devDependencies": {
        "karma": "1.5.0",
        "karma-chrome-launcher": "2.0.0",
        "jasmine": "2.5.3",
        "karma-jasmine-html-reporter": "0.2.2",
        "karma-ng-html2js-preprocessor": "0.2.0"
    }
}

This will add a folder called node_modules in your webproject. The packages will be installed automagically just like with Nuget. You should be so lucky that node_modules is already mentioned in .gitignore, but if it isn't, make sure at once! You don't want those packages in source control.

It will also add an option to your package runner called test that you can use for running the tests.

Setting up Karma

Karma is basically a simple webserver who's main mission is to execute tests with some framework. I looks for a test framework, files to feed it and how to report the results. Here's a typical karma.conf.js file lying somewhere in a client-side tested project. In Umbraco it's Umbraco.Web.UI.Client\test\config\karma.conf.js. I like to just keep it at root and set Build Option to None so it's not included when I publish.
module.exports = function(config) {
    config.set({
        files: [
            "umbraco/lib/angular/1.1.5/angular.js",
            "umbraco/lib/angular/1.1.5/angular-mocks.js",
            "umbraco/lib/jquery/jquery.min.js",
            "App_Plugins/MyPlugin/**/*.module.js",
            "App_Plugins/MyPlugin/**/*.js",
            "App_Plugins/MyPlugin/**/*.html"
        ],
        preprocessors: {
            "App_Plugins/MyPlugin/**/*.html": ["ng-html2js"]
        },
        frameworks: [
            "jasmine"
        ],
        reporters: [
            "progress",
            "kjhtml"
        ],
        browsers: [
            "Chrome"
        ],
        client: {
            clearContext: false
        },
        ngHtml2JsPreprocessor: {
            prependPrefix: "/",
            moduleName: "my.plugin"
        }
    });
}
The first part is a list of files to serve with the webserver. You'll typically start with the least dependent on top, and end up with the details of your implementation near the bottom. Remember to include your module declarations before your implementations.
Second we'll process all HTML for the views. This has been mitigated in newer versions of Angular, but for our purposes we need to "fool" angular with generated JavaScripts containing our views. Including "ng-html2js" makes this process automagic, so we really don't have to mind. This is due to some quirks with making the Angular $http service testable.
Further, with frameworks and reporters, we tell Karma to use the Jasmine framework, and report the test results using progress and kjhtml. Progress shows an increasing amount of red or green dots - one per test - while running your suite. KJHTML writes the test names and results to the browser as well (highly useful).
Finally we say we want the result in Chrome and that the results should be kept around (clearContext: false). If you leave clearContext to the default true the test-results will be gone as soon as the testrun completes.
The final bit is just boilerplate config for the HTML-to-JS conversion. Remember to set your  module name.

Bringing the ContentEditor Controller under test

For the content templates, I didn't add any GUI tests. The existing setup doesn't include the "ng-html2js" module, so we'll ignore that for now. What I did test extensively was the logic I added to the existing Umbraco.Editors.Content.CreateController. It's the Angular controller controlling the dialog you see when you create new content in the backoffice. We'd added some of the serverside logic, so it was time to flesh out the UX when creating content from templates.
Just to get going, I started verifying what was already there. The first thing that happens is that it shows allowed types like so:
To prove it, I had to look around a bit in the existing code. Turns out it retrieves and shows items from an allowedTypes field on the scope. And that's it:
function contentCreateController($scope, $routeParams, contentTypeResource, iconHelper) {
    contentTypeResource.getAllowedTypes($scope.currentNode.id).then(function(data) {
        $scope.allowedTypes = iconHelper.formatContentTypeIcons(data);
    });
}
Let's prove that with a test:
describe("create content dialog", function() {

    var allowedTypes = [
            { id: 1, alias: "x" },
            { id: 2, alias: "y" }
        ];

    it("shows available child document types for the given node", function() {
        expect(scope.allowedTypes).toBe(allowedTypes);
    });

});
A Jasmine test consists of a describe() call with a top-level description and a function in which several it() calls define the behavior. It's like a test fixture and a test in NUnit. To verify stuff we call expect() to create an assertion object we can  use to define expectations. In this case, we expect that an array of allowed types is present on the controller's scope.
Of course, we don't have a scope yet, and much less anything that initializes that field on it. If we run the test, it'll fail because scope.allowedTypes is undefined. To run the test, you'll double click "test:unit" in the Task Runner Explorer.
The Angular testing helpers (angular.mocks.js) comes with some functions to start up a subset of your Angular application for testing. The two main methods are module and inject. Coupled with Jasmine's beforeEach we can bring Umbraco under test and get a hold of the controller factory:
describe("create content dialog", function() {

    var scope = {},
        controller,
        allowedTypes = // omitted for readability

    beforeEach(module("umbraco"));

    beforeEach(inject(function($controller) {
        controller = $controller("Umbraco.Editors.Content.CreateController");
    }));

    // omitted for readability

});
The module call will load all of the services, controllers, directives and whatnot from the "umbraco" module. At least the ones pointed to from karma.conf.js. It's the testing equivalent to adding ng-app="umbraco" in an HTML page.
The inject call is a special call that will be analyzed for dependencies and fed Angular's internal services. We need the controller factory named $controller. It is named a bit misleading since it actually creates controllers. We can ask it for the one registered as "Umbraco.Editors.Content.CreateController". However, it won't do us much good yet. We saw that it depends on the contentTypeResource in Umbraco's resource services. It will be concrete, and it will try calling the WebApi endpoint for content on the Karma webserver. It isn't there!

Stubbing out the content resource

Fortunately and at the same time horrifying, JavaScript is dynamically typed. There are no casting exceptions, no nothing. We can pass anything to anything. As long as we're dilligent with tests, there's nothing wrong with that. It's actually a benefit. In C# we'd have to write full fakes or dig out some scary mocking framework. JavaScript IS one big stubbing framework.
To get rid of the contentTypeResource dependency, we can merely create one. It doesn't have to have al the functions the original has, just the ones we use in the controller. Normally we'd just initialize it in the describe scope, but we need to return an Angular promise from the getAllowedTypes function. In order to create a promise, we need the $q service from Angular. If you're sharp you remember we can have Angular services handed to us in the inject function, so we extend it with another parameter we store in the outer scope:
    var scope = {},
        q,
        // rest omitted for readability

    beforeEach(inject(function($controller, $q) {
        q = $q;
        controller = $controller("Umbraco.Editors.Content.CreateController");
    }));

    // ...
Now we can easily create a fake resource. When it's ready, we can pass the $controller function a map of dependencies to use instead of the ones it would inject in production.
    var scope = {},
        q,
        contentTypeResource
        // rest omitted for readability

    beforeEach(inject(function($controller, $q) {
        q = $q;
        contentTypeResource = {
          getAllowedTypes: function() {
            var def = $q.defer();
            def.resolve(allowedTypes);
            return def.promise;
          }
        };
        controller = $controller("Umbraco.Editors.Content.CreateController", {
            contentTypeResource: contentTypeResource
        });
    }));
Finally to get something on our scope, we need to create one from Angular's  $rootScope instance and pass that to $controller as well:
    beforeEach(inject(function($controller, $q, $rootScope) {
        // ...
        scope = $rootScope.$new();
        scope.currentNode = { id: 1234 };
        controller = $controller("Umbraco.Editors.Content.CreateController", {
            $scope: scope,
            contentTypeResource: contentTypeResource
        });
    });
Test test now passes! The resource it calls calls the function we created. The iconHelper.formatContentTypeIcons is real code running over our fake result, but it doesn't do any IO, so it's fine in a unit test.
If you're wondering how I figured out I needed to set scope.currentNode with a map and an ID, it's all right. I debugged! I dug around at runtime until I found out "how they do it". Make sure to enable debug in web.config so you get non-minified scripts to read.

Adding our features

If there were more untested code in the controller, I'd keep adding tests proving the existing untested code before venturing further. I'll be completely confident I won't break the existing behavior when I add my code. It's not entirely true - we can't test the stuff that's initiated from the view when not testing through GUI. We can only emulate it. But at least we're safer than when just hacking away.
Speaking of views, with that slim controller, how is content creation actually initiated? Turns out it's a simple hashbang link in the view:
    ...
    <a href="#content/content/edit/{{currentNode.id}}?doctype={{docType.alias}}&create=true" ng-click="nav.hideNavigation()">
    ...
I want to be able to show another dialogue if there are blueprints, so it can't be hardcoded like that. We could've interpolated the whole href attribute, but I thought it better to extend the click event. The whole URL goes into a function on the controller instead and the ng-click attribute is modified to match:
    ...
    <a ng-click="goToAction(docType)">...</a>
    ...
The controller:
    $scope.goToAction = function(docType) {
        $location.path( "/content/edit/" +
        $scope.currentNode.id +
        "?doctype=" +
        docType.alias +
        "&create=true"
        );
        navigationService.hideNavigation();
    }
But wait! How can we test that the browser location was already set without redirecting the whole testing routine? After all, it's running in a fake browser (PhantomJS). We do it the same way we got rid of the contentTypeResource. We fake it. However, we have another way of spying on, and prohibiting the actual $location service in Angular.
The really awake readers out there will have arrested me for not doing TDD right now. I'm writing production code before a test! It's all right. Sometimes you'll have to pop out to the browser and just hack for a while. As long as you keep it short, then hurry back and  harness the beast in a test. Amongst other things like "hacking" it's known as spiking and stabilizing.

Jasmine spies

Back to spying on the $location service. As before, we need to get hold of it from the inject call during startup:
    var // ...
        location;
    
    beforeEach(inject(function ($controller, $q, $rootScope, $location) {
        // ...
        location = $location;
        // ...
    }
Now we can employ Jasmine's spyOn method to verify that $location.path() was called. The assertion object has a special method for spies called toHaveBeenCalledWith(). We can use that after spying on path:
    it("creates content directly when there are no blueprints", function() {
        spyOn(location, "path");

        scope.goToAction(allowedTypes[0]);

        expect(location.path).toHaveBeenCalledWith("/content/content/edit/1234?doctype=x&create=true");
    });
Alas, this doesn't pass. Turns out location.path was "only" called with the absolute URL. There's some special magic voodoo going on that I can't explain. The fact of the matter is that the querystring was sent in there, but it can only be traced on a searcher that's been returned by location.path. When spying on functions, we can also instruct Jasmine not to do anything and return something of our choice instead. We can add a searcher to spy on and have it used as such:
    var searcher = { search: function() {} };
    spyOn(location, "path").and.returnValue(searcher);
    spyOn(searcher, "search");
Then we modify the existing expect add an extra assertion on the searcher:
    expect(location.path).toHaveBeenCalledWith("/content/content/edit/1234");
    expect(searcher.search).toHaveBeenCalledWith("doctype=x&create=true");
The test passes. So we have successfully refactored the existing code to be testable.

What about blueprints

Now we're ready to add some functionality. When there are content templates available we want to see a list of those instead of creating the content. Let's add a test. While I'd been building my safety net, Shannon had extended the server side content resource API to expose blueprints as a dictionary on allowed content types. I could extend my fake allowedTypes with some templates:
    var scope,
        allowedTypes = [
          { id: 1, alias: "x" },
          { id: 2, alias: "y", blueprints: { "1": "a", "2": "b" } }
        ],
        // ...
So if we select allowedTypes[1] we should see a selection of templates instead of going to the create page. So what should we verify? I added another div to the create dialog and added a couple of simple ng-show="[boolean]" directives. It seemed the easiest thing to do. The new div needed to show a list of blueprints, so I added an iteration over docType.blueprints.
    <ul class="umb-actions umb-actions-child" ng-show="selectContentType">
      ...
      <li ng-repeat="docType in allowedTypes | orderBy:'name':false">
        <a ng-click="goToAction(docType)">
          ...

    <ul class="umb-actions umb-actions-child" ng-show="selectBlueprint">
      <li ng-repeat="(key, value) in docType.blueprints | orderBy:'name':false">
        ...
And there in the view is now what I need to assert:
    it("shows list of blueprints when there are some", function() {
        scope.goToAction(allowedTypes[1]);
        expect(scope.selectContentType).toBe(false);
        expect(scope.selectBlueprint).toBe(true);
        expect(scope.docType).toBe(allowedTypes[1]);
    });
To have the test pass, we just add the required code to the controller:
    function contentCreateController($scope,
        $routeParams,
        contentTypeResource,
        iconHelper,
        $location,
        navigationService) {

        contentTypeResource.getAllowedTypes($scope.currentNode.id).then(function (data) {
            $scope.allowedTypes = iconHelper.formatContentTypeIcons(data);
        });

        $scope.selectContentType = true;
        $scope.selectBlueprint = false;
        $scope.docType = {};

        $scope.goToAction = function(docType) {
            if (docType.blueprints && docType.blueprints.length > 0) {
                $scope.selectContentType = false;
                $scope.selectBlueprint = true;
                $scope.docType = docType;
            } else {
                $location.path( "/content/edit/" + 
                    $scope.currentNode.id +
                    "?doctype=" +
                    docType.alias +
                    "&create=true"
                );
                navigationService.hideNavigation();
            }
        }

    }
It passes, and we're off to testing that it passes more parameters when creating content from templates.

The outcome

I'm sure you're getting the hang of it now, so to avoid ruining your workday I'll cut it short. From here there was more features, more refactoring, better names and whatnot. The final tests show a nice little summary of what happens when you create content templates:
  • create content dialog
    • shows available child document types for the given node
    • creates content directly when there are no blueprints
    • shows list of blueprints when there are some
    • creates from blueprint when selected
    • skips selection and creates first blueprint when configured to
    • allows blank to be selected
    • creates blank when selected
    • hides blank when configured to
What's that? Configuration? Yes, it's not very well documented yet, but here's a couple of hidden features:

Configuring content templates

Say you have a content type where you've prepared a template and want your editors to use only that template. You can add a JavaScript file to a package.manifest of your own and override the content template UI configuration as such:
angular.module("umbraco").value("blueprintConfig", {
    skipSelect: true,
    allowBlank: true
});
The skipSelect value is false by default. When set to true content template selection will be skipped if there is only one content template available.
Another case is when you have several templates, and you don't want to allow your editors to create documents from scratch at all. You already noticed the allowBlank value above, so I'm sure you've deducted that the next example will be:
angular.module("umbraco").value("blueprintConfig", {
    skipSelect: true,
    allowBlank: false
});
With which editors will either just get content based on the only content template for a type, or choose between the content types available, but not from scratch.
Now how about if you want to make this independent per document type? 
Well, you'll have to say so on the issue tracker for now, and there is an open PR with the following options:
angular.module("umbraco").value("blueprintConfig", {
    skipSelect: true,
    allowBlank: false,
    contentTypes: {
        "landingPage": {
            skipSelect: false,
            allowBlank: true
        }
    }
});
In which case the defaults will be overridden if the content type happen to be of the document type "landingPage".
If you're hungry for more JavaScript testing, I've open sourced the workshop material from Umbraco UK Festival 2017. Give it a shot, otherwise look for another workshop at some upcoming conference.
And with that, I wish you and all yours a very happy holiday!

Lars-Erik Aabech

Lars-Erik is on Twitter as