I work as a QA at Knowit Experience (Formerly Creuna) and the first time I saw Umbraco was 2 years ago when my colleague Dennis Adolfi introduced me to it, telling me that it was the greatest thing since sliced bread 😉!  I have been through other CMSs like Magento, Wordpress and Episerver, so Umbraco was a welcome addition. It was a rocky start, but after a while it grew on me more and more.

As a QA I am the one responsible for making sure the quality of the developed feature fits the bill. Since Umbraco is the friendly CMS, I always try to put some attention to the CMS itself as well, not just the lovely stuff visitors see on the site.

The CMS in general has gone from being “the tech persons domain” to include everyone. All parts of the CMS should be easy to understand and configure, which brings me as a QA to spend more time going through the CMS and file bugs there as well. Those CMS bugs which was a very low priority before, has been upgraded to now be a high priority. Now a CMS with workaround solutions can be very time consuming or even destructive for the editor experience.

I think the expression “Easy to learn, a lifetime to master” is very fitting for Umbraco. It is easy to get started with the basics, but as soon as you get those admin rights and the new tabs in the CMS appear, you've got a totally new world to explore.

With this flexible product, my responsibility as a QA is also to promote that writing a story with proper requirements and acceptance criteria is even more important. If you put a little more effort into the creation of the issue (user story, bug, epic, etc); the easier it will be for the developer and QA to understand what it is about. This makes the coding and testing less of a guessing game (I hate guessing) and easier to understand. Another plus is that if you write the issue in a constructive way, other people in the project will understand as well.

So what does a good user story look like?

• First off, a good title which sums up the content of the story is nice.
• If you write the story in the Given When Then format, developers can easily adapt this into unit tests.
• The more effort you spend on acceptance criteria, the easier it will be to develop and test.
• Don’t forget to add parts of the feature which is out of scope, especially if design sketch is included in the story.

So, an example of a poor user story:

Title: Block for a form
Story: Users should be able to write things on the page
Acceptance Criteria:

• A block
• Fields to write in
• Button to send

---

An example of a good user story:

Title: Form block for the article page

Story:
(Given) As a user,
(When) I want to be able to see a form on the page,
(Then) So that I can fill in the fields and submit my form.

Acceptance Criteria:

1. The form should be wrapped in a block
2. There should be a title
3. There should be a preamble
4. The form should be in two columns
5. When submitting, there should be a message informing the user the form has been sent
6. The form should contain the following fields:
   1. Name (Text field)
   2. Last name (Text field)
   3. Email (email field)
   4. Phone (Text field, only numbers 0-9 and +)
   5. Comments (Text Area)
   6. Choice (Radio buttons)
   7. Submit button
7. Validation errors:
   1. Red border around the field
   2. Label which explains what the error was
   3. User should be focused on the first error

Out of scope:

• The option to send the form again.

 

Which story would you like to work on? So, I ask you people, how often do you develop something that you don’t know the value or purpose of? If you answer once or more, there is room for improvement.

With some effort, both the developer and the tester will understand who the feature is for, why it's being developed and what purpose it serves. As a tester, this gives me the understanding of how I should test it.
Is it for the editors? Then I need to think as an editor.
Is it for the visitors of the page? Then I need to think as a visitor.

How do I test then?

Umbraco is very flexible and the number of possibilities makes my job harder and more interesting, but here are a few of the points I consider when testing:

• Test boundary values (can I fill in “anything” in the fields?) in both the CMS and fields on the site.
• Can it handle special characters? (Since I am a Swede, those ÅÄÖ will be used…)
• Does the images or video work, will they play properly?
• Does the linking work, internally and externally?
• Do the translations work properly?
• Does the block render properly?
• Do you get proper error messages when something goes wrong?
• Do I get the correct responses when we've got interactions with other systems?
• And the more and more important one, does it work/look good in a mobile viewport?

Since I am a QA, I can't resist to return some feedback on the CMS as of now. I have limited myself to three "Wow!" and three "Whoa?", which is my equivalent of "Pros and Cons". 

Wow!

• The customization of the CMS UI, power to the editor.
• The shared blocks function, so easy to navigate around the blocks.
• The block edit view overlapping, no new navigation away from the current page when checking out a block on the page.

Whoa?

• Where is the current block being used? I tend to get nervous to delete or move things where I don't know where they are used.
• Translation tab can be a bit scary sometimes, hard to get a good overview if you not a "tech" person.
• Basic error messages for the editors, hard to understand what went wrong. Experienced this myself and heard it from clients.

 

To conclude:

Getting started with Umbraco can be an eye opener for someone used to other CMSs. It is great, but as with all new things, it takes time.

Please let the investigation, refinement of requirements and story writing take its time, we all will benefit from it. If we start with quality work right off the bat, the road to success will be less bumpy. If you work this way today, that is great. If you don't, I hope this post has boosted your morale to try to spend more time before coding and most important; ask questions and object before starting!