Migrating a legacy application

Heads Up!

This article is several years old now, and much has happened since then, so please keep that in mind while reading it.

A Little Background

In this discussion, presented mostly chronologically with a few side-quests thrown in, I hope to take you through the journey with my clients of choosing a platform to base their web business; not just power their eCommerce website, but as a platform and a marketing tool to build on for the foreseeable future. The requirements they had, some of their goals, some of their doubts and fears leading eventually to the conclusion which is ….

… well you’ll just have to read it.

[TL:DR - we chose Umbraco]

Setting the Scene

My clients have run a successful trade and retail bar and restaurant supply business for many decades, originally starting as a small local supplier in 1977.  They launched their main website Drinkstuff.com in 1998 right in the middle of the dot-com bubble, enduring the bubble bursting and every hurdle that came since to the point where they are now a modestly sized pan-european supplier to retail and trade

Their software platform has had a less successful story (until relatively recently). Originally built by a series of third parties, each on top of the previous platform and growing “organically” but not in the most structured or sustainable way, whilst it serves the business well enough there were some pretty hefty limitations holding back their ambitions.

The basic architecture of their business is roughly sketched out below through the medium of 90s clip-art (this is NOT an accurate representation of the architecture, purely for illustrative purposes)  : 

The software, and largely the business, is split into two parts plus the customer facing website, all managed with a fairly monolitic system, described below.

The Website - This is the main point of interaction with customers for browsing and placing orders. Originally built on Classic ASP, despite helping get the business growth to where it is today it’s definitely showing its age.

The Back Office Systems - Here everything is managed using a multitude of in-house and a few third party tools including :-

  • A Retail ERP system for managing products, suppliers, customers, sales channels
  • A basic in-house CRM system for customer management of our trade customers
  • A Pricing system for managing our various channels, both in-house and third party
  • A custom in-house CMS for managing the product catalogue pages on the website
  • And a Wordpress blog

The Warehouse Systems - The heart of the business, where all stock is managed, moved, picked, packed, labels printed and dispatched, and where returns are handled, and a small amount of in-house production happens. Uses a combination of :

  • PCs with barcode scanners at packing stations
  • IPads for reporting and monitoring
  • Handheld barcode scanners running a custom built Xamarin App running on both iOs and Android
  • Thermal Label printers for courier postage labels
  • Industrial Laser printers connected to windows apps for printing invoices

The software written on a mix of technologies - some modern, some quite old. The documentation is sporadic at best and liberally distributed over a range of intranets, wikis and word docs on various network shares. Test coverage is … not quite where we would like it. Making any changes is not always an easy thing, automated testing is a challenge. Whole swathes of code refer to business functions which no longer exist, and with the exception of a few, most only understand how their specific business function should work - nobody had an overall. All of this to be done without losing any of the SEO we had built up on our old site - We had our work cut out.

The Plan

The business is growing too quickly to rely on old technologies, especially where the speed of change was massively hampered by it. So - we needed a new platform on which to move the business forward for the next few years, and hopefully decades. 

The business looked at several platforms over a period of several months, including : 

Magento : One of the biggest self hosted eCommerce platforms out there. Open source, built on php and with a large community and a large plugin support. It comes with a community edition as well as enterprise solutions, and a whole host of features including a fully featured CMS.

Shopify : A hosted platform, but one of the biggest  in the industry. Another theme-based, customisable platform with a built in CMS, but this time with a per transaction fee as well as a monthly fee.

Sitecore : One of the biggest .net CMS systems with a built in marketing platform, personalization engine and a host of features.

A plain old MVC build : completely custom, do what you want to, any way you like. However we were aware just how big a bite it would be building our own CMS and eCommerce application. There's a natural resistance to reinventing the wheel.

Umbraco as a platform : The CMS platform that’s versatile enough to do everything well, but doesn’t get in the way of building what you want

So Umbraco won - Woohoo! Here's why.

Magento and Shopify would be a wholesale rebuild on a new hosting platform as well as the eCommerce platform. There would also be a lot of additional work required to either work with the existing warehouse management systems we had in place, or a lot of work to replace these with other systems, a whole other journey we would have to go through.

And Sitecore …. Well nobody has time for that sort of license fees, plus the question of the warehouse still remains, albeit it’s less of a platform swap since Sitecore runs on .net, we could still connect the eCommerce and CMS from the Sitecore platform with the warehouse and back end systems.

A custom build would have worked too, but a lot more work - and given one of the reasons for choosing a mature platform was the extensibility with 3rd party community and paid packages, as well as the long term support.

But which version of Umbraco?

At the time of making this decision, the stable platform was version 7, the new kid in town version 8? Version 7 has a lot going for it - mature codebase, battle tested and well documented. The question of support came up though - how long would v7 remain supported

The  main concerns going with version 8: How stable was v8 as a platform for a production site? How long would v8 remain supported? The first point required some research, building a proof of concept site and many long discussions with the community, but on balance it was felt that it would probably be OK. We also had the actual build time for the project for any major issues to be ironed out. Umbraco being open source software, but there also being support contracts available if ultimately required helped with this.

In terms of the lifespan of v8 this is a little unknown - how long before it would be replaced? Looking at the past, v6 didn’t have a huge lifespan, being initially released in Jan 2013 but quickly replaced by version 7 in November 2013. Version 7 however, lasted until v8 was released in Feb 2019 - just over 5 years later. 

And the future?  The idea of ultimately moving to .net Core with project Unicore was also brought up as a positive, allowing greater hosting platform choices and support into the future. All good things as far as a platform choice is concerned. With v8 also being used extensively on cloud, as well as the community uptake and very active discussions, v8 was the right version for us to take.

Now the work began ...

The Solution (so far)

Since we were launching a replacement for an existing eCommerce site, with an existing Category tree, existing products, brands all with an existing management system we had to plan how we would transition from a custom application into Umbraco. Some of these were brought into umbraco and some remained separate

Categories and Brands

In the previous system categories were managed with hierarchies, and each maintained lists of products which were presented on the front end site. These were most simply imported into umbraco as we simply defined the document types and allowed them to be created in the right hierarchy. 

We need to keep a one-to-one link between the categories in Umbraco and the categories in the older system for one reason - products. The existing system, at current count, has over 28,000 individual products. These are already managed with a full back end which allows the creation of product variants, to set prices for the website, Amazon, Ebay, to manage the attributes etc.

One of the great things we can do is build a custom property editor, and by building one which allowed us to query the categories table from the old system and store it as an ID in the new system, we were able to map the categories in Umbraco to their counterparts in the old system. Sprinkle on a bit of server side validation to ensure each category was unique and we had everything we needed to render categories.

On top of this, we used Route Hijacking to allow us to augment the category controllers and add additional properties to the base model (using the excellent Models Builder) - this allows us to include products in the Category model. So for example for the category pages, our view model looked like this :

The Product class (green) came directly from the legacy database, and had no representation in Umbraco, the CategoryModel (Blue) was generated using modelsbuilder, and the CategoryViewModel (green), which inherited from CategoryModel extended it to include the Products in the category.

In the categoryModel, we used the custom property AdminCategory to store the ID of the category on the old system, which was used to then look up the products in that category.

Customers

Our existing customer database was managed in a separate database, but we wanted to import these into the umbraco members, and hook it up with the authentication with a custom membership provider. Having a functioning membership library built into the platform let us just build on it and use it pretty well, and whilst we will have to make a few changes to allow us to use Umbraco members in the rest of our system, this is a price worth paying.

Blog

The existing blog was built in wordpress - this had to go. We needed the blog to be an integrated part of the website in the umbraco back end, and we wanted to import all the wordpress content. In the past, we would have used a product like the excellent CMSImport from Richard Soeteman but as it wasn’t yet ported to v8, we came up with our own solution. 

We wrote a surface controller to create blog posts, and a console application to read wordpress blog posts from a wordpress export and using these two in a fairly short amount of time were able to import all the wordpress articles into the Umbraco blog and retain all the lovely SEO goodness. Win!

Additional features

One of the key features of the existing system is information dashboards. These are built for each team to specifically have key information to help them do their jobs. The number of useful reports we can use across our business is vast, and a mechanism to quickly and easily put these together is fantastic. Some examples (not even close to the full list) of reports we currently generate are

  • Number of products awaiting approval before being launched
  • Number of orders from each of the sales channels, and broken down by delivery service
  • Number of orders with issues raised by customers
  • Lists of supplier purchase orders and due-in dates for stock management
  • Gross Profit percentage per product
  • Total numbers of breakages in the warehouse
  • Average time to pick an order
  • Total numebr of picks per warehouse user

There were a vast number of these dashboards that were needed, and again Umbraco provides, with the ability to use custom dashboards into the back end. The article by poornima as well as the documentation from Umbraco provides some really useful tips and examples on how to do this. 

Conclusion points and Discussion

This journey has been painted like a fait accompli, but in reality it’s a work in progress, with the migration from a stand-alone custom application to well under way. Umbraco has allowed us to extend a completely stand-alone application with some awesome new features. The final solution for how the system works together looks a little like this.

Whilst this does look more complex, it’s actually separating the behemoth that we had before into the two key areas for the business - the Umbraco website and the back office and warehouse application - it’s all about separation of concerns.

The Umbraco website is customer facing and handles all the ecommerce application aspects including

  • Listing Products, categories, brands
  • Basket and checkout functionality
  • Customer registration, order history, account management

The back end application is responsible for a lot of the back office operations including

  • Inventory, product management
  • Order fulfilment (warehouse dispatch operations)
  • Stock management
  • Supplier management and ordering
  • Reporting

There is obviously inter-application communication going on, since the Umbraco site doesn’t maintain knowledge of the product database, or stock information, just as the back end application has no knowledge of the blog, or any of the other front end content.

Umbraco here has allowed us to do several things in our business - it’s kept the core back-end operations separate from the presentation and sales portion of the business. Each is catered to being the best it can be. We get to benefit from all the awesome CMS features of Umbraco to build an eCommerce application and slot it neatly into our business. 

And we get lots of bonuses, like Forms, URL tracking, media management - all out of the box, and we haven’t even talked about some of the cool things we can do with Content Applications. We plan to document this journey in a blog series as we complete more of this build, and sometime next year will launch this. Follow us on social media for more info on @drinkstuff

One thing I am really hoping for is some discussion around what we’re doing, and I would love to compare what other people are doing when integrating applications into Umbraco, especially eCommerce. Please drop me a line with your thoughts - I genuinely appreciate them.

Thanks to :

  • Poornima - for her assistance in writing this article, and for her excellent article on dashboards
  • The excellent Umbraco 8 training courses which brought our team up to speed with the v8 changes - particularly Jeavon and Gabriel
  • The umbraco community at large for all the excellent blog articles and discussions which helped us in more ways than we can count (and continue to rely on)

 

Carl Sargunar

Carl is on Twitter as