We have all seen the home improvement television shows where homeowners set out to renovate their house and a camera crew follows the renovation through its ups and downs. In all of these shows, there is always that one moment when an unexpected expense is found and the contractor has to tell the homeowner that they need to reroute plumbing, or move a wall or some other unintended cost that is standing between the homeowners and their desired outcome.
In my experience, the data migration track of any enterprise application implementation initiative can often feel like this type of unplanned but necessary expense in order to get the outcome the company wants in its new system. Hub and Templated based data migration approaches struggle to react to the changes that are inevitable in every migration project and are not able to provide value to the organization after the migration ends. In contrast, a metadata-led approach to data migration leads to faster decision making by all parties, faster and more natural engagement of the business and lays the foundation for higher order value of data post go-live.
What is the heart of the System?
There are two schools of thought as to what should live at the heart of a migration solution:
1. Templates that are designed to be applied to a hub and extended for the needs of the initiative
2. Configuration driven from metadata that quickly adapts to the needs of the initiative
The template-led approach using technical hub-based tooling is typically based on an implementation style that tries to fill the gaps that exist when trying to take data tooling (such as ETL tools) and apply that tooling to the use case of data migration. These tools are powerful for the purpose of moving and massaging data, but typically fall down when applied to the unique business processes that occur in a data migration initiative. A data migration is a unique data activity in a company’s Data Journey that requires many varied personas executing a complex development lifecycle that, if not managed correctly, introduces unnecessary risk and time to a migration initiative. Think about the amount of hops it takes using a hub-based approach with templates to get from business user to business analyst to consultant to developer to QA team member back to the business analyst just to get one requirement fulfilled.
In contrast, a metadata-driven approach allows the business users who are closest to the needs of the data and its use in business processes to set the requirements directly in the migration solution. When metadata is the core of the system, smart code auto-generation logic can be applied which minimizes the gap from requirements to delivery by up to 75%.
Ideally, a metadata-driven solution that is also accelerated by templates provides the best of both worlds. Make no mistake, what matters most is what is at the core of the solution. If that core is metadata, then there is a common framework built-in natively whose benefits cannot be duplicated no matter the depth of features in a technical hub nor the breadth of the template library. The organizations that will win in the digital economy will move too fast for templates built on hubs to keep up with the demands of change and collaboration digital transformation requires.
Responsiveness to Change
One of the constants on the over 3,500 data initiatives performed by our software is the constant changes that happen during all phases of these initiatives. It is because of this constant and high rate of change that we spend a lot of time and thought on how we can make our solution more responsive to change so that the delta between the change being discovered to implemented is as low as possible. After spending over 2 decades on this problem, we have centered on metadata as the conduit by which we can lessen the impacts of change and keep initiatives on track.
Working with metadata allows for a “configuration not coding” approach to data migrations. By using a configuration first approach – and allowing the smart machine to write and execute the code – the amount of people and steps involved in the process are much lower than with manual and template and hub style approaches. Metadata is the common language spoken by both the people and the machines in the process and by bringing that language to the forefront, those lines of communication are clear and put into a process such that any change can be absorbed and managed with minimal impact. By linking all of the stages of the migration, all of the transformations, all of the validations and all of the analytics to metadata first, a common foundation is laid which can withstand shifting winds of change that blow all around it.
Let’s look at a real life example: Improved analytics and insights. A common change that happens in a migration is the introduction of new attributes or fields that the business will start to care about once live on the new system. One example in the world of procurement in spending limit. In a template driven approach with hubs, there would be a template that will typically take data from a data store and load that data into a spending limit field with some basic checks to be sure the value going in is a number. That is a good technical solution, but there is a lot left out in that scenario - like all of the business understanding and requirements. A metadata driven approach would allow for the business users to configure and debate in real time and in the language of the business, how the spending limit for suppliers should be calculated and use reference data and rules to drive the setting of the values so that the data is not just technically ready to load but also fit for purpose to drive the desired savings and economies of scale throughout the procurement process. The business can evolve, change their mind and do the natural work of debating the best solution without being bogged down by having to recode days of work changing and reimplementing templates within a hub base model.
Value After Go-Live
Beyond driving acceleration in the design and build phase of any migration initiative, a metadata centric approach leads to acceleration and less friction through the entire initiative. By its very nature, metadata is more easily understood by all people and personas involved in an initiative and is therefore the basis for almost all requirements and business process discussions when it comes to data. Conversations between people take place using metadata and in those conversations, it is high-value intellectual property for any business such as what the data means, to what business processes the data applies, who owns the data, where the data lives, how the data is supposed to be used and many other key areas of knowledge.
This knowledge is a key accelerant to a data migration initiative, providing the basis for auto-generation, report distribution and streamlined and clear communication. These same benefits, among many others, have value beyond a data migration initiative by accelerating any and all data initiatives that touch or reference the data that was migrated. By using a data migration solution that captures this knowledge and stores it so that it can be used for all data initiatives going forward takes a data migration beyond getting to the right go-live to having value well beyond the migration to the future of the business. Going to a new application, be it ERP, CRM, SRM or any other should not be thought about as the end of a journey but rather the beginning of a journey to higher value of data for the enterprise. By using knowledge, centered on metadata, as the bridge from migration to long term value organizations can use data migration from something that once was thought of as an unexpected cost to a strategic accelerator on an organization’s digital and cognitive journey.
Data is a complex thing and like every other part of your organization is dependent on a complex interrelationship with a variety of other entities to be fully leveraged. Download Gartner’s 2019 Magic Quadrant for Metadata Management Solutions and see how Syniti’s decades of experience has resulted in our recognition as a Visionary in this field.