Why ETL is Not Enough when Delivering a Data Migration

If you’re working with data, data migration is most likely in your near future. Organizations are continuously looking to optimize their investments, and that typically means upgrading systems, moving to the cloud, or harmonizing and cleaning data.  

What’s seen as a time-consuming, costly endeavor can actually be turned into a valuable opportunity. Too often, data migrations become a drain on resources as a result of relying on an oversimplified “lift and shift” approach. This is because Extract, Transform, and Load (ETL) tools only account for three out of the eight steps necessary for an effective data migration and are simply not good enough to deliver an effective data migration. An advanced data migration (ADM) methodology must look well beyond ETL  - not just to be successful, but to actually accelerate long-term value. 

What is the goal of a data migration? 

Let’s start by getting to the heart of the “why.” What are we typically looking to do when performing a data migration? 

1. Optimize spend on software and services to reduce total spend
2. Create a minimized risk profile so the project is delivered on time and on budget 
3. Deliver value during and post-migration with what we learn about the systems, rules, and knowledge that’s captured as part of that migration effort 

What happens when Advanced Data Migration is taken out of the project?  

For the customer, removing ADM increases risk and services spend while lowering total business value. Services spend increases because there’s a lot of automation that is lost without ADM.  

The risk profile also increases because of a lack of visibility. There are a lot of tasks that you must now complete using spreadsheets, emails, and meetings, versus a methodology-based, workflow-driven process. While the “pre-migration” value (delivering the migration on-time and on-budget) hasn’t changed, the amount of post-migration value has significantly decreased as a lot of what we can reuse for ongoing data management after the migration will be lost.  

As that risk increases, the things that we didn’t know or couldn’t predict get so large that it tips the scales. You’re going to spend even more on services because project timelines will explode. Standing costs increase because other teams can’t do their jobs while other tasks are still underway. The risk of complete failure of the migration as well as complete loss of the business value is exponentiated.  

Why is that? 

After nearly 30 years of business, we can point to three reasons why an ETL-only migration causes failure and loss of business benefits: 

1. Poorly Written ETL Code vs. Autogenerated Code: Autogenerated code based on standards & best practices makes it easier to maintain, easier to debug, and ensures a consistent approach across the migration team. It also saves time and effort that could be better utilized on other migration activities. 
2. Lack of Project Visibility and No Single “Version of the Truth”: Without ADM, you can’t consolidate critical rules, mappings, design decisions, audit logs, and assigned tasks in a single system for one version of the truth. Complete tracking and visibility make it easy to quickly make course corrections or change scope.
3. Lost Business benefits: Missed opportunities to remove legacy systems, improve data quality, ensure proper business process execution, and plan for business-as-usual data operations post-migration, causes your new environment to atrophy more quickly, increasing data management costs in the long run.

1. Poorly Written ETL Code vs. Autogenerated Code.

One reason data migrations don’t deliver on value is poorly written code. Think of it this way - an ETL tool is a programming tool. For someone with less programming experience, it’s easy to mistakenly create a lot of really bad code.  

The migration engine we use at Syniti autogenerates good code based on standard templates and best practices, making it easier to maintain, debug, and deploy. This autogenerated code takes out the complexity scale required to use the tool on its own.  

Look at the example below of an actual data services job. What would be more useful after a migration? The clean, intuitive approach on the right is obviously going to result in reusable rules for ongoing data quality and data management activities post-migration. 

2. Lack of Project Visibility 

Without ADM, we lose all ability to automate and capture things like mapping, adding data that’s missing from our data sets, or documenting rules that need to be created. 

What’s left? Most often, it’s Excel. We've seen customers trying to document really complex migrations with literally 5,000 or more spreadsheets. With spreadsheets that get emailed around and updated, there is no change control process. Users are left wondering if they have the current version or if what’s written in the code actually reflects the rules in the spreadsheet. It’s nearly impossible to determine. 

In this manner, there is no one version of the truth nor project control for the work being done to deliver the migration.  

With ADM, there is complete tracking and visibility for every rule, every step, and everything consultants are doing in the tool. When things come up, you’re able to quickly make course corrections, put more resources where they’re needed, and if needed, change the scope of certain deliverables - all while making sure that the business goals are achieved.  

Due to this level of project predictability, Syniti customers consistently experience a positive ROI in under 12 months. What’s more, they have confidence that their migration projects will be done right the first time, on time.  

3. Lost Business Benefit 

Whether you’re moving to the cloud or implementing S/4HANA, you can drastically decrease the level of investments needed by right-sizing your migration data payload. Why move all your data if you only need a subset of that to run? Thinking about right-sizing your data ahead of time helps to decrease the costs and risks associated with that deployment.  

Not doing so results in a lot of missed, valuable opportunities. For example, archiving off that data in a way that is approved by your auditors. You may be able to decommission legacy systems and remove extra operational costs from the environment that IT is responsible for.  

Deduplication of data and making sure you have streamlined datasets is another opportunity missed with ETL. In the case of S/4HANA, for example, you need to understand how to do business partner harmonization. If you’re going to merge all your customers and suppliers and vendors into one system, you want to make sure there are no duplications, and it's simplified and streamlined to run the business. This will make it much easier to manage that critical master data even after the migration. 

Improvements in master data and transaction data quality is something we focus on a lot at Syniti in order to execute business processes flawlessly. When delivering a migration, we hit 99.9% or better data quality levels, versus what is typically achieved with a stand-alone data quality tool. Poor Data Quality impacts a customer's investment in their new system and the expected business benefits.  

Conclusion 

If you’re going to spend the money on data migrations, don't waste valuable dollars on ETL software that on its own, is incapable of ensuring success and maximizing the value for business-as-usual operations. 

Syniti Knowledge Platform streamlines every step in the data migration process with solutions like data integration and replication, data harmonization and matching, and data quality.

The recommended solution of choice for all RISE with SAP programs, SAP Advanced Data Migration by Syniti has a proven history of reducing migration costs by 38% on average. SAP Advanced Data Migration optimizes software and services spend with business value pre- and post-migration.  

Don't flush investments in technology, infrastructure, and resources down the toilet. Let’s use it, let’s capture it, and let’s make sure the next project or next migration is even cheaper, faster and delivers more value.  

Kickstart your data migration process with the enterprise-grade software that keeps that value going even after the migration is over. Ask about how Syniti’s compendium of features and solutions can be purpose-fit for your specific needs today.