ETL/Excel may appear to be the path of least resistance on the surface, but there are plenty of risks with this approach to data migration.
Key Takeaways:
- ETL + Excel = High Risk: Traditional data migration using ETL tools and Excel often leads to disjointed teams, version control issues, and skyrocketing costs.
- Lack of Real-Time Visibility: Executives and IT leaders struggle to get real-time insights, increasing the risk of missed deadlines and loss of stakeholder confidence.
- Manual Coding Increases Errors: Limited automation in ETL/Excel approaches leads to inconsistent results, higher failure rates, and longer testing cycles.
- Moving Poor Quality Data: Simply moving data doesn’t guarantee success—poorly harmonized data slows down time-to-value and increases storage costs.
- No Knowledge Reuse: ETL/Excel doesn’t capture transformation logic for future use, making each new data project slower and more expensive.
- Data Security is at Risk: Emailing spreadsheets around creates serious data security vulnerabilities that can lead to financial and reputational damage.
In building your approach to digital transformation, one key topic always arises – how are we going to move the data? There have been many approaches over the years, and for as long as migrating data has been around, Excel has sat at the heart of it. As technology developed, the concept of ETL (Extract, Transform and Load) arrived to offer a smoother (on the surface at least) method of getting data out of a legacy system and onto the new platform; however, there’s always been a problem – an Excel/ DIY approach using solely ETL technology works if the source landscape or data construction is complex.
At Syniti, we’ve seen many consequences from ETL/Excel projects over the years and witnessed the costly ramifications that it brought businesses. The reason the majority of these enterprises got into trouble is that they failed to understand the risks of the approach. There are plenty of ETL tools available so we won’t go into the specifics of any one of them, but let's analyze the general risks of an ETL/Excel approach.
With an ETL-based approach to an enterprise-scale migration project, the data team, business people, and developers work independently while project managers work furiously in the background to hold things together. Excel is typically used to gather mapping requirements, collect missing data, and document transformation requirements that will be implemented in ETL tooling. With the potential for thousands of Excel spreadsheets being worked on and mailed around, version control can become a huge issue, as can testing and quality control. For most companies, the answer to this controlled chaos is to throw expensive human resources at the problem to try to wrestle it to completion, but this does not come without radically increased costs.
Bottom Line: Greater risks of a failed go-live and increased project costs
With so much individual work in a migration project, it’s a big challenge for program leaders to compile real-time status updates for themselves or for executive leadership. Without easy-to-access dashboards, fueled by real-time, activity-based insights into project status and timeline, project leaders can struggle to pivot projects based on the reality of the situation, which leads to increased risk. Executives too, are more likely to be more distrustful and concerned by the whole program's direction which can lead to a lack of resources or worse.
Bottom Line: Greater risk of not delivering on-time and on-expectation, and of the boss getting frustrated.
With thousands of data mapping and data harmonization challenges to address, developers and data stewards can quickly become overwhelmed. Relying on an ETL/Excel approach means limited or no automated code creation and a lack of no-code development capabilities. Additionally, ETL tools do not prevent the creation of poorly written ETL code. This significantly increases the need for bespoke modifications and long rounds of testing, increasing the chances of complications and delays.
Bottom Line: Increased need for custom code generation and more points of potential failure
Getting data “lifted and shifted” from the source to the target system isn’t the definitive success factor for a data migration. While a technical data migration is the fastest way to move from one system to another, if the data isn’t configured, harmonized, and mapped properly to the target system’s demands then the new system simply won’t be as functional as desired. Not “rightsizing” the data also means you are paying more for storage and the infrastructure required to maintain it. This lack of data quality and rightsizing focus in the ETL/Excel approach can cost a business valuable time and resources, limiting the effectiveness of the target system when it switches on.
Bottom Line: Risk of elongated time-to-value, unhappy users/ customers, expensive IT upkeep
While ETL and Excel (combined with a lot of manpower) may get a project over the line, it doesn’t naturally capture all the knowledge generated during the project and make it available for re-use in future data migrations and data projects. Organizations looking to conduct another data migration, data quality, master data management, or any other data initiative, will need to start from square one to rebuild any rules from the migration again from scratch.
Bottom Line: Your next project is going to cost more and take longer than it needs to
The ETL/Excel approach has serious flaws in regard to its security of data. Mailing spreadsheets around with personal, private and business data exposes a business to a number of deliberate hacks and accidental human errors. Without a central repository, securely protected by enterprise-grade firewall technology, it’s only a matter of time before a vulnerability in the ETL/Excel process causes significant financial and reputational consequences for the enterprise.
Bottom Line: Your data is vulnerable without a central data repository
ETL/ Excel may appear to be the simplest or the path of least resistance on the surface, but as we’ve shown here in this blog there are plenty of risks with taking this unautomated, unaligned approach to enterprise data migration. The good news is that ETL and Excel is not the only way to tackle data migration. If you’d like to learn more about how Syniti has successfully delivered thousands of successful enterprise migration projects, always complete with a BORING GO LIVE®
While delivering a frictionless data migration isn’t easy, with the right tooling, experience, and best practices, it is possible.
Trying to accelerate a data migration while ensuring success is a difficult task if you don’t have the expertise, technology, and laser focus on data...
When choosing to spend money on data migration, be sure to choose a solution that sets you up for success and streamlines every step in the process.