01 Jun Data Asset Migration – How do we move to the cloud?
Becoming a data-driven organisation requires a new approach towards migrating to modern cloud data platforms. With significant investments in existing legacy data platforms, many organisations struggle to integrate new platforms that provide greater scale and agility at a lower cost. These challenges are further exacerbated by a rapidly changing technology landscape and potential business disruption to end-users.
So why are migrations so difficult? For many organisations, the sheer multitude of legacy, on premise, native cloud and SaaS based systems presents many technical challenges for data engineering teams, particularly data ingestion, transformation and storage. To complicate matters, the major technology vendors have developed comprehensive and evolving data platforms, with a high degree of sophistication required to architect, design and implement these often discrete services into working solutions. Another dimension is the huge success and impact from the open source community around big data ingestion and processing engines.
There are several business drivers for migrating your organisation’s data assets to the cloud however we believe they can be distilled into a couple of key areas: (i) digital transformation; and (ii) cost reduction supported by simplification. We also see a common strategic problem for many organisations in determining the right approach required to manage and deliver a data asset migration program, including how to engage the right level of business, operational, finance and IT stakeholders. Migration approaches with strong themes of technology uplift and simplification to align with strategic cloud data platforms are prevalent. For large and complex data assets there is also a practical desire to transition off the legacy asset incrementally. However, because legacy data assets evolve over time there is often insufficient knowledge of the current state to understand how it can be optimally segmented for transition.
An up-front understanding of each migration iteration’s functional and technical scope is required, as well as knowledge of the key data and logic dependencies for each. To assist in this planning, we have developed “migration maps” to visualise small, manageable, end-to-end migration components, including critical dependencies. This analysis enables optimal prioritisation and sequencing of migration components. Ultimately it will enable a significant reduction in migration effort and cost as well increase the effectiveness of legacy decommissioning.
InfoCentric also has an overarching migration framework that supports accelerated target solution design and migration planning using automation and data discovery tools to identify ‘migration maps’. To optimise your data platform migration endeavours our framework also incorporates push button code generation, automated test case development, and an iterative approach to data reconciliation.