How organisations avoid the pitfalls of data migration | Exonar

Written by Anna Cotton | Mar 30, 2021 3:02:00 PM

How organisations avoid the pitfalls of data migration in 2021

5 key considerations to ensure smooth sailing as you journey to the cloud

Report Methodology

The insights shared within this report were acquired during a virtual roundtable discussion with a select group of IT, data, security and change management specialists. The event was hosted by Exonar and chaired by John Harris, CTO at Global Talent. Also present at the event were:

Present at the event were:

Enterprise Data Manager in the energy industry Enterprise Architect in a financial services firm
Business Change Manager for a critical national infrastructure organisation
IT Project Manager also in critical national infrastructure
Chief Data Officer at a data company Lead Architect at an automotive company
Information Security Risk Analyst at an online retailer
Data Consultant at a professional services company

We all know the cloud is the 'holy grail'

85% of businesses worldwide are making use of cloud technology to store information

For years we’ve been bombarded by stories about how much better life is in the cloud. Once we’re in the cloud we’ll be nimble and able to spin things up quickly, take advantage with pay-as-you-go models, have the capability to analyse data in a thousand different ways and at the speed of light. Yep, in the cloud, everything is perfect...

Except it’s not.

The reality is that 1 in 3 cloud migrations fail completely, with the remaining 2 in 3 experiencing ‘moderate’ improvements .

“The future is framed by the view that the grass is always greener, so cloud is seen as Nirvana. We think all our problems will magically go away but they really don’t.”

Enterprise Data Manager

The journey always starts well...

Let’s acknowledge that cloud migration is a journey. And like all journeys there are several paths to get there (some more challenging than others) so it’s about how you navigate the path. Fortunately, many businesses have embarked on this road before. By looking at their maps, we can (hopefully!) avoid the potholes they’ve hit along the way and avoid the same fate.

A Lead Architect explained how his company has three major divisions, each at different stages of cloud maturity,

“Some bits of the business are as far along as doing infrastructure-as-code, and we recently completed a SAP migration, which was very successful. But our reality is that the large majority of our data is still on-premise.”

So what should you be considering to ensure your migration is smooth sailing?

1. Solve the data black holes

10-years’ ago, unless you were specifically asked about data, it wasn’t touched. We didn’t have strict legislation dictating what we did with our data, we owned our data centres and so weren’t burning costs storing it, and analytics wasn’t really a thing, so data didn’t matter too much. But now we’re in 2021, where everything in our world is digital and connected.

Every business has unknown data that’s been accumulated over several years. But you can’t ignore it anymore. From a compliance perspective alone, the regulator won’t look favourably upon you if you simply ‘lift and shift’ that mass of unknown data from your on-premise servers up to the cloud. In their eyes, you’ve made a deliberate decision to move that data, which means you must have a reason to retain it, and implemented appropriate controls to secure it.

The Enterprise Data Manager commented,

“We all make lots of assumptions about the volume of data we hold and the sensitive material contained within it, but it’s just that, an assumption. There’s nothing to underpin those assumptions, which is something we’re now looking into.”

So where should you start?

While our panel all had confidence in their structured data sets, it was the unstructured data and organically grown architectures that caused them the most concern. As the Lead Architect explained:

“Our biggest struggle is understanding the unknown apps within the business. We don’t know what a lot of them are doing. And because they’re unknown we can’t see how they all link together – you could pull on one thread and suddenly find yourself needing to move another bit, and then another. And then there’s the terrabytes of data sitting behind those unknown apps – we need a holistic view so can make an informed strategy about what to do with it.”

Our IT Project Manager empathised:

“Our problem is the file shares, which we know contain lots of unstructured data. If we pick it all up and just move it to the cloud, we know those same problems with information security would be waiting for us after the migration.”

But our Enterprise Data Manager has discovered a strategic way to tackle to data deluge:

“Interestingly we’ve not extracted our significant and sensitive data yet. Right now, it’s only the stuff we need to do the bulk of the processing that has been migrated to the cloud. Over time, we know this will grow, but only as that environment gets hardened and we have the opportunity sift through the rest of the legacy data.”

Things to think about:

Cloud is unlikely to ever be your ‘Nirvana’. Data is an ever-evolving picture that you need to keep up with it as best you can.
Have robust retention policies that tell you exactly what to do with data to overcome the inertia, confusion (and reluctance) to delete data.

2. Use tools to democratise data

One of the biggest barriers to overcome with cloud migration is all in the mind – people know they have a lot of data to deal with and don’t know where to start, it all feels overwhelming so they kick the project down the track for as long as possible.

Our Information Security Risk Analyst explained her situation:

“We’ve had conversations about what data we should move and what to leave behind, but we tend to describe it in terms of risk. The rest of the business then thinks it’s too hard to tackle so they just say we’re not doing it. Which is where I struggle – do I just drop the conversation? I can’t, we have a responsibility to that data. So do I say we’ll try to clean the data first? But then how do you start that process? And whose responsibility is it?”

The simple truth is that the more an organisation understands about its data, the easier it is to remove any risks associated with migrating it to the cloud.

Our IT Project Manager feels he’s found a possible first step to achieving this single view of his organisational data:

“We’re currently running a proof-of-concept to try to get the unstructured data into a structured state. If we can prove to the business that it’s possible and that it’s not a costly exercise, it should make our data conversations a lot easier, and ultimately the migration smoother.”

Exonar’s Chief Revenue Officer, Tony Buck, shared some insight into how Exonar Reveal has developed since the company was founded:

“Exonar’s starting point was to help businesses identify PII. But very quickly we recognised that more conversations were about security and privacy. Then over the last year, the conversation has changed towards cloud migration – partly driven by what’s happened over the last 12-months.

“The biggest benefit of data discovery technology, is that when businesses get that index of their structured and unstructured data, it’s a good way of understanding holistically what’s in the information. We never designed Exonar as a cloud migration tool, but the visibility and the democratisation it provides, means people can understand what’s in the data so they can make informed decisions based on it.”

Things to think about:

Build a compelling business case based on the conversations you have with the business about what they expect to find in that legacy data, and what would be difficult for them to find.
Run a pilot project using real use cases to help the business understand how the tools enable them to make cleansing data a less onerous task.

3. Secure key stakeholders

Within every business you’ll find champions who will help sponsor your data initiatives into the boardroom. Identifying who these people are and getting them to focus on what matters most will fast-track your success.

Our Business Change Manager explained how she is currently working to acquire sponsorship for her data initiatives:

“I’ve engaged a couple of people from each area of the business and over the last couple of months have tasked them with going through their data to understand what’s in there. The aim is to build a depth of knowledge about what we have and try to formalise that at a high-level. Ideally, I want to build a process for how we could use tools to help clean up our data.”

Our Enterprise Data Manager had a slightly different strategy for securing senior buy-in, instead, focusing on how he positioned the conversation with the C-suite:

“I push the view that our world exists within 2 hemispheres: ‘the fragile world’, where we keep our inventory, sensitive information, plans and procedures, and ‘the agile world’, where we have the systems and platforms that enable us to move at scale and pace. In our sector the transformation agenda is all about the need for certainty, so my focus is always on the fragile world and how to ensure that environment is protected, structured and hardened.”

In the experience of the Data Consultant, he shared how securing sponsorship for cloud migration projects is a lot easier for newer companies because they don’t have to deal with legacy data:

“Well-established companies aren’t set up in the right way to take advantage of cloud. There’s a lot of tech debt there and they need to go through an extensive data cleansing exercise before they can even begin to think about advanced analytics. Whereas greenfield companies can get to advanced analytics almost immediately.”

Things to think about:

Identify who key stakeholders are within the business that will support your initiatives in the boardroom.
Typically, data as a risk will always trump data as an asset, so think about how you frame your conversations.

4. View data as an opportunity and a risk

Most organisations inherently view data as a risk. And when it comes to cloud migration, the looming figure of the regulator and the potential fines imposed for simply lifting and shifting that unknown data is enough to keep most people awake at night.

But we can’t ignore the fact that once that data is visible, its value can be distilled into precious insights to power the business.

As our Business Change Manager shared:

“As a business we are risk-averse, but we know we need to focus on the opportunity. My hope is that if we can use the data leaders in our business, they will help pull that enthusiasm through.”

While our Information Security Risk Analyst, whose organisation works with consumer data, shared how viewing data as an asset can actually make cloud migration a lot easier:

“For us data is absolutely about knowing customers, which is why from a BI perspective, we know exactly where all that data is. Naturally, a lot of that information lives in the cloud because that’s where it’s needed. It’s very simple to make the business case.”

Things to think about:

Do you have a mechanism in place to ensure your retention policy is followed and nudge people into doing the right thing with data?
List your compelling use cases for data as an asset – imagine you had visibility of all your data, what would you do?

5. Prevent re-polluting data

We’re in this mess because 10-years’ ago we weren’t staying on top of our data. Today, it’s estimated that unstructured data accounts for about 90% of the digital universe, and it’s growing at a rate of 65% per year. Just imagine what your business would look like if you failed to stay on top of it now...

Cleansing your legacy data will take time and effort. And once you’ve taken appropriate remediations to either move data, or delete it at scale, you need to establish robust mechanisms to stop bad habits creeping back in.

Our Enterprise Architect explained how his business has two parts, one focused on investments, which is heavily regulated so data is managed in structured apps, and the other focused heavily on PII relating to its members:

“We know we need to do something with our data. We have policies but our problem is that no one really implements them. Therefore, we’re looking at ways to make things easier for people, like how we could automatically classify data so it becomes easier to manage.”

Our Chief Data Officer is a cloud veteran having completed multiple migrations over the years. He was happy to share his wisdom with the panel:

“Cloud is a journey I’ve been on several times and I’m always coming at migration projects from a data perspective and how we can leverage data for business intelligence purposes. Using cloud in this space is game- changing so you need to stay focused on how to share and migrate data so the business can leverage BI tech. Get it right and you build a real-time data pipeline that you can leverage for data science and machine learning.”

Things to think about:

What will you do to ensure cleansing data remains an ongoing priority rather than a one-time exercise?
Imagine what a real-time data pipeline could do for your business – how could it impact the experience you deliver to your customers?

In summary...

Our panel all agreed that it’s never a case of moving to the cloud and all your problems are solved. But there are some practical things you can do to improve your chances of success:

Solve the data black holes: gain a single view of your data and use robust data policies to overcome the reluctance to delete data.

Use tools to democratise data: build a compelling business case based on real business challenges and run a pilot to prove it.

Secure key stakeholders: identify who they are and think about framing those conversations around data as a risk.

View data as an opportunity and a risk: implement a mechanism to nudge people into displaying the right behaviours with data, and imagine what’s possible with complete visibility of data.

Prevent re-polluting data: ensure data cleansing remains an ongoing priority and push it on the agenda by relating it back to customer experience.

Would you like to attend our next (virtual) roundtable event?

Although places at our roundtables are by invitation only, we’re always interested in hearing from people that would like to get involved.

To register your interest, please contact:

Anna Cotton

Phone: +44 (0)7779 270558

Email: anna.cotton@exonar.com

View full post