Overcoming the challenges of big data projects

Communication Service Providers (CSPs) are well known for having the advantage when it comes to the volume of network and customer data they hold. They aren’t new to handling huge volumes of data and are generally considered to be better at exploiting it than some other industries.

However, when it comes to Big Data projects, CSPs face challenges too.

Here we explore some of the hurdles CSPs may encounter on a Big Data project.

Buy in from the business

Before looking at the technical and project issues, we must start at the beginning, the planning stages which include getting the project signed off and ticking all of the right boxes.

Challenge:

The Business Case

The outcomes and benefits to the business are key when taking on any project, determining the need and key deliverables. Before starting a Big Data project it can be difficult to showcase the potential value without having a few insights and pieces of evidence to support that the insights from the data will add value.

Solution:

Working on a small but strategic basis to start can help to address this challenge and may help when it comes to formulating a stronger business case to showcase to the top management of a CSP in order to help with buy in.

Challenge:

The Legal and Compliance requirements

Considering the amount of customer data a CSP holds, there is often a drive to monetise this as much as possible. This creates an issue when it comes to regulation and customer privacy

Solution:

A solid foundation of Data Governance should be implemented. This helps protect customer data wherever it is, as well as mitigating the risk of it being used inappropriately, and ensures the goals of the Big Data project can still be met.

Getting technical

There are 2 fundamental technical challenges CSPs will need to address when undertaking a Big Data project, we address them both below.

Challenge:

Data Sources and Volumes

As technology advances including use of connected devices, the Internet of Things (IoT) and 5G, the sources from location services, apps, social media and streaming all create previously unthinkable volumes of data from more sources than ever before. The question is how to manage it all.

Solution:

It is crucial for CSPs to be able to consolidate all of these data sources and volumes into a single version of the truth in a format that can facilitate analytics and actionable insights.

Challenge:

Internal CSP Data Silos

Within the CSP data will be stored in different systems, across different departments, processed and stored in different ways, sometimes over such a long period of time it is almost impossible to have a complete view of what data is actually held across the CSP. These data silos create the challenge of being able to share and consolidate all of the data in order to make a success of your Big Data project.

Solution:

The first step towards consolidating your data management systems is to run and audit across the business to identify what processes and systems are contributing to the silos and what data is held where. This information will support effective decision making on how to move forwards.

People, resources and development

One of the common hurdles businesses run into, CSP or not, is the struggle of time and resources. Whether you don’t have the skills to complete a project in house, or the time available vs the complexity of the project itself.

Challenge:

Skills Shortage

The resource and skills required on a Big Data project are often underestimated. The ability to understand, evaluate and implement the technology is key, alongside the expertise to undertake advanced analytics and the business acumen to turn insight into action. Often CSPs may have one or a few of these skills in house, but could be missing a critical piece of the puzzle.

Solution:

Access to outsourced consultants through your supplier or a partner can often mean fulfilling the missing aspect temporarily or permanently, or working with a supplier to obtain support days for short term projects or training that will upskill your in-house team.

Challenge:

Development required to integrate solutions

More often than not, achieving all of your Big Data goals requires a number of connected technologies to create a holistic solution. This can lead to an increase in development work, as well as the requirement for skills across multiple systems.

Solution:

Vendor partners of systems you may be using, or consultancies that work across multiple platforms can often connect the dots between the various technologies and tools required. They can pull together a solution specifically designed against the outcomes and deliverables of the Big Data project.

As technology advances and data volumes and sources grow, so does the demand to do great things with Big Data. With this increase in demand comes a need to overcome these challenges, generate data insights and turn them into actions.

Data insight and intelligence will help CSPs keep ahead of their competition and keep their customers happy, as long as they act quickly.