Big data's key drivers in 2015

How Australian organisations approach the challenges of big data will be key to their success. There are five areas business leaders should watch in 2015.

1. Democratisation of big data analytics. Over the past year, there has been incredible growth in the use of cloud-based data analytics services, and the cost-effective nature of the cloud will only accelerate this trend. Even organisations that once thought advanced data analytics were out of their realm of possibility can now begin managing and analysing both structured, as well as unstructured, data quickly and cost-effectively.

Essentially, the cloud will offer a greater array of choices for organisations to hit their desired big data benefit/price point trade-off, as well as lower the bar for companies looking to experiment with big data, particularly unstructured data.

2. Unstructured data grows. Unstructured data volumes - comprising things like human information, from social media, video, audio and pictures, machine sensor data, Internet of Things (IoT) data, and business data in various formats of work documents - will continue to grow at a mind-boggling rate.

According to Gartner, IoT data (excluding PCs, tablets and smartphones) will grow to 26 billion connected devices by 2020.

Organisations will increasingly seek solutions that can tie structured and unstructured data sources together virtually and generate connected intelligence from social media and video analytics. This will give greater context to the structured data that most organisations have come to rely on.

3. Predictive analytics becomes the norm. Predictive analytics will evolve beyond the next ‘cool thing’ to a ‘you better have it or else’ type of functionality. Reengineering for big data will be critical as business processes must be geared towards action at the speed of insight.

There’s no value in identifying what customers are doing every minute of the day if you can’t respond predictively and proactively. By the time you’ve ETL’d (extract, transform and load) the data in some warehouse or Hadoop cluster, it’s too late.

Organisations will re-engineer their big data environments to enable information streams, from both within and beyond the enterprise, to be accessed, analysed and shared in real time. This will be the key to increasing revenue, improving knowledge worker productivity and lowering costs.

4. Big data will change IT operations. Companies who ‘get’ big data are going to apply big data principles and practices to their internal IT operations first and foremost, long before they’re used for marketing and customers.

For years, we’ve heard that ‘IT is the business’. Big data will become the basis of competition and growth for individual firms, and the most logical place to exploit big data benefits will be an analysis of IT machine data itself - identifying ways to reduce waste and maximise productivity across the IT environment.

Big data analytics will also play a role in identifying IT security threats, which are constantly growing and evolving. It will also deliver connected intelligence across IT operations domains, generating insights that drive innovation and critical business advantage.

This process will reinvigorate the traditional service desk, and the transformation to a big data service desk will bring businesses capabilities to deliver service anywhere.

5. Big data for everyone. Today’s universities can’t seem to train data scientists fast enough for CIOs. Many in the industry view data scientists - those individuals with engineering and business skills, as well as the statistical savviness - as the key to analysing and deriving value from the big data companies generate.

But the current lack of so-called ‘big data talent’ should not hold businesses up in terms of launching big data initiatives. Rather, the key will be empowering business analysts of today with tools they already know.

In fact, the idea of a ‘data scientist’ may very well be played out in a couple of years, and the ‘data savvy business person’ will likely emerge as the new rock star.

However, it’s important to note, if a company assigns big data to its existing big BI teams, they’re almost guaranteed to fail. The tech, thinking and approaches that lead to success at BI almost guarantee failure at big data. It takes a whole new approach.

In addition, the availability of data analytics in the cloud presents a huge opportunity for developers, and we expect the big data developer community to increasingly emerge as a hotbed of innovation.

Cloud-based big data services represent decades of significant intellectual property around managing, accessing and analysing a wide range of data, including unstructured information that developers can now use like they have used Amazon Web Services or open source systems.

Developers have just begun to exploit the value of big data, especially unstructured data, and this trend will only accelerate in the next few years.

Image courtesy Tom Chance under CC