Big data getting faster, smarter, stronger

Big data is evolving, with three emerging trends set to restore Australian businesses' confidence in the technology, according to Deloitte.

Australian businesses have, in recent times, struggled to make the business case for big data, as they found themselves unable to prove a link between big data investments and financial performance.

“Our experience reveals that the reasons may be more structural than strategic,” Deloitte consulting partner Tim Nugent said. “The resilience of the Australian economy dampened pressure on local businesses to dig deeper for insights and to invest in more data-driven processes.”

But key trends in big data, such as the push for real-time operational insights, promise to change the state of play, he said.

The mining industry, for example, is taking the lead in big data adoption in Australia. “The business case for real-time operational insight is easy to make when minutes of downtime equate to millions in losses,” Nugent said.

The sector is also used to dealing with large volumes of data and has less need to interpret a variety of unstructured data, making velocity the last standout big data challenge.

Big data vendors are meanwhile starting to address the issue of costly big data implementation costs with smarter systems designed to integrate with existing business intelligence and analytical tools, reducing the integration and labour costs involved.

The final key trend involves increasing collaboration between businesses, research and education institutions on big data innovations, creating a stronger knowledge base, Nugent said.

“The community’s ability to bring together practitioners from different disciplines creates a rich network for self-learning and knowledge exchange.”

But the onus will be on the technology community to help fill the skills gaps left by underinvestment in data management capabilities. “Australian universities are doing their part but simply do not have the resources to meet the scale of the challenge,” Nugent said.

“In the current skills landscape, every company that is serious about using big data needs a plan to develop and access external data talent. Without this type of open engagement, Australian industry risks an encounter with critical shortages in data skills.”

Image courtesy of Alexandre Duret-Lutz under CC