Is better BI going to save your business?

Business intelligence systems and analytics have leapt forward in recent years. Gone are the days of retrospective reporting and data warehouses that were updated each day. We are now in the era of using data to make real-time decisions with live data from many sources.

Business intelligence - it almost sounds like the punchline to a bad joke when you say it out loud. But nothing could be further from the truth. In the data rich environment today’s corporations operate in, the capacity to understand what is happening around them is an important weapon. In the post-GFC world, where market volatility is still an everyday concern, the ability to anticipate what is coming next separates tomorrow’s winners from the also-rans.

It’s likely that in the future, businesses will compete on the quality of their data and ability to make decisions based on more intelligent algorithms as much, or perhaps more, than their ability to reduce costs or create new products.

It’s impossible to discuss analytics and business intelligence without segueing into a discussion about big data. But, as we said in the November/December 2012 issue, we are quickly moving to a time where the term ‘big data’ will cease to exist - everything will just be data. Data from multiple sources, arriving faster with little or no discernable structure, will be the norm and not the exception.

“Depending on your view of what big data represents, there’s a layer of complexity on that that a lot of organisations are wrestling with,” says Mark Sands from QlikTech. “Fundamentally, that comes back to a huge emphasis from the IT community, and vendors in particular, who are driving this,” he added.

Guy Harrison, from Dell Software, says, “I think it’s pretty clear that over the last five years, there was at least one big shift in the BI software market. That was the emergence of self-service BI, sometimes called agile BI.”

Before this, business intelligence and analytics were largely based around the regular extraction of data from corporate systems. These extractions were then rendered into data cubes that were used for retrospective reporting and analysis.

Joe Collins, from CAMMS, says, “There’s a driving force to connect BI components to impacting directly how organisations are run. There’s a much stronger emphasis in driving execution of strategy and much closer alignment with the budget process and long-term financial planning.”

Businesses are spending significant amounts of money on better data collection, collation and analysis tools. Interestingly, Michael Pain, Accenture Australia’s Analytics Lead, said, “Return on investment that they’re seeing, it’s fair to say, is mixed. There are a range of reasons for that. One of them is the idea that a lot of organisations, when they invest in analytics, don’t start with the right questions.”

This is a key. Business analytics require the business to have some idea of the outcome they are trying to achieve. Analysis for its own sake or because everyone else is doing it is not a good reason for investing.

Although it’s important for businesses to have solid plans for deploying analytics and business intelligence tools, there needs to be a balance. “In analytics, we believe a business case has to have a balance between clear and proven outcomes but also an element of experimentation,” said Pain.

While reporting after the fact remains an important business imperative, the ability to make decisions in real time wasn’t supported. While it was great for the C-suite, it didn’t help line managers who wanted to be able to make more immediate decisions. As a result, line managers began creating their own systems in Excel or using desktop database applications. The trouble was that different parts of the business could use the same data and, by applying different analysis techniques and their own definitions, end up producing inconsistent results.

Agile BI systems make it possible to deliver the data more consistently, in real time and in a form that allows line managers to make real- or near-time decisions with confidence. Rather than working with raw data and then manipulating it manually, they are able to work with data sets that are better tailored to their needs and delivered in a more timely manner.

Collins clarified this saying, “All data has a cause and an effect. It’s not just about reporting the data, it’s about trying to link traditional metrics to corporate performance.”

We are already seeing this in retail where companies can see what you’ve browsed and purchased online, use demographic data they’ve gathered and then target specific advertising or product recommendations.

The way this works requires a shift in how businesses store and manage data. In the past, most of the organisation’s data was held in a structured database and then distributed outwards to the data warehouse. In today’s world, we’re looking at massive volumes of data, from multiple sources, and much of it is unstructured. According to Harrison, this means were more likely to see a Hadoop-style cluster as the main data repository with in-memory processing systems used to extract and manipulate the data for analysis.

However, the data modelling that used to happen at the start of the analysis process is performed after the data enters the database, not before. Harrison calls this “just-in-time modelling”. Businesses that aren’t able to leverage the unstructured data are “just going to lose” in his view.

It’s tempting to see the issues around using data as being largely technical. However, “You can change and manipulate data until the cows come home but unless you have a direct connect to the business that’s all you do,” added Collins.

The good news for the CIO is that many of the skills needed to support the business in using business intelligence tools already exist in the IT department. However, they’re often channelled towards a different purchase. Using live data from systems to monitor and manage performance is something many systems managers have been doing for some time.

Applying similar techniques, we’ve seen companies such as Sportsbet, Coles and iiNet use BI-like tools from Splunk to monitor their systems in real time and manage performance. For example, by monitoring server performance during peak periods, they are able to reassign resources dynamically to ensure that systems not only remain available but perform at optimal levels.

How information comes into a business and is used to deliver benefit to the bottom line is critical. Grant Christian, from Information Builders, says, “The information delivery chain is much more than just delivery or presentation of information, it extends out into the cloud with social analytics in one direction, but then also extends right back to the data and its quality, making sure it is correct on input or, when it is derived from other information, that the quality is inherent with the transactional process, that all information comes from the same source. Then we need to ensure all users have access to the information, and how it is presented will depend on the level of detail required at the time.”

The question then becomes about whether business users are able to ask the right questions of the data. Business users are becoming increasingly sophisticated in the types of insights they want and in using that information more wisely. While the role of the data scientist is becoming well known, he’s seeing the marketing scientist, who understands the intersection between data, algorithms and business issues, as a new role that’s developing.

IT delivery teams have, when they’ve worked effectively in delivering solutions, worked with end users to determine requirements that were used to develop and deploy systems. With BI, Sands prefers the term “business user” rather than “end user”. This is because end user suggests the end of a process. But business is ongoing activities, so by shifting the focus away from what users want to what the business needs, BI projects have a greater chance of delivering business value.

As business intelligence and analytic tools become more prevalent within organisations, there is an obvious business opportunity for software vendors. As a result, there have been many new entrants in the market leading to what Christian describes as the risk that “these are not yet enterprise class and [that businesses end up] with many disparate solutions to manage and maintain”.

Many BI projects that we’ve seen have been very IT-centric; much of the focus has been on the technology. However, successful BI projects put the business analysis with the business user and the data management, which is a very technical discipline, with the IT department.

“We’re enabling the technology side of the house to focus on the things that they’re better at. It isn’t producing reports. It’s managing data. It’s security. The tools enable business users go to places they haven’t be able to get to before,” said Sands.

Looking ahead

So, what’s next? Are the next few years going to be simply incremental changes on what we are seeing today? Now that big data is just data and business intelligence and analytics tools are quite mature, it’s easy to imagine a period of consolidation.

Mark Sands from QlikView sees a future with even more diversity. “If we look at how the market has evolved over the last few years, we have recognised that there isn’t a one-size-fits-all solution to delivering BI. What we absolutely are seeing is that organisations are recognising that there’s a capability that their businesses require them to take a much more agile app, or even take a throwaway approach to allow users to self-serve. As we move forward that’s only going to accelerate.”

In addition, he noted that there are still many organisations that haven’t invested in business intelligence or that many business users still haven’t got access to these tools. “If you look at numbers from IDC, only 27% of the people in a typical organisation are served in some way by that organisation’s business intelligence investments. Really, we see a lot of the evolution in reaching out to those 73% of users. Growth is going to be a fundamental element of business intelligence.”

As more users get access to the tools, there will be changes in how the tools are used and deployed. Dell expects a broadening of the user base as staff with lower level technical skills access data. This will drive further innovation in the tools so that user interfaces make it easier to gain meaning from data and present information that can be used for intelligent decision making.

The reality is that the next few years will see a perfect storm of business needs and evolving technology. Grant Christian from Information Builders said, “Big data, cloud, social media, predictive analytics and mobile. More than buzzwords, these technologies are allowing organisations to leverage data to accelerate business. Used together, these five business intelligence trends are converging to transform the way critical business decisions are made, and opening opportunities for new services and new revenue.”

Image credit ©iStockphoto.com/Henrik Jonsson