In the rush to AI, tech bills are running sky-high

Artificial Intelligence (AI) is doing more than just blowing up the airwaves — it’s blowing up cloud bills for businesses that dive in prematurely.

In the wake of ChatGPT, and the multiple large language models that soon followed, AI advances are making headlines across the globe.

While there are untold benefits enterprises can gain from AI — ranging from optimising business processes, improving existing products and even creating new products — there are a number of pitfalls businesses must avoid in order to reap the rewards.

Perhaps the most crucial, from a business perspective, is managing the cost of running these models.

Money doesn’t grow on trees, but it can leak from the cloud

No business leader is implementing AI for AI’s sake. They’re doing it to generate a return on investment (ROI) through either greater efficiency, unlocking new business opportunities or myriad other potentialities.

But the greater the cost incurred when running generative AI, the greater the required benefit to achieve ROI.

It is the nature of the beast that massive amounts of general-purpose and purpose-built data are needed to drive AI engines. All that data must be stored, managed and secured — and each of these functions incur a fee.

Given the capacity, performance and scale requirements needed to effectively manage these models, many early adopters sought to run them on the public cloud.

Although the aim of these early projects was to transform the business, the end result for many has been a severe case of bill shock. As the initial reckless abandon gives way to a more mature consideration of how to leverage these seemingly sentient programs, organisations are taking a more nuanced approach. Increasingly, an LLM will be trained in the public cloud before being run on an organisation’s own infrastructure to allow for greater cost control.

Not only does this make sense from a cost perspective, but it can also mitigate other unintended consequences.

Keeping data under control

While cost efficiency is an important challenge in any IT project, so too is maintaining security and control over data.

One of the biggest pitfalls enterprises must avoid when implementing AI models is running afoul of data security and sovereignty regulations.

This is particularly difficult to manage effectively in the public cloud — even with traditional workloads. With the new breed of AI models, this challenge is turned up to 11 as the legislation governing their use and training is still being developed.

For multinational organisations, data sovereignty poses a significant risk. As nations continue to consider the privacy implications of AI, it appears unlikely there will be a global consensus governing AI models and the data used to train them.

This requires enterprises to have control over their data and how it is used. An LLM used in the Australian arm of the business, for example, should be trained only on Australian data and used only in Australia. Introducing data from European customers into the Australian model, for example, could introduce regulatory risk.

If the EU can force Apple to standardise the chargers on its devices, it’s a safe bet it will ensure its citizens’ data is not used overseas.

The point is we’re only just starting to figure out what questions to ask in the regulatory space. It will be some time until we have even a glimpse of the answers. Until then, enterprises need to maintain control of their data and applications — something that is much harder to achieve on infrastructure you don’t own.

Regulations aside, there are multiple security concerns to consider. For example, if developing a customer service chatbot, proprietary product data would need to be included in its training. This is extremely sensitive information that organisations would be unlikely to feel comfortable keeping in the public cloud.

Further, the vast majority of today’s enterprise AI projects are designed to deliver a competitive advantage. That advantage can only be achieved if the models and data used to train them are kept secure.

Ultimately, the excitement around AI is well founded. It has the ability to completely reimagine how businesses, governments and society as a whole operate. But in these early days, we must be cognisant of the risks and we must manage them.

And although innovation isn’t free, the price of admission should never be a surprise.

Image credit: iStock.com/Feodora Chiosea