Why having an observability strategy is critical for effective AI adoption

Artificial intelligence (AI) tools are already delivering significant benefits for organisations of all sizes; however, increasing numbers are becoming concerned about the associated rising costs.

Development overheads and the usage of cloud-based platforms can quickly result in project budget overruns. The objective, therefore, is to harness the power of AI while at the same time keeping spending in check.

There are two key reasons that AI costs can spiral out of control. The first is that training and running AI models and querying data requires massive amounts of computational resources, often on a cloud platform, which results in higher consumption charges.

A second reason is that AI applications perform frequent data transfers. Additional data transfer costs may occur because the applications need more frequent data transfers between edge devices and cloud providers.

The solution to rising AI costs

For organisations to succeed with their visions for AI, they need to address these sources of escalating cost, and this can be achieved by establishing a solid FinOps strategy. FinOps, where finance meets DevOps, is a public cloud management philosophy that aims to control costs.

At the same time, organisations also need to consider AI observability. This is the use of AI to capture the performance and cost details generated by various systems within an IT environment.

Effective AI observability can also provide IT teams with recommendations about how to curb these costs. As a result, AI observability supports cloud FinOps efforts by identifying how AI adoption boosts costs because of the associated increase in usage of storage and compute resources.

Increasingly, organisations are acknowledging that they can’t get the business benefits from AI without paying strict attention to associated costs. AI observability can help them understand the ROI on their AI investments as AI hype gives way to real adoption.

Managing AI costs with observability and FinOps

There are a range of strategies that can help organisations to keep a lid on AI costs while at the same time enjoying the benefits the technology can deliver.

Taking both cloud and edge-based approaches

Cloud-based AI allows organisations to run AI in the cloud without the hassle of provisioning and managing on-premise resources. Edge-based AI enables organisations to run AI functions on edge devices — such as smartphones, cameras and sensors — without having to send the data to the cloud. By adopting a cloud- and edge-based AI approach, teams can benefit from the flexibility, scalability and pay-per-use model of the cloud while also reducing latency and bandwidth costs.

Adopting a containerisation strategy

Containerisation enables organisations to package AI applications and dependencies into a single unit. This can then be easily deployed on any server with the necessary resources. This approach optimises costs by enabling organisations to use dynamic infrastructure to run AI applications instead of designing for peak loads.

Undertaking continuous monitoring

Once an organisation has trained AI models based on its data, it is important to monitor algorithm performance over time. This monitoring helps to identify areas of improvement and drift. Drift is the decline of the predictive power of a model as a result of the changes in real-world environments that aren’t reflected in the model itself. Over time, models can easily drift from these real-world conditions and become less accurate.

Maintaining optimisation

As well as undertaking continuous monitoring, IT teams also need to maintain optimisation. This involves improving the accuracy, efficiency and reliability of AI by using techniques such as data cleaning, model compression and data observability. Optimising AI models can help save computational resources.

Manage the overall AI lifecycle

Important management tasks include creating, deploying, monitoring and updating AI applications. Managing the AI lifecycle means ensuring that AI applications are always functional, secure, compliant and relevant by using tools and practices such as logging, auditing, debugging and patching.

Combine generative AI with other technologies

While generative AI is clearly a powerful tool, it is no silver bullet. The true potential of generative AI comes from using it in conjunction with predictive and causal AI tools. Predictive AI uses machine learning to identify patterns in past events and make predictions about future events. Causal AI is a technique that determines the precise root causes and effects of events or certain behaviours.

As organisations continue to adopt AI and put it to work in a variety of innovative ways, many will become concerned about the associated operational costs. By combining the strategies of AI observability and FinOps, these costs can be contained and maximum benefits delivered. Consider how following such a strategy could benefit your organisation.

Based in Sydney Australia, Rafi joined Dynatrace in 2007 and has more than 20 years’ experience in the IT industry focusing on business, application and IT service management. His experience includes working for a number of Australian-based technology startup innovators. Prior to joining Dynatrace he held a senior management position at technology startup Proxima Technology.

Top image credit: ArtemisDiana