Explainable AI: building trust and creating value

Generative AI is transforming the way businesses operate, expanding the potential of creativity and efficiency and offering innovative solutions at unprecedented speeds. APAC is expected to increase investment in the area by 140% in 2024 and beyond, as businesses continue to adopt generative AI to tap into opportunities for growth and gain a strategic advantage in everything from product design to decision-making.

As advanced models become increasingly popular, however, the focus is sharpening on ROI and tangible business value, with a premium placed on the trust required for businesses to implement the technology in a meaningful way.

Explainable AI plays a critical role in building trust and unlocking the full business value of AI when properly integrated.

How Explainable AI enhances trust and performance

Explainable AI (XAI) is designed to make decision-making processes transparent and understandable to people. Unlike traditional AI models, XAI shares insights into how and why it reached certain outcomes which can enhance accountability and fairness and provide an opportunity to correct bias in AI systems.

With the following benefits, XAI can enhance value for businesses and mitigate the risks associated with AI:

• Transparency: Understanding how AI systems arrive at decisions helps us trust those decisions and feel more confident in adopting the technology.
• Reduced resistance: A ‘black box’ approach to AI, whereby an AI system works in a vacuum, its processes entirely opaque, can cause suspicion. Explainability, on the other hand, fosters a collaborative approach where human expertise and AI work together to come up with the best solutions.
• Error identification: XAI techniques identify errors and biases in the model’s training data or decision-making process. This allows for more efficient troubleshooting and model improvement, leading to greater confidence in the model’s capacity to deliver value.
• Targeted optimisation: By understanding the factors influencing model outputs, businesses can focus optimisation efforts on specific areas for greater efficiency and accuracy.
• Ethical considerations: XAI allows businesses to identify and address potential bias or discrimination in a model, promoting ethical and responsible AI development.
   

By leveraging these features, businesses can use XAI to cut development times and costs through faster identification of problems, while using more accurate and trustworthy AI contributions to deliver better results across sales forecasting, fraud detection and customer service.

Using XAI to set smart guardrails

For all their myriad capabilities, AI systems also come with limitations that businesses must be aware of including, for example, bias or discrimination, a tendency to hallucinate, security vulnerabilities, and ethical concerns around privacy and accountability.

XAI can go a long way to helping businesses mitigate these limitations and risks by identifying them and contributing to the establishment of appropriate guardrails. For instance, by explaining how AI arrives at its outputs, XAI reveals areas where the model is less reliable so businesses can focus their guardrails on those specific weaknesses.

XAI can also differentiate between outputs the model is highly confident about and those with lower certainty. Guardrails can then be tailored to provide additional human oversight. Better understanding of an AI system’s limitations can also ensure we don’t over-rely on its outputs and again set guardrails that ensure human involvement remains critical for high-stakes decisions.

Imagine, for example, an AI system used for loan approvals. XAI might expose that the system struggles to assess non-traditional income sources. An appropriate guardrail could be implemented to require human review for any loan application with such income sources.

Or consider an AI system used for employee performance evaluations. XAI could reveal the model finds it difficult to assess qualities like team collaboration or creativity — areas that are less easily quantifiable compared to productivity metrics. Taking this into account, businesses can establish safeguards that identify evaluations where these qualities are essential, prompting a human review to guarantee a thorough assessment.

When responsible AI meets Explainable AI

For AI to generate meaningful value, businesses should consider adopting innovative and responsible practices, such as LLM evaluation, to systematically assess the performance and effectiveness of AI systems. By evaluating accuracy, fluency, relevance, robustness, ethical concerns, and bias, developers can ensure their models are both effective and ethically sound.

Using an LLM-LLM evaluation (LLM guided evaluation) in conjunction with other techniques, including traditional metrics, allows businesses to get a more comprehensive assessment when unsure of which model is best suited for business needs. This process involves defining specific tasks for the model, such as generating EDMs, establishing success criteria like customer engagement metrics, and researching and shortlisting LLM options. Standardised benchmark tasks, evaluated under rigorous quality control by human reviewers, are then developed to set a performance baseline before testing the shortlisted models. Finally, the models are assessed using metrics such as accuracy, coherence and relevance to determine the best fit.

Together with XAI, the evaluation helps ensure the chosen tool is being used to deliver trusted results.

Moreover, an LLMG evaluation provides scalability by allowing standardised processes to be applied across models and tasks, while at the same time delivering the flexibility for businesses to tailor evaluation criteria to specific requirements and adjust success metrics as needs and applications evolve.

Generative AI offers a world of possibilities for businesses. Explainable AI allows them to tap into those possibilities efficiently, accurately and ethically by exposing weaknesses, establishing guardrails and evaluating for specific needs.

*Shreshta Shyamsundar is a Distinguished Technologist at Infosys, with over 20 years’ experience in leading digital engineering, cloud transformation and digital modernisation programs. He actively contributes to global industry forums to promote best practices and innovation in AI, digital and cloud technologies. He has been instrumental in driving API strategies in the era of generative AI, emphasising the importance of developing an API mindset to achieve true business potential through new models, partnerships and distribution channels.

Top image credit: iStock.com/Saksit Sangtong