Cloudera launches AI inference solution

Data, analytics and AI platform provider Cloudera has launched a new AI inference solution designed to streamline the deployment and management of large-scale AI models.

Powered by the NVIDIA NIM range of microservices, Cloudera says its AI Inference solution can help enterprises advance GenAI solutions from pilot phases to full production. Using NVIDIA Tensor Core GPUs, developers can test, customise and deploy enterprise-grade large language models with up to 36 times faster performance and nearly four times faster throughput compared with CPUs.

The solution is integrated with Cloudera’s AI Model Registry to enhance security and governance by managing access controls. The platform supports developing and optimising solutions based on open-source large language models including LLama and Mistral.

Other key features include the ability to run workloads on-premises or in the cloud, conduct A/B testing and canary rollouts for risk-managed deployment, enforce model access with enterprise security controls such as access control and service accounts, and access standards-compliant APIs for model development, management and monitoring.

Cloudera Chief Product Officer Dipto Chakravarty said Cloudera AI Inference has been designed to help customers address some of the biggest barriers to GenAI adoption, including compliance risks and governance concerns. The platform protects sensitive data from leaking to vendor-hosted AI model services.

“We are excited to collaborate with NVIDIA to bring Cloudera AI Inference to market, providing a single AI/ML platform that supports nearly all models and use cases so enterprises can both create powerful AI apps with our software and then run those performant AI apps in Cloudera as well,” Chakravarty said. “With the integration of NVIDIA AI, which facilitates smarter decision-making through advanced performance, Cloudera is innovating on behalf of its customers by building trusted AI apps with trusted data at scale.”

Image credit: iStock.com/YUTTHANA JAIDEE