Touchcast, a start-up that caches frequently asked generative artificial intelligence queries to make AI systems more efficient, is raising US$100-million from venture backers, including OpenAI investor Microsoft.
New York-based Touchcast stores and serves up responses for commonly used AI prompts, reducing the computational resources and energy required for using models like OpenAI’s GPT-4. The aim is to help developers and businesses deploy generative AI more broadly with lower costs and faster response times.
The Accenture-backed Touchcast will have a valuation of at least $350-million after the new funding, CEO Edo Segal said, declining to provide the details of the capital Microsoft has infused. Touchcast is in discussions with large Silicon Valley investors, he said in an interview.
Businesses are deploying AI systems to streamline daily tasks, but skyrocketing demand has laid bare a critical bottleneck in electronics, particularly GPUs, the scarce chips used to train and operate AI models. Touchcast’s approach could be an innovative way to make better use of computing and energy resources, Segal said.
The start-up recently launched what it calls cognitive cache content delivery technology, which is akin to having several small library desks in different parts of a giant library instead of a single main desk, making it easier to access material. The technology helps improve the performance of the AI system.
“When users ask the same question, they go to the large language model over and over,” Segal said. In the era of the AI-powered web, “a new class of infrastructure is critical to deliver these magical experiences at internet scale, sustainably”.
Cognitive cacheing
Touchcast is working with Microsoft to create systems to speed up and broaden the distribution of generative AI queries. It’s deploying the cognitive cacheing technology across Microsoft’s Azure data centres, helping to run Azure OpenAI tools to service customers and bring down costs significantly.
“The explosive growth of generative AI is on track to unlock trillions in economic value,” Segal said, “but its fate hinges on solving an escalating compute and energy crisis.” — Tom Giles and Saritha Rai, (c) 2024 Bloomberg LP