AT&T runs an average of 45 billion AI tokens a day, and it built what it calls a smart router to automatically point each prompt at the most cost-effective model for the task. Open models — open-source and open-weight both — now power about 25% of the company’s AI usage, and Andy Markus, its chief data and AI officer, expects that to reach 70% to 80% over time. Switching from closed proprietary models to open ones has already saved 80% to 90% in certain applications, he said. “You hear the stories of the ‘token apocalypse,’ and we’re not scared of the token future,” Markus said. “It’s something that we feel like we can manage.”
AT&T has over a thousand internal AI uses, from back-office legal and finance work to field technicians and core network operations. Summarizing and analyzing customer service call transcripts runs entirely on open models. So does the network itself: OTel, an open model AT&T customized with telecom-specific data, supports agents that detect the root cause of network issues. Open models can also run in AT&T’s own data centers rather than rented cloud infrastructure, trimming costs further. Markus has tested a range of them, including powerful new models from China, which he says keeps AT&T from becoming “beholden to any one solution.” Critics have argued that open models, many of which come from China, present potential risks to national security.
Markus describes data protection and intellectual property as the reason open models are nonnegotiable for AT&T. “We have to keep our information safe and protected, and we have to control the inputs and the outputs of the models,” he said. “The enterprise data is the gold mine, and the tools are just a way to mine the gold. The concept of AI sovereignty has become truly paramount to us.”
Chirag Dekate, an analyst at Gartner, says enterprises that start on premium frontier models reach an inflection point where the costs become unsustainable, and that they also want to avoid depending on a single vendor and to keep their intellectual property away from AI labs. Gartner expects open models to underpin more than 50% of business use cases within two years, up from less than 10% today.
Read more at The Wall Street Journal.