You can run a Chinese AI model on a laptop. The models are powerful, capable, in high demand, and help address a business problem: spending on AI tokens.
That is changing the economics of AI services and creating competitive problems for frontier models.
In its IPO prospectus released on Monday, Anthropic said that as a wide variety of models becomes available, “customers may conclude that incremental performance gains at the frontier do not justify the additional cost.”
Chinese AI models accounted for more than half the reported token volume on OpenRouter, in its global count, in the week ending on Oct. 25. About a year earlier, only one Chinese model appeared among the nine named on the site’s weekly usage chart. OpenRouter provides access to multiple AI models through a single API.
Red Hat trying out Z.ai model
For its own internal use, open source enterprise software provider Red Hat is piloting a self-hosted version of GLM, an open-weight model from Z.ai in Beijing. Chinese tech giant Alibaba’s popular Qwen model is also available at a smaller scale.
Engineers were initially nervous about whether open-weight models would be as capable as proprietary frontier models, said Stephen Watt, distinguished engineer and vice president of the office of the CTO at Red Hat. But they discovered that they were “good enough,” he said.
Watt said open-weight models, primarily from China, had reached an efficacy level comparable to Anthropic’s Opus 4.6 in certain use cases. He said engineers had concerns about moving away from frontier models, but Red Hat’s usage data suggested open-weight models were adequate for many tasks.
“You’d be crazy not to start on a frontier model on your AI journey, but after a while, you’d be crazy to stay on it because it just eats all your margin,” Watt said, referring to the comparatively high cost of most closed models.
Business users, engineers and developers can allocate workloads based on the capabilities of AI models. They can host models internally, in closed environments.
Geopolitics and AI
But some are apprehensive about using Chinese models, especially because of geopolitics, which is fueling interest in U.S.-developed models. The hesitation about Chinese models, where it exists, is usually based on geopolitical concerns. This is creating a pent-up demand for Western-built open-weight models, such as those developed by Meta and Reflection AI, Watt said.
DataCamp, an AI learning platform used by about 6,000 organizations, had its AI tutor analyze some 1.4 million hours of learning in the most recent month. It has been evaluating Gemma 4, a Google open-weight model for tutoring. Chinese models could be a problem for some governments and enterprise users, said co-founder and CEO Jonathan Cornelissen.
But for now, the major limitation of open-weight models is infrastructure. The providers that run the open-weight models can’t deliver them at the speed or scale that is needed, Cornelissen said. The latency is not competitive, he said. Some have the speed but lack capacity, while others fall short on response times.
For now, DataCamp is moving from Anthropic to OpenAI, which Cornelissen said he sees as less costly. It has not moved its tutor to an open-weight model.
Jonathan Steele, a partner at Beermann LLP in Chicago and founder of cybersecurity consultancy Steele Fortress, uses a Qwen model locally on a MacBook Pro for billing and other sensitive tasks. He also uses Claude and ChatGPT for other work.
Questions about open-weight quality
The economics of open-weight models may be compelling, but their usefulness depends on infrastructure and output quality.
Comparing costs isn’t straightforward. If an open-source model might take half an hour to produce a draft but needs five hours of attorney review, while a frontier model might take 15 minutes to draft and 30 minutes of revisions, it’s hard to determine which is more expensive.
But some companies, like Red Hat, are finding ways to get value from Chinese models. Its Z.ai pilot is being used by as many as 1,400 engineers, with about 70% of their inference traffic going to the open-weight models. Engineers have a budget for using frontier models and allocate between the open-weight and proprietary models based on the work they want done.
For Red Hat, controlling cost was a key driver. Frontier-model spending was growing unconstrained. Engineers, with the arrival of open-weight models, are limited to a $300-a-month quota per associate to encourage them to choose the right models for the task.
The quotas are experimental, and engineers can request exceptions from managers. The cost of externally hosted models went from runaway to flatlining to “radically decreasing” now, Watt said.
For the future, geopolitical issues involving Chinese AI models may intensify. The U.S. Department of Homeland Security this month barred federal civilian agencies from using Chinese AI developer DeepSeek’s AI technology, citing “multiple threat vectors,” including potential transfer of data to China. But the order doesn’t distinguish between DeepSeek’s hosted versions and those a business can run on a local server.
Some stick with U.S. models, for now
Albert Ziegler, head of AI at Xbow, an autonomous offensive security company, said it had tested GLM but was not using it in production. But he didn’t rule out open-weight models. His criteria are performance relative to cost, safety benchmarks and customer acceptance.
In Xbow’s tests, GLM sometimes needed substantially more tokens to solve a problem than a closed model, offsetting much of its price advantage. For its production applications, other closed models, Grok from Elon Musk’s SpaceXSI, in this case, were more efficient, Ziegler said.
A broad concern for Ziegler is the danger open-weight models might bring.
In a year, the open-weight models will be better than today’s closed models, “and by quite a bit,” Ziegler predicted. That could be scary, he argued. “People can host a model of that caliber on their own private cluster without any restrictions.”
Patrick Thibodeau has worked for several decades as an enterprise reporter, focusing on IT and workforce management, ERP, high-skills immigration, tech policy and high-performance computing