Eugene Cheah is Co-Founder and CEO of Featherless.ai, a serverless inference platform for open-weight models, and a core contributor to the RWKV project. AITech Edition sent him questions on memory supply, specialised models and sovereign AI, and he answered them in writing.
Why is the AI scaling bottleneck shifting from GPUs to HBM?
High Bandwidth Memory is extremely rare and expensive, and, above all, difficult to manufacture. Once a model is deployed into production, things get very real, financially speaking. The inference phase, when algorithms are used to respond to user queries, is the most costly. In production, you are limited by how fast weights are read from memory on every token, and not by the chips’ computational power.
A good example is consumer hardware. Nvidia GPUs frequently sell out, but the cards with the most VRAM are the hardest to get. Underneath the hype of the whole shortage of chips lies the fact that the main problem that holds deployment back is memory capacity, and not computing.
To survive economically and stay profitable in this environment, developers and organisations have to design their systems around physical memory constraints.
Could HBM shortages become a bigger constraint on AI growth than GPU supply?
The tech industry is fixated on one main metric: the raw number of GPUs accumulated by hyperscaler enterprises and governments. A common opinion is that the winner of the AI race is whoever has the most processors. This idea is familiar because it mimics previous technology booms where scaling hardware solved every engineering hurdle. However, this misses a bigger issue: the global shortage of HBM today. Without enough memory, even the best computing clusters will sit idle, unable to run at full speed, waiting for data to arrive.
Recent research shows that the global semiconductor ecosystem is facing an unprecedented memory chip deficit, with hardware growth heavily restricted by raw material availability. Another report on High Bandwidth Memory manufacturing supply found that production capacity among the world’s leading chip producers is entirely sold out through 2028, turning physical packaging and memory distribution into a serious challenge for the entire AI market.
What changes when companies train models on their own proprietary data?
General models achieve maximum effectiveness of 70% on specialised tasks, but enterprise deployment requires at least 90% effectiveness. Bigger pre-trained models will not solve this problem. The true solution is to train small open models on enterprise-specific data for each business process.
AT&T, together with Microsoft and AMD, demonstrated this approach with OTel 2.0, a specialised open-source model built specifically for the telecoms industry, using 400 billion telecom-specific tokens selected from more than 1 trillion processed tokens.
Specialised open models have more advantages: they not only enable businesses to have hardware and computing independence, but also reduce their monthly bills. Relying on a single vendor’s chip ecosystem or cloud monopolies creates unsustainable overhead for growing businesses, whereas open-weight architecture enables companies to deploy their models on alternative hardware platforms. With open alternatives such as GLM 5.2 you can reduce monthly inference costs by up to 95%, while matching or beating larger generalist models on specific tasks. For one customer, we managed to reduce their monthly inference bill from approximately $150,000 to $7,600.
Why is sovereign AI becoming increasingly important for enterprises and governments?
It provides essential operational resilience and total control over data. Using proprietary cloud or chip monopolies includes inefficient overhead costs and lock-in that cannot be sustained. Using open models on private cloud with zero logging policies guarantees total privacy of both sensitive corporate data and always-on 24/7 coding agents.
In addition to operational efficiencies, sovereign AI is becoming a necessity to protect scientific and cultural sovereignty. There’s growing friction in areas such as mathematics, as we’ve seen from the recent Navier-Stokes controversy and the Association for Human Mathematics’ statement, which defends the preservation of human authorship and reasoning in the face of AI. Why is that important when talking about governments and businesses? Because by processing their research, workflows and institutional knowledge through closed models, they run the risk of losing ownership of their intellectual property and output. Sovereign AI lowers that risk.
How should organisations navigate AI sovereignty and regulations such as the EU AI Act?
At its very heart, sovereignty is nothing other than absolute control of your own territory. By reducing dependencies on external entities, you can cut out extraneous cogs in the system, as well as failure points and supplier risks. And by having a proprietary closed system doesn’t make safety or compliance any easier.
Rather than seeing this as a disadvantage, early compliance with EU and other territories’ regulations becomes a unique strength for companies. Operating on compliant open models helps organisations keep their sensitive corporate assets secure, and enables businesses to use AI without interruption and with fewer regulatory hurdles.
To navigate the EU AI Act and still maintain sovereignty, organisations are best placed to use open-source models that run on regional and zero-logging infrastructure. They can also combine open models with runtime guardrails and live policy monitoring, which enables teams to audit necessary workflows. This way organisations are not only complying with regulation, but also safeguarding their corporate intellectual property.
From a cybersecurity point of view, masking AI infrastructure through closed interfaces makes it hard to detect vulnerabilities until a major breach occurs. Open-source models enable security researchers to detect vulnerabilities and create security safeguards much faster.
With 7,000+ languages worldwide, what is the cost of leaving language diversity out of AI?
Only about 100 out of 7,000 languages that are currently being used around the world are served by mainstream frontier models. By ignoring languages such as Moroccan Arabic or Tagalog, we are denying millions of people access to technology in the language they speak. This causes technological discrimination as entire groups are forced to adapt to the culture of the vendors.
When diversity is not considered, businesses lose access to rich global markets. It shouldn’t be this way. I am a strong advocate for AI to serve the global community, not just a small group of languages and cultures. That’s why, with Featherless, we want to provide computational resources and serverless hosting services to individual developers building local language datasets.
What should enterprises rethink about their AI strategy as the industry moves beyond simply scaling bigger models?
I’d recommend that the industry stop measuring productivity with raw token volume. This generates vanity metrics that drive up computing costs and generate significant latency without any meaningful increase in business value. Businesses and dev teams should evaluate their AI strategy based on price per task completion, as it reflects the true economics of production deployment. Using an enormous frontier model for routine tasks such as document extraction or classification is inefficient.
We need to be using the long tail of specialised models and reserve big general models for very infrequent unstructured tasks. This slashes your inference costs dramatically and increases your efficiency at the same time. One way we’ve put this into practice is with our open-source library Simple Jev, which turns open-source models into zero-shot classification engines. It minimises conversational text and focuses purely on fast decision-making, which in turn cuts compute use and drops inference costs to as low as $0.03 per million input tokens with free output tokens.
