🔍 Read the full analysis: Should Companies Bring AI In-House? IBM’s CEO On The Bet on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
IBM CEO Arvind Krishna argued on the company’s October 21 earnings call that open-weight models can deliver about 90% of frontier-model performance at a fraction of the token cost, making in-house or private-cloud deployments attractive for some businesses. An independent benchmark puts the best currently downloadable model at 80% of the leading proprietary model; the cost case also depends on hardware utilization, staffing and the model’s licence.
IBM CEO Arvind Krishna said on the company’s October 21 earnings call that open-weight AI models can be about 90% as effective as leading proprietary models at 10% to 25% of their token cost, strengthening the case for some companies to run AI on their own infrastructure. But an independent benchmark reviewed October 26 put the best model with downloadable weights at 80% of the top proprietary model, and the economics of buying servers depend heavily on how consistently they are used.
In the call’s question-and-answer session, Krishna described open-weight models as improving against proprietary systems and said enterprises could begin an on-premises deployment with “two or three racks.” He said open models cost “25% to 10% of the token cost of the frontier models” while being “90% as effective.” Those figures were his characterization, not a result tied to a named benchmark. IBM sells infrastructure used for on-premises AI, giving the company a commercial interest in the approach.
The independent comparison cited by ThorstenMeyerAI.com uses the Artificial Analysis Intelligence Index, which applies the same ten evaluations to models and identifies whether their weights can be downloaded. On October 26, Xiaomi’s MiMo-V2.6-Pro led the downloadable group at 46.3, while proprietary Claude Opus 5.5 scored 57.6. That puts the open model at about 80% of the proprietary leader’s score. The gap has narrowed from the end of 2024, when the leading open model scored 51% of the proprietary leader, though the score difference in points grew from 7.5 to 11.3.
A possible exception is Kimi K3.1 Pro, which scored 55.1, or 96% of Claude Opus 5.5 on the index. But Moonshot had only promised to release its weights “by Nov 4,” according to the source; until they are available, it is not an open-weight option. The source also notes that Kimi’s 2.8-trillion-parameter scale would itself require substantial data-centre resources. IBM separately said its llm-d project can run large models at five to ten times lower cost, a company claim the source says it could not independently verify.
IBM's 90% claim, checked
Open weights reach 80% of the leader
Krishna says open-weight models are "90% as effective". On an independent index the best model you can download today reaches 80% of the best closed one, up from about half two years ago.
Best open model as a share of the best closed
Artificial Analysis Intelligence Index, current version, best downloadable vs. best proprietary model at each date.
What Krishna said
Illustrations from the earnings call, not a named benchmark.
- Late 2024"70" vs. "20 or 30"Krishna, Q3 call, 21 Oct · no named benchmark
- Late 2025"70 to 50"Krishna, Q3 call, 21 Oct · no named benchmark
- Now"90% as effective"Krishna, Q3 call, 21 Oct · no named benchmark
Top downloadable models today
Intelligence Index score and licence type. Read the licence before the leaderboard.
| Model | Index | Licence |
|---|---|---|
| MiMo-V2.6-Pro Xiaomi | 46.3 | MIT |
| DeepSeek V4.1 Flash 1017 DeepSeek | 45.1 | MIT |
| GLM-5.3 Z AI | 44.8 | own commercial licence |
| Step 5 Preview StepFun | 43.7 | own commercial licence |
| Muse Spark Open Meta | 38.9 | licence with conditions |
In points the gap did not shrink: 7.5 points at the end of 2024, 11.3 today. Closed models moved too.
Sources: Artificial Analysis, Intelligence Index v4.3.2 (read 26 Oct 2026), IBM Q3 2026 earnings call transcript (21 Oct 2026), Artificial Analysis on Kimi K3.1 Pro (23 Oct 2026).
Powered by Thorsten Meyer AI
The Economics Depend on Utilization
For businesses, the choice is not simply between a cheap model and an expensive one. Running an open-weight model in-house can give an organization greater control over where data and logs are stored, while reducing reliance on a model provider’s hosted service. Krishna said banks and traders want AI to run “in a way that is sovereign,” with underlying data and logs kept out of others’ hands. That is his account of customer priorities, not evidence that every on-premises system meets a particular security or regulatory requirement.
The infrastructure arithmetic can change the apparent savings. The source compares a listed eight-GPU H200 server at $172,643 with AWS rental of the same GPU class at $63.30 per hour. At continuous full utilization, it estimates the purchase cost could be recovered in roughly 114 days; at 25% utilization, the estimate rises to about 15 months. Those calculations exclude staffing, cooling and networking, and do not by themselves establish total cost of ownership. An organization needs suitable engineering skills and a steady workload to make a hardware purchase pay off.
That capability is a constraint Krishna acknowledged. He asked, “Where are my kernel engineers? Where are the people who worry about the underlying network?” He said that skills gap remains for enterprises other than the most sophisticated. In his view, the frontier cloud is more expensive but “the easy answer.” Companies weighing a private deployment must account for the people and systems needed to operate it, not just the price per token.
Products Worth Considering
Jackery Explorer 2000 v2 Portable Power Station with Two 200W Solar Panels
Jackery Explorer 300 Portable Power Station, 292Wh
LG UltraGear 32GX850A-B 32-Inch 4K UHD Glossy OLED Gaming Monitor
As an affiliate, we earn on qualifying purchases.
A Narrowing, Not Closed, Model Gap
Krishna used a simplified account of benchmark progress on the call: in late 2024, he said, a top model might score 70 while an open model scored 20 or 30; by late 2025, he described the comparison as 70 to 50. He then framed the decision as whether to pay much more for the best model or use one he said was “90% as good.” These were illustrative remarks, rather than results from a benchmark he identified.
The Artificial Analysis figures give a more specific, but still limited, comparison. Its index shows the best open model rising from 51% of the proprietary leader at the end of 2024 to 74% at the end of 2025, then 80% on October 26, 2026. The leading models and index scores changed over those dates, so the figures describe relative positions on the index, not a controlled test of one model pair over time. Proprietary models improved too; the point gap was larger in October than at the end of 2024.
Cost comparisons also depend on where a model is run. The source says hosted DeepSeek V4.1 Flash 1017 costs 3% to 6% of Claude Opus 5.5’s list price, while closed-model GPT-6 Luna is listed at $0.10 per million input tokens and $0.50 per million output tokens. That means an open model is not automatically the cheapest option for every workload. Licences also differ: some models use MIT licences, while others impose custom commercial or community terms. A high benchmark score does not, on its own, grant permission for any business use.
Costs and Model Access Still Vary
The benchmark comparison does not settle how a model will perform on a company’s own tasks. The index uses ten evaluations, while Krishna’s examples included routine work such as summarizing email, examining contracts and answering customer calls. Actual results may vary by task, model version, deployment and evaluation method; the source provides no company-specific tests to show whether the 90% claim holds for those uses.
The cost figures are also incomplete. The server comparison excludes staffing, cooling and networking, and the source does not provide a full cost model for a particular enterprise’s workload. Token prices, hardware utilization and serving requirements differ. Nor does the fact that a model’s weights are downloadable mean its licence permits every commercial use. Kimi K3.1 Pro’s weights had not yet shipped as of the cited October 26 review, so its near-leader score could not then be treated as an available open deployment.
IBM’s commercial interest is relevant when weighing its claims: the company sells on-premises systems and expects demand to support its Spyre cards. The source also reports that IBM mainframe revenue fell 20% in the quarter, citing The Register. That context does not disprove Krishna’s cost argument, but IBM’s statements about model savings and its llm-d project should be distinguished from independent measurements.
Watch for Weights and Workload Data
The next near-term check is whether Moonshot releases Kimi K3.1 Pro’s weights by its promised November 4 date, and what licence accompanies them. The model’s benchmark score becomes relevant to in-house deployment only if the weights are accessible on terms that allow the intended use and an organization can support the required compute.
For companies considering a purchase, the next step is to compare candidate models on their own tasks and calculate costs at realistic utilization, including staffing, networking and cooling. Krishna’s remarks lay out IBM’s case for private infrastructure, but the source does not report a new IBM customer commitment, deployment result or independently verified savings figure. Those details will be needed to judge how often the claimed performance and price advantage translates into operating results.
Key Questions
What did IBM CEO Arvind Krishna claim about open-weight models?
Krishna said open-weight models can be 90% as effective as leading proprietary models at 10% to 25% of their token cost. He did not tie that percentage to a named benchmark in the remarks described by the source.
How close are the best downloadable models to proprietary leaders?
On the Artificial Analysis Intelligence Index reviewed October 26, 2026, MiMo-V2.6-Pro scored 46.3, compared with 57.6 for Claude Opus 5.5. The open model’s score was about 80% of the proprietary leader’s.
Does running AI in-house always cost less?
No. The cost depends on utilization and operating expenses. The source’s server comparison estimates a much longer payback at 25% utilization than at full load, and excludes staff, cooling and networking costs.
Is Kimi K3.1 Pro available to run as an open-weight model?
Not as of the October 26, 2026 review cited here. Moonshot had promised the weights by November 4, but they had not yet shipped. The licence and practical hardware requirements also matter.
Why might a company choose a private AI deployment?
Krishna said some banks and traders want more control over data and logs. A private deployment can support that goal, but the source does not establish that every on-premises system automatically satisfies a company’s security or regulatory requirements.
Source: ThorstenMeyerAI.com
