# Cost per Token:Why Closed-Model Economics Are Pushing Enterprises Toward OpenWeights
> Alex Karp says the OpenAI–Anthropic token model has
failed. We compare real inference costs, the hidden IP cost, and what enterprise teams
should do next.
- URL: https://innergate.ai/blog/cost-per-token-openweights
- Published: 2026-07-16 15:47:00+00
- Updated: 2026-07-23 11:04:18.966363+00
- Keywords: cost, model, models, data, task, tokens, cloud, token, open, what
---
**Short answer:** Cost per token has become the deciding variable in enterprise AI architecture. On sustained workloads, self-hosted inference with open-weight models costs between 0.001 and 0.04 USD per million tokens in electricity, versus 2.50–15.00 USD per million tokens on equivalent cloud APIs. As autonomous agents become standard — multiplying token consumption per task — the AI invoice has stopped being marginal and started being volatile. And the invoice is only the visible half: Karp's sharper warning is the IP cost — proprietary data, prompts, and accumulated know-how flowing out of the enterprise and toward the model provider. His CNBC comments on July 1, 2026 gave media voice to a shift the data already showed: enterprises are abandoning tokenmaxxing and choosing the right model for the right task at the right cost — increasingly an open-weight model, running inside their own perimeter, where both costs are eliminated by construction.

* * *

### What is the token-based business model — and why is it under fire?

The token model is the billing scheme in which AI usage is charged by the volume of text processed and generated, measured in tokens. It is the dominant commercial model of the large AI labs: the customer buys neither software nor capacity — it buys consumption.

> Definition — token: the elementary unit of text that language models use to process and generate content (word fragments; roughly 4 characters per token in English). On cloud APIs, it is also the billing unit: you pay per million input and output tokens. One million tokens corresponds to roughly 750,000 words.

While AI was experimentation, the token invoice was a footnote. In production — with RAG pipelines injecting context into every request and agents iterating dozens of times per task — it has become one of the hardest budget lines to forecast. That is the context for Karp's intervention.

### What exactly did Alex Karp say about OpenAI and Anthropic?

**Answer:** On CNBC's Squawk Box (July 1, 2026), Palantir's CEO criticized OpenAI's and Anthropic's token revenue model, summing it up as "something has gone completely wrong."

According to CNBC's report, Karp described the mindset that has taken hold in enterprises as one of burning through tokens without extracting measurable value — while risking, in the process, the transfer of their intellectual property to third parties. He also named what technically sophisticated customers want instead: control over their compute, their models, their data stack, and their competitive edge. In Karp's view, open-weight models are the answer for business leaders frustrated with the frontier labs. The market reacted: Palantir shares rose about 8% that day.

Two reading notes. First, Palantir is not a neutral observer: it sells data and AI platforms, and days earlier it had announced an expanded partnership with Nvidia to build custom models for US government agencies — the critique serves its positioning. Second, and this is the point that matters here: the underlying economic argument does not depend on Karp. It shows up on the invoice.

### What is tokenmaxxing — and why has it become a budget problem?

**Answer:** Tokenmaxxing is the practice of maximizing AI token consumption without measuring the return per completed task — treating token volume as a proxy for progress. The term established itself in 2026 mostly for the wrong reasons. Microsoft's own CEO, Satya Nadella, publicly acknowledged the pull of the habit while urging employees to avoid frontier models for problems that are not frontier problems, against a backdrop of climbing infrastructure costs. Investor Chamath Palihapitiya quantified the gap: he estimated an enterprise could pay around 105,000 USD per month for GPT-5.5 Pro versus around 2,740 USD per month for an open model that is equivalent on many tasks, calling the pricing gap between open and closed models enormous. And US press reported companies including Uber and Microsoft capping internal access to AI coding tools after budget overruns. The structural reason is architectural: agentic workflows multiply model calls. A task is no longer one prompt — it is a cycle of planning, execution, verification, and correction, each iteration carrying context. An illustrative example, with conservative values:  

| Parameter (illustrative example) | Value |
| --- | --- |
| Agent iterations per completed task | 12 |
| Average context per iteration | 10,000 tokens |
| Consumption per task | ≈ 120,000 tokens |
| Cost per task on a cloud API (at 10 USD/M tokens) | ≈ 1.20 USD |
| Cost per task self-hosted (at 0.02 USD/M tokens, electricity) | ≈ 0.002 USD |
| 1,000 tasks/day, 30 days — cloud API | ≈ 36,000 USD/month |
| 1,000 tasks/day, 30 days — self-hosted (electricity) | ≈ 72 USD/month |

The agentic multiplier turns cent-level differences per million tokens into differences of tens of thousands of euros per month. That is why the relevant metric is no longer the list price per token — it is the cost per completed task.

### What does a million tokens actually cost — cloud API vs. self-hosted?

**Answer:** On sustained workloads, market data points to 2.50–15.00 USD per million tokens on cloud APIs, versus 0.001–0.04 USD in electricity on a self-hosted, open-weight deployment.

> Definition — open weights: models whose weights (the trained parameters) can be downloaded and run on your own infrastructure — for example Llama, Mistral, Qwen, or DeepSeek. They are distinct from fully open-source models (which would also publish training data and code) and from closed models, accessible only through an API. Once deployed on-premise, an open-weight model has no per-token marginal cost charged by a third party: the cost of inference is energy, hardware amortization, and operations.

| Component | Cloud API (closed model) | Self-hosted (open weights) |
| --- | --- | --- |
| Marginal cost per 1M tokens | 2.50–15.00 USD | 0.001–0.04 USD (electricity) |
| Hardware | Included in the pertoken price | Own CAPEX; at production volume, amortized within months |
| Budget predictability | Varies with consumption | Essentially fixed after the investment |
| Deployment control | The provider's | The organization's |
| Data location | The provider's cloud | Your own perimeter |

Component Cloud API (closed model) Self-hosted (open weights) Marginal cost per 1M tokens 2.50–15.00 USD 0.001–0.04 USD (electricity) Hardware Included in the pertoken price Own CAPEX; at production volume, amortized within months Budget predictability Varies with consumption Essentially fixed after the investment Deployment control The provider's The organization's Data location The provider's cloud Your own perimeter Three honest caveats belong in this comparison. First, the 0.001–0.04 USD figure is electricity only — total cost of ownership (TCO) includes hardware, operations, and redundancy; even so, at sustained volume the gap remains one of orders of magnitude. Second, self-hosting only pays off with utilization: for sporadic loads or unpredictable spikes, the cloud API remains the rational choice. Third, absolute frontier capability still sits, today, in closed models — the question is what fraction of your real workload actually requires it. And performance? Open-weight inference on dedicated hardware is now productiongrade: in our internal benchmarks of the Gluteus server (vLLM stack), we measured 1,560 tokens/s of throughput with an average TTFT of 709 ms — numbers and methodology published on our technical page.

### What does "the right modelfor the right task at the right cost" mean?

**Answer:** It means routing architecture: a component that classifies each request and sends it to the cheapest model capable of solving it at the required quality.

> Definition — LLM routing: the architecture layer that distributes requests across multiple models — small, mid-size, frontier — based on task complexity, cost, and latency and data-locality requirements. The goal is to optimize cost per task without degrading quality where quality is critical.

A substantial share of enterprise tasks — classification, data extraction, summarization, question answering over internal documentation via RAG — can be solved by open models in the 7–70 billion parameter range, at a fraction of the cost. Frontier models are reserved for the complex reasoning that genuinely requires them. In a system with routing, tokenmaxxing becomes impossible by construction: every expensive call has to justify itself. That is the correct reading of the line that closes the argument — AI architecture is no longer just about quality; it is about cost per task, deployment control, and scalability.

### **Beyond the invoice: what is the "IP cost" of consuming closed models via API?**

**Answer:** The IP cost is the transfer of proprietary data, prompts, and accumulated knowhow to the model provider — a cost that never appears on the invoice. This is the sharper half of Karp's critique, and the part most coverage buried under the pricing story. Tokens are only the visible expense; his warning was that enterprises end up on the losing side of the trade twice — as he put it, "I'm going to get no value and they're going to get my IP." In his account, executives privately concede that token spending yields little, while engaging with the labs exposes the data and know-how their market advantage is built on. Palantir's nine-point manifesto, published the day before the interview, states the mechanism plainly: weights are the distilled form of an institution's hard-won knowledge — "controlling your weights is controlling your fate" — and letting a third party control them means letting the alpha of your business migrate to theirs.

> Definition — alpha (in this context): an organization's compounding competitive advantage: proprietary data, domain know-how, refined processes and workflows. The claim against pure API consumption is that this advantage leaks outward — the provider's models and products get better at your business faster than your own institutional capability does.

How does the leak happen in practice? Mostly not through training on your data — the major labs contractually commit not to train on enterprise API traffic by default. The exposure is architectural rather than merely contractual: every call ships context outside your perimeter (prompts, retrieved documents, source code, customer records); your best prompt libraries, evaluation suites, and fine-tunes accumulate as assets on infrastructure you do not control; and the protections that do exist are terms of service — revocable, renegotiable, jurisdiction-dependent — rather than properties of the network topology. And the endgame fear, that a lab could end up competing with its own customers, is no longer treated as hypothetical in the public debate: investor David Sacks, endorsing Karp's warning, pointed to Figma as his example, arguing the company was blindsided when Anthropic launched a competing design tool after one of its executives had served on Figma's board. Whatever the merits of that particular account, it is now a scenario boards are pricing in. Note the shape of the fix — including Palantir's own. The expanded Palantir–Nvidia deal reportedly pairs Nvidia's open Nemotron models with deployments inside US government agencies and critical infrastructure, in classified and air-gapped settings. In other words: even the loudest critic of the token model answers it with open models running inside the customer's perimeter. On-premise, open-weight deployment closes the IP channel by construction — prompts, embeddings, evaluation data, and fine-tuned weights stay on your infrastructure, the know-how compounds internally, and no contractual clause has to do work that network topology can do instead. What Palantir-style deployments have long sold to governments, open weights now make available to any enterprise with sustained workloads.

### How does this connect to Europe's AI sovereignty agenda?

**Answer:** Directly: the private cost and IP arguments and the public control argument converge on the same architecture — open models, run on infrastructure under your own jurisdiction. Palantir's manifesto frames the IP argument as a matter of sovereignty — weights and data as assets an institution must keep under its own control. Six days after it was posted, on July 7, 2026, the European Commission published its Action Plan on Cybersecurity and Artificial Intelligence \[COM(2026) 577 final\], which points to already-available open models as an immediate basis for cyber resilience and calls for the development of sovereign European capability, warning against dependence on frontier systems built in other jurisdictions. On the demand side, Gartner forecasts that sovereign cloud spending in the EU will more than triple between 2025 and 2027, and Accenture estimates that 62% of European organizations are actively looking for sovereign AI solutions. None of this dictates what any given organization is obliged to do — that is a question for its legal advisers. What is an engineering matter is what the architecture enables: data locality inside your own perimeter, independence from any single vendor, and marginal inference cost under your own control.

### What should technical and finance teams do now?

**Answer** — four practical priorities:

1\. Measure cost per task, not per token. Instrument your current workloads: input and output tokens per completed task, including every agent iteration and all RAG context. Multiply by the list price and compare against your current invoice — it is the only honest starting point. Our cost-per-token calculator runs this comparison with your own inputs, on-premise vs. cloud API, over 3 years.

2\. Pilot open weights on sustained, sensitive workloads. Pick a use case with stable volume (classification, extraction, internal RAG), evaluate an open model on your own infrastructure, and measure quality per task against cost. The decision stops being ideological and becomes a benchmark.

3\. Design routing from day one. Default to the smallest model that solves the task; escalate to larger models only where quality demands it; reserve cloud APIs for spikes and for frontier capability. Portability across models — open and closed — is the best protection against single-vendor lock-in.

4\. Treat prompts, evals, and fine-tunes as IP assets. Inventory which workloads ship proprietary data outside your perimeter, and which model artifacts — prompt libraries, evaluation sets, fine-tuned weights — are accumulating on infrastructure you don't control, and under what terms. Route the alpha-bearing workloads to controlled infrastructure first, and keep every artifact portable. Karp's line is for the headlines; the decision is for the spreadsheets. Model quality is no longer the only axis of choice: the winners will not simply be the companies using the biggest model, but the ones running the right model, for the right task, at the right cost — able to prove it on the invoice, and not paying for it in IP.

* * *

### Sources

**News coverage (interview and reactions):**

-   CNBC, "Palantir's Karp bashes OpenAI, Anthropic token model: 'Something has gone completely wrong'", July 1, 2026 — [https://www.cnbc.com/2026/07/01/palantir-karpopen-ai-anthropic-tokens.html](https://www.cnbc.com/2026/07/01/palantir-karpopen-ai-anthropic-tokens.html)
    
-   Benzinga, "Alex Karp Bashes OpenAI, Anthropic Token Model", July 2, 2026 (Satya Nadella's remarks and Chamath Palihapitiya's estimates) — [https://www.benzinga.com/markets/tech/26/07/60239308/alex-karp-bashes-openaianthropic-token-model-something-has-gone-completely-wrong](https://www.benzinga.com/markets/tech/26/07/60239308/alex-karp-bashes-openaianthropic-token-model-something-has-gone-completely-wrong)
    
-   Yahoo Finance, coverage of the interview and enterprise context (internal caps on AI coding tools; Palantir–Nvidia Nemotron deployments in classified and air-gapped settings), July 2026
    
-   Benzinga, coverage of David Sacks's remarks on the All-In Podcast (proprietary data risk; the Figma example), July 2026
    
-   Palantir, nine-point "AI sovereignty" manifesto, published on X on June 30, 2026
    

**Regulation and market data:**

-   European Commission, Action Plan on Cybersecurity and Artificial Intelligence, COM(2026) 577 final, July 7, 2026
    
-   Gartner, Predicts 2026: AI Sovereignty (EU sovereign cloud spending, 2025–2027)
    
-   Accenture (sovereign AI adoption among European organizations) Self-hosted vs. cloud API inference costs: Silicon Data, Featherless.ai, CloudZero, [VDF.ai](http://VDF.ai)
    

**First-party data:**

-   Innergate internal benchmarks — Gluteus server, vLLM stack (1,560 tok/s; average TTFT 709 ms)