🔍 Read the full analysis: The Mistral Large 4 Trade-Off: Global Appeal, Limited Fit For Agents on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Mistral Large 4 scored 38.4 on Artificial Analysis’s Intelligence Index v4.3.2, up from 9 for its predecessor, but below leading US and Chinese models. The source report argues that its benchmark position, comparatively high task cost and output volume, and observed hallucinations make it a difficult fit for long-running agents; some of those concerns are analysis or hands-on observations, not independently established findings.
Mistral has released Large 4, a multimodal model that scored 38.4 on Artificial Analysis’s Intelligence Index v4.3.2. That is a large rise from Mistral Large 3’s score of 9 on the same index version, but the reported result remains below leading US and Chinese models, raising questions about its fit for agentic tasks and its value at the listed price.
Large 4 has 1 trillion parameters, with 49 billion active, and accepts text and images while producing text. It has a 512,000-token context window. Mistral has made it available as a research preview through its API. The company says model weights are due at the end of October; until then, users access a proprietary model, and the licence for the planned weights has not been published.
Artificial Analysis’s index places Large 4 at 38.4 points, behind the listed US frontier models and several Chinese models. The table in the source report puts Claude Opus 5.5 at 57.6, Google’s Gemini 4 Argon at 52.6, GLM-5.3 at 44.8 and Kimi K3 at 43.6. Large 4 is close to DeepSeek V4.1 Flash, at 39.5, and ahead of DeepSeek V4 Pro 0813, at 36.0. These are scores on that index, not a universal measure of every use case.
The source report says Mistral Large 4 costs $1.36 per million input tokens and $4.18 per million output tokens, with cached input priced at $0.14 per million tokens. Mistral is offering a 50% discount for the first two weeks, according to the report. It also says Artificial Analysis measured $1.13 per Intelligence Index task for Large 4, compared with $0.25 for GLM-5.3-Flash and $0.27 for DeepSeek V4.1 Flash. The source does not provide a full account of task-cost calculation in the supplied material.
Mistral Large 4: best outside the US and China — and still not a model to run your agents on
The headline is true: France has the most intelligent model outside the US and China. The independent data says the rest: every US and Chinese flagship scores higher, the best by 19 points. It costs 4× more per task than Chinese open models that outscore it, and it’s 2.5× as verbose as the median model.
~two-thirds of Opus 5.5. Level with OpenAI’s small model, Luna.
Eighth among open models once weights ship — behind seven Chinese ones. Beats GLM-5.2 and V4 Pro, loses to their successors.
Cohere doesn’t compete at this tier — reported ~14% hallucination at ~9% accuracy, because it declines most questions. A field of one.
The Index is now agentic-heavy — Briefcase, GDPval, AutomationBench, Terminal-Bench. Errors multiply across steps: tolerable in chat, fatal over a two-hour run.
AA v4.3.2Output tokens to complete the Index. On an agent, verbosity is cost and latency on every step.
AAConfident false assertions in hands-on use. US frontier has largely moved past this — Gemini 4 Argon: 15%. In fairness Chinese open models are worse (Kimi K3 51%, DeepSeek V4 Pro 94%). In an agent, a fabrication is a wrong premise every later step builds on.
AUTHOR’S TESTING · not an AA figure- Cyber defence: 50 on the AA Cyber Index; 82% CyberGym-E2E (ahead of Luna’s 78%). Likely top-3 open model on cyber.
- Documents & images: 19% GDP.pdf (+18 vs Large 3); 100 images per request.
- Speed: 116 tok/s, 1.46s TTFT — well above median.
- The jump: Large 3 scored 9 on this Index. 9 → 38 is real progress.
- Jurisdiction: French parent, EU hosting, weights promised end of October.
- Legally bound buyers (defence, classified, DORA, health data): now the best European option by a wide margin. Wait for the weights, check the licence, pilot on cyber and documents.
- Everyone else, for agentic or long tasks: don’t. A US frontier model is meaningfully more capable; GLM-5.3-Flash is more capable and 4× cheaper.
- Note: Preview — Mistral says RL is still running, so scores may move. That changes next month’s decision, not today’s.
Mistral says it has “essentially closed the gap.” It has closed the gap to where the Chinese open-weights field was a few months ago, while that field and the US frontier have both moved on. On every independent measure that matters for agents — intelligence, cost per task, verbosity and factual reliability — Large 4 is not a frontier model. “Most intelligent outside the US and China” is true mainly because almost nobody else outside those two countries is competing. Use it if you have to. Don’t use it because of the headline.
The Cost of Running Agents
The index used in the report includes agent-oriented evaluations, such as knowledge work, software workflows and coding tasks. That makes its result relevant to companies considering models for systems that plan and execute multiple steps, though a single composite score cannot predict performance in every deployment.
The report’s author argues that a capability gap can compound over a long sequence: an error early in a workflow may shape later actions. The same report says Large 4 generated 200 million output tokens across the index, against a median of 81 million for comparable models. If reproduced in a customer’s workload, high output volume could add both latency and cost. These benchmark observations should not be treated as a guarantee of a particular company’s operating expense.
For buyers, the trade-off is between greater model capability and deployment cost, alongside questions of reliability. The report describes confident false statements in its own hands-on testing. That is an attributed observation rather than a published, controlled comparison, but it points to a practical risk: an unverified claim in an agent workflow can be carried into subsequent steps. Teams would need to test the model on their tasks and add suitable verification before assigning consequential work.
Products Worth Considering
LG UltraGear 27-Inch QHD OLED Gaming Monitor, 240Hz, 0.03ms, G-SYNC Compatible, FreeSync Premium Pro, HDMI 2.1
Logitech MX Mechanical Wireless Illuminated Keyboard, Tactile Quiet, Graphite
Bose QuietComfort Wireless Bluetooth Headphones with Active Noise Cancellation, Mic, USB‑C Charging, Deep Bass, 24‑Hour Battery – White Smok
As an affiliate, we earn on qualifying purchases.
A Major Jump From Large 3
The score of 38.4 marks a steep improvement over the 9 recorded for Mistral Large 3 and the 14 for Medium 3.5, according to the source report’s comparison using the same index version. It is evidence of substantial progress for Mistral, but it does not put Large 4 level with the highest-scoring models in the table. The report frames it as a significant advance for a European lab while noting that its index ranking remains behind several competitors.
Artificial Analysis’s Intelligence Index v4.3.2 combines several evaluations, including AA-Briefcase, GDPval-AA, AutomationBench and Terminal-Bench 4.0. The source says reinforcement learning on Large 4 is still underway, and Mistral has said scores may change. The result is therefore a snapshot of a preview model, not necessarily its final performance.
Preview Scores and Reliability
Large 4’s final benchmark performance is not settled. Mistral says reinforcement learning is continuing, and its weights and licence terms have not yet been released in the material provided. The promised end-of-October weights could allow broader scrutiny, but the release timing and licence details remain to be confirmed.
The source’s account of hallucinations comes from the author’s hands-on testing and does not specify a standardized test protocol, sample size or error rate for Large 4. The index scores and task-cost figures are attributed to Artificial Analysis, while the claim about confident false statements is the author’s own observation. It is not clear from the supplied material how performance varies by prompt, task type, or tool configuration, or whether the preview’s behavior will match the eventual weights.
Cost comparisons also depend on workload. The quoted per-task figures describe performance on Artificial Analysis’s index tasks; they do not establish the price of a typical customer workflow. Token mix, caching, task length, discounts and retries can change the result. Buyers should treat the reported prices and benchmark results as useful comparisons, not as a complete procurement forecast.
Weights and Further Testing
The next key milestone is the planned release of Large 4’s weights at the end of October. Mistral has not published the licence in the supplied material, so users will need to check the terms when they become available rather than assume the model will be open for any use.
Until then, the model remains a research preview on Mistral’s API. Mistral’s ongoing reinforcement learning could shift benchmark results, while independent tests and customer trials can clarify how it performs on specific workloads. Organizations considering it for agents can compare its price, output use and error rates against alternatives on their own tasks before making a broader deployment decision.
Key Questions
What is Mistral Large 4?
Mistral Large 4 is a text-and-image input model with a 512,000-token context window. The source report describes it as a 1-trillion-parameter model with 49 billion active parameters, available as a research preview through Mistral’s API.
How did Large 4 score against other models?
It scored 38.4 on Artificial Analysis’s Intelligence Index v4.3.2. That is up from 9 for Mistral Large 3 on the same index version, but below the leading US models and several listed Chinese models.
Is Large 4 open-weight now?
No. The source says it is currently a proprietary API preview. Mistral has promised weights for the end of October, but the licence was unpublished in the supplied material.
Why does the report question Large 4 for agents?
The report points to its index score, higher reported output-token use, task costs and the author’s observation of confident false statements. The hallucination observation is not presented as a standardized test result, and real-world suitability will depend on the task and safeguards.
How much does Mistral Large 4 cost?
The source reports standard pricing of $1.36 per million input tokens and $4.18 per million output tokens, with cached input at $0.14 per million. It also reports a 50% discount for the first two weeks; buyers should check Mistral’s current pricing before use.
Source: ThorstenMeyerAI.com
