AIThis post was created with the assistance of artificial intelligence (AI).

🔍 Read the full analysis: Welcome Inkling 2 Mini, Cutting-edge Performance For Your Local Setup on ThorstenMeyerAI.com

Before you orderOffer from Amazon

Get the latest gadgets delivered free with Prime

  • Fast, free delivery on millions of items
  • Prime Video, Amazon Music and more included
  • Member-only deals all year
Start your free Prime trial Free trial for eligible customers · Cancel anytime
As an affiliate, we earn on qualifying purchases.

TL;DR

Thinking Machines Lab has released Inkling 2 Mini, a multimodal mixture-of-experts model the company describes as having about 26 billion total parameters and activating about 3 billion per token. The release includes support for several inference runtimes and a research version with adjustable layer passes. Its benchmark claims, hardware needs and real-world performance have not been independently established in the supplied material.

Thinking Machines Lab has released Inkling 2 Mini, a multimodal model intended for local AI setups that the company describes as having about 26 billion total parameters, with roughly 3 billion active per token. The release also includes support for several inference tools and a research checkpoint that allows users to change how many times part of the model runs; the company’s performance claims have not been independently verified in the material provided.

According to the release, Inkling 2 Mini accepts text and image input and has a claimed 1 million-token context window. Its architecture uses 24 physical decoder layers. The first six and final six run once, while 12 middle layers are reused for three passes. The company describes this as 48 logical layer visits per token, with shared weights in the repeated section rather than a separate set of weights for every visit.

The model is a mixture of experts: at each layer visit, a routing system selects eight experts from a pool of 128. Thinking Machines says it is distributing expert weights in NVFP4 and MXFP8 formats, and includes a DSpark drafter intended to speed generation with a small memory cost. These are details of the release, not independently measured results.

The fixed-pass model is listed as supported by Transformers, llama.cpp and MLX; the release also names SGLang for day-one inference support in datacenter deployments. A separate research checkpoint, TML-d24-varloop-research, permits fewer passes through the repeated layers. The company says reducing passes may improve speed, with a possible trade-off in quality.

At a glance
announcementWhen: Announced in the source material; publi…
The developmentThinking Machines Lab has released Inkling 2 Mini, with multiple inference-runtime options and a research checkpoint that can vary the number of repeated-layer passes.
AI Models · Briefing

Inkling 2 Mini for Local AI

Thinking Machines Lab’s new multimodal model combines reused layers with expert routing. Its practical speed, hardware needs and benchmark results still need outside validation.

26B
Total parameters, about
3B
Parameters active per token, about
1M
Token context window

A loop reuses model layers

The design reuses middle-layer weights to build capacity, while routing selects part of the expert pool at each layer visit.

1
Start with 24 layers

The first six and last six physical decoder layers run once.

2
Repeat the middle

The same 12 middle layers run in three passes, for 48 logical layer visits per token.

3
Route to experts

At each visit, the model selects eight experts from a pool of 128.

Options for inference

The release offers several runtimes and a research checkpoint with adjustable loop passes.

Fixed model

Multiple runtimes

Transformers, llama.cpp and MLX support the fixed-pass model; SGLang supports datacenter inference.

Research variant

Fewer passes

The research checkpoint allows one or two passes. The company says this can be faster, with a possible quality trade-off.

Claims need outside checks

The company reports strong benchmark comparisons, but the supplied material does not include independent replication or a full score table.

Reported results
  • Leads in 32 of 48 benchmarks against named models.
  • Reported tie with Qwen3.8-27B on Terminal-Bench 4.0.
  • Trails on AutomationBench-AA v1.0.6.
Still unclear
  • Complete memory requirements and device-specific speed.
  • Independent image and long-context performance.
  • Quality at reduced loop counts.

What local users should assess

Runtime availability is a starting point; actual usefulness depends on hardware and workload.

Memory useGeneration speedImage handlingLong-context behaviorOutput quality
CaveatThe release does not specify a complete file size or minimum hardware configuration.
The takeaway
Inkling 2 Mini brings an interesting local inference design, but independent tests and hardware-specific measurements will show whether its reported promise holds in practice.

What Local Model Users Gain

Inkling 2 Mini adds another option for people choosing models to run on their own equipment, and its multiple runtime integrations could make it easier to test across different software setups. The combination of image input and a claimed million-token context window may also appeal to users who want to process more than short text prompts. Whether those features work well in practice remains a matter for testing.

The architecture is designed to increase capacity without assigning unique weights to every logical layer, while routing only a subset of experts for each token. That design may affect compute and storage trade-offs, but it does not by itself establish memory use, generation speed or device compatibility. Buyers and developers will need measurements on their own hardware before treating the model as a practical fit for a local setup.

The company’s reported benchmark leads could draw attention to the model, but their significance is limited without full scores and independent replication. For now, the announcement establishes availability and technical claims, not a verified performance advantage over alternatives.

Products Worth Considering

As an affiliate, we earn on qualifying purchases.

How the Repeated Layers Work

The central design choice is to run the same 12 middle physical layers three times. The six layers at the beginning and end run once, giving the company’s stated total of 48 logical layer visits per token. According to its technical description, learned messages carry information between passes, while a learned gate controls how much information is passed forward. The repeated layers still have to execute on each pass, and the source says each visit retains its own key-value and short-convolution state.

Thinking Machines compares Mini with its Inkling 2 Small model. Both are described as executing 48 layer visits per token, but Mini has a 2,304 hidden width versus 4,608 for Small and routes among 128 experts per layer rather than 384. The material characterizes Mini as substantially smaller but gives neither a complete file size nor a minimum hardware configuration. The release also points to earlier exploration of looping architectures, including Nanbeige; that reference does not establish how the approaches compare in performance.

For its benchmark claims, Thinking Machines says Inkling 2 Mini is state of the art in its class on the Artificial Analysis Intelligence Index v5 and leads in 32 of 48 comparisons with Gemma 4 31B, Muse Glimmer, Qwen3.8-27B and Nemotron 3.5 Super VL. It also reports a lead on FrontierCode v2.0, a tie with Qwen3.8-27B on Terminal-Bench 4.0, and a lower result on AutomationBench-AA v1.0.6. These comparisons are company-reported; the supplied material does not include the full score table or outside confirmation.

“Inkling 2 Mini is a mixture-of-experts model with 24 decoder layers, but the 12 middle layers run three times in a loop.”

— Thinking Machines Lab, in the supplied release material

Hardware and Benchmark Questions

The supplied material does not establish minimum memory requirements, measured generation speeds on consumer devices, or how the model performs across specific local hardware. It also does not provide independent evaluations of image handling or the claimed long-context behavior. A model’s parameter count and architecture alone are not enough to determine whether a particular computer can run it comfortably.

The company reports benchmark standings, but the source does not include a complete score table or independent replications. It is also unclear how much quality changes when the research checkpoint is run with one or two passes, or how those settings affect speed in real use. The material refers to a full model card, but that document was not supplied, so additional technical and evaluation details could not be checked for this article.

Independent Tests Will Clarify Fit

The next useful evidence would be the full model card, reproducible benchmark results and reports from users testing the supported runtimes on specified hardware. Those tests could show how much memory the model requires, whether its generation speed is practical, and how performance changes across image, coding and long-context tasks.

Testing of the variable-pass research checkpoint will also help establish whether fewer passes produce a useful speed improvement and what quality cost, if any, follows. Until those details are available, the release confirms the model’s stated design and software support, while its local hardware fit and comparative performance remain open questions.

Key Questions

What is Inkling 2 Mini?

Inkling 2 Mini is a multimodal mixture-of-experts model released by Thinking Machines Lab. The company says it supports text and image input and has about 26 billion total parameters, with about 3 billion active per token.

Which inference tools does it support?

The release lists the fixed-pass model for Transformers, llama.cpp and MLX, and names SGLang for day-one inference support in datacenter deployments. Actual compatibility and performance may depend on the software version and hardware.

Can users change the number of layer passes?

The separate TML-d24-varloop-research checkpoint allows the number of passes through the repeated middle layers to be changed. Thinking Machines says fewer passes may speed inference but could affect quality; the supplied material gives no independent measurements.

How much memory does Inkling 2 Mini need?

The source material does not give a minimum hardware configuration or complete memory requirements. The stated parameter count is not, on its own, enough to determine the memory needed for a particular runtime or setup.

Have the benchmark claims been independently verified?

Not in the material provided. The benchmark standings are reported by Thinking Machines Lab, and the source does not include a full score table or independent replication.

Primary source: Hugging Face · via ThorstenMeyerAI.com

You May Also Like

The Tension Between Mistral’s AI Goals And European Sovereignty

Mistral faces tensions between its rapid growth and the challenge of maintaining European data sovereignty amid global AI competition.

The adder at the heart of Intel’s 8087 floating-point chip

An in-depth look at the 69-bit adder at the core of Intel’s 8087 floating-point coprocessor and its significance in computing history.

OpenAI Dev Day 2026: How To Read The Latest Announcements

OpenAI’s DevDay on 29 September 2026 pushed ChatGPT from chatbot to agent platform. Here are the five announcements builders should read closely.

The AI Work Revolution Without The 15-Hour Week

A 2026 essay compares Keynes’s century-old leisure prediction with income growth, the 40-hour workweek and early AI time-savings.