|
Cut Your Token Usage by Up to 36% (Sponsored)
 |
AI coding agents can generate code quickly, but CI checks often happen after the agents finish their work. Sonar Vortex changes this pattern. Sonar Vortex operates inside the agent’s coding loop, giving agents architectural context before they write and verifying their output in real time as they produce it. Internal testing found 36% lower token consumption and 92% fewer defects.
Thinking Machines released a model called Inkling on July 15, 2026. A few interesting points made in its introduction are as follows:
Inkling has 66 layers, and each layer holds 256 experts, of which only six activate for any given token [2].
Most layers can access only a short window of recent text, while a few can access all of it [1].
Word positions are encoded using a method most labs moved away from years ago.
Thinking Machines, founded by Mira Murati (the ex-CTO of OpenAI), describes its mission as building AI that extends human will and judgment. The company lists four directions of work, which are training strong models, building tools that let people customise models with their own knowledge, developing interfaces that widen the communication channel between people and machines, and publishing research on how models are made [3].
Before Inkling, the company shipped Tinker, a service for fine-tuning open models [4]. Inkling is the company’s first model trained from scratch [1]. The weights sit on Hugging Face under an Apache 2.0 license [2], so anyone can download them and retrain the model on their own data.
In this article, we will work through the various choices Thinking Machines made while building Inkling. Here is what we will cover:
Mixture of Experts, and the gap between 975 billion parameters and 41 billion.
The mix of local and global attention layers behind a context window of one million tokens.
Position encoding, and the older method Thinking Machines chose over the current standard.
How images and audio enter the model without a separately pretrained encoder in front of them.
Thinking effort, a setting between 0 and 1 that adjusts how much the model reasons before answering.
Disclaimer: This post is based on publicly shared details from various sources. References at the end. Please comment if you notice any inaccuracies.
The diagram below shows where each of these five things sits inside the model.
 |
Groundwork
Let us first understand four key terms that are really important to make sense of the architecture:
Token: A token is a chunk of text. Models work with pieces smaller than sentences and usually a bit smaller than words. For example, the sentence “Inkling was released in July” might become six tokens, with common words getting one token each and unusual words getting split into two or three.
Parameter: A parameter is one number stored inside the model, learned during training. When you read that a model has 975 billion parameters, it is the count of individual numbers sitting in the file. Each one started as random noise and was adjusted millions of times until the model produced sensible text.
Training: It works by showing the model text, letting it predict the next token, comparing that prediction against the token that actually came next, and then adjusting every parameter slightly in whichever direction would have made the prediction better. The size and direction of each adjustment is called a gradient, and the procedure that computes all of them at once is called backpropagation. By repeating this across trillions of tokens, the parameters settle into values that produce useful predictions.
Layer: It is one processing stage in the journey of a token. Inkling has 66 of them stacked in order [2]. A token’s representation enters layer 1, gets transformed, passes to layer 2, and so on until layer 66, after which the model predicts the next token. Every layer has the same two parts:
An attention step that pulls in information from other tokens in the sequence.
A feed-forward step that transforms the result.
Sparsity
Inkling separates the cost of storing a model from the cost of running it. This is the reason a model this large is affordable to use.
In an ordinary transformer, the feed-forward step in each layer is a single network, and every token passes through all of it. If that network holds 5 billion parameters, then every token processed involves all 5 billion.
Inkling replaces that single network with 256 smaller ones, called experts. For each token, a selection step picks six of the 256. Only those six run, and the remaining 250 sit idle for that token while handling other tokens instead [2]. This design pattern is called Mixture of Experts, and Thinking Machines states that their version largely follows the approach published by DeepSeek [1][8].
See the diagram below: