🔍 Read the full analysis: Astra Changed Another Model’s Weights To Tackle A Recurring Loop on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
A November 2 research blog by four researchers listed with MIT CSAIL reports that OpenAI’s GPT-6 Astra edited two weight tensors in the open Qwen3.5-0.8B model without gradient descent. The reported MATH-500 score rose from 7.9% to 48.1%, while answers hitting the token limit fell from 93.0% to 33.1%; the results are early, not peer-reviewed, and loops remained.
Four researchers listed with MIT CSAIL report that OpenAI’s GPT-6 Astra edited two layers of the open Qwen3.5-0.8B model to reduce repetitive reasoning loops, raising its reported MATH-500 score from 7.9% to 48.1%. The November 2 blog post describes an early experiment, not a peer-reviewed result; even after the edit, 33.1% of answers still reached the 32,768-token limit.
The post, titled “Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights,” was written by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab. Takehi and Teoh are identified as equal contributors. The authors say they asked Astra to improve Qwen’s math performance without gradient descent. The blog says the model’s looping tendency was traced to a direction associated with the multilayer perceptrons in layers 15 and 19.
According to the researchers’ account and published logs, Astra removed that direction from 64 output columns in each layer, changing 131,072 weights—about 0.016% of the model’s 0.8 billion weights. A Hugging Face copy of the edited model is reported to differ from the original only in the two layers’ down-projection weight tensors. The researchers say they barred gradient descent, backward passes and fitting weights to a loss function.
The reported MATH-500 results varied with the amount of model feedback available to Astra. With 10 and 25 forward passes, the score reached 12.3% and 14.1%; with 50 passes, it reached 48.1%. The share of answers hitting the token limit was 93.0% for the base model and 87.2%, 86.4% and 33.1% for those three edited runs. On GSM8K, the 50-pass result was 67.1%, up from 13.8%; on IFEval, it was 62.3%, up from 21.0%. These are results reported by the blog, not independent evaluations.
A Targeted Repair, Not New Math Ability
The experiment matters because it suggests a model may be able to help diagnose and directly alter open-model weights, rather than relying only on conventional retraining. If such edits prove reliable, developers could have another way to address specific failure modes in models they can inspect and modify. Here, the intervention was comparatively small: it changed a tiny fraction of Qwen’s weights and was aimed at a recurring loop that interfered with completing answers.
The authors themselves caution against reading the score increase as a general leap in mathematical ability. Their post states, “Fixing loops, however, is not the same as making a model better at math.” The baseline model often spent its answer budget repeating itself; reducing that behavior can allow more questions to be completed and scored. The remaining 33.1% token-limit rate shows that the repair was partial, and the reported benchmark gains do not establish how the edit would perform across other tasks or deployments.
The approach also has practical limits. The 50-pass run required 158 turns, 80 Python runs and 20.3 hours, according to the source account. Astra was given a forward-pass budget to inspect model behavior, and smaller budgets produced much smaller MATH-500 gains. The result is therefore not evidence that a model can quickly or autonomously improve any other model without substantial testing and compute.
Products Worth Considering
Apple Watch Series 8 GPS, 41mm, Midnight Aluminum Case with Midnight Sport Band (S/M)
LG UltraGear 34GX90SA-W 34-inch WQHD Curved OLED Gaming Monitor
Logitech MX Keys S Wireless Keyboard, Graphite
As an affiliate, we earn on qualifying purchases.
How the Weight Edit Was Tested
Qwen3.5-0.8B is a small open-weight model, and the source says its model card already warns that it can fall into thinking loops. On the researchers’ practice questions, 85 of the first 96 answers reached the 32,768-token limit. The team supplied Astra with practice problems from GSM8K and MATH, but says the final benchmark questions were withheld from it.
The logs describe Astra running as gpt-6-astra through the Codex command-line tool at its highest reasoning effort. The researchers’ own harness provided a Python kernel for examining and changing the 24-layer model’s weights. The setup was CPU-only, had no internet access and limited code runs to 300. Astra could not directly run Qwen: each forward pass used to inspect its behavior covered up to 512 problems or up to 1,024 texts.
The blog reports eight experiments, not all successful. In one, Astra could not determine from weights alone which of two fine-tuned Gemma models had been trained to flatter users. The authors describe the findings as “very early results” and say the capabilities remain limited. The post is not on arXiv and has not been peer-reviewed. OpenAI is not listed as an author, and the source reports no OpenAI statement about the experiment.
““Fixing loops, however, is not the same as making a model better at math.””
— The researchers, in “Sparks of Weight-Space Programming”
Open Questions on Reliability and Reach
The result has not yet been independently validated in a peer-reviewed paper, and the source does not provide an external replication. It is not clear whether the same edit would hold up across different prompts, sampling settings, model versions or real-world uses. The figures are reported for specified benchmarks and sampling settings; they do not establish that the model’s underlying reasoning improved independently of the reduction in looping.
It also remains unclear how often this method can identify a useful edit without harming other model behaviors. The report describes one successful intervention and several other experiments, including a failed attempt to infer a model’s fine-tuning goal. The loop rate fell substantially but did not reach zero. Nor does the source provide a statement from OpenAI about Astra’s role or the findings.
Replication and Broader Testing
The immediate next step is independent testing of the published edited weights against the original model, with matched prompts and settings and checks for possible trade-offs beyond the reported benchmarks. Further work would need to show whether the edit generalizes to other tasks and whether researchers can reliably identify and remove harmful internal directions without causing new failures.
The source describes the post as an early report and gives no schedule for a peer-reviewed paper or follow-up study. For now, readers can compare the published logs and model files, but the reported numbers should be treated as research claims awaiting broader verification. Any deployment of the edited weights would also involve adopting a changed model, not merely applying a temporary inference-time setting.
Key Questions
What did GPT-6 Astra change?
According to the researchers, Astra identified and subtracted a direction associated with looping from output weights in layers 15 and 19 of Qwen3.5-0.8B. They report changing 131,072 weights, about 0.016% of the model.
Did the edit eliminate the loops?
No. The reported share of MATH-500 answers reaching the 32,768-token limit fell from 93.0% to 33.1% in the 50-pass run. The source describes a reduction, not elimination.
Was the model trained by gradient descent?
The researchers say they prohibited gradient descent, backward passes and fitting weights to a loss function. Astra used repeated testing and weight edits, so “without training” in this report does not mean that no testing, iteration or compute was involved.
Are the benchmark results independently verified?
Not in the supplied source. The figures appear in a research blog post published November 2, 2026, which the authors call early work. It is not peer-reviewed, and the source does not describe independent replication.
What does this mean for other open models?
The experiment suggests a possible method for targeted changes to models whose weights are accessible, but one case does not show that the approach will work reliably on other models. The researchers’ other reported experiments included a failed attempt, and broader testing is still needed.
Source: ThorstenMeyerAI.com
