GPT-5.6 Sol's ARC result changes when the agent keeps its working memory

OpenAI has published a concrete reminder that GPT-5.6 Sol's measured ability can depend heavily on the agent around it. On ARC-AGI-3's public set, the company says Sol moved from 13.3% with the benchmark's official harness to 38.3% after its Responses API harness retained reasoning between turns and used context compaction. OpenAI also reports roughly six times fewer output tokens in that configuration. This is a harness result, not a new model or a change in the independent leaderboard result.

ARC Prize's published results provide the clean comparison point: GPT-5.6 Sol at Max is listed at 13.33% on the public set and 7.78% on semi-private tasks. Those figures match the starting point of OpenAI's public-set comparison rather than the new 38.3% configuration. The gap does not establish that either result is wrong; it shows that an agent benchmark reports the behavior of a model, API-state policy, and task harness together. A score from one stack should therefore not be presented as a model-only property.

Technically, the change is familiar to builders of multi-step agents. OpenAI explains that the official loop discarded the model's private reasoning after each action and dropped older actions with a rolling context window. Its alternate loop passes a prior response ID, preserving reasoning, and replaces truncation with compaction. That makes it less necessary for Sol to reconstruct a plan after every tool call. It also makes the result a test of stateful orchestration, so a stateless request-response integration should not assume it will reproduce the gain.

OpenAI explicitly recommends the Responses API, retained reasoning, and compaction for API developers. For a production test, separating those switches from model and reasoning-effort changes is the useful discipline: compare a current harness with a stateful version on the same task set and record both success rate and token cost. The company’s 38.3% and six-times-fewer-output-tokens claim is limited to public ARC-AGI-3 and is vendor-reported, so it is evidence to reproduce rather than an independent measure of general agent performance.

That limitation should temper price-performance arguments around Sol. The public evidence shows a higher result only with particular context management enabled, while ARC Prize's published semi-private score remains 7.78%. Neither source shows that the improvement carries to individual toolchains, customer data, or hidden tasks. The narrower practical conclusion is that, for long iterative Sol workflows, conversation state and context policy deserve the same test coverage as prompt and model selection.