GPT-5.6 Sol's Chat Update Makes the Harness Part of the Product

OpenAI's August 6 update changes how GPT-5.6 Sol behaves in ChatGPT for Plus and Pro users: the company says it has tuned the model for more focused answers and better factual reliability, and added a slider that lets users choose how much thought an answer receives. The scope is important. OpenAI explicitly says this optimized Sol version is available only in ChatGPT, while the GPT-5.6 Sol version used by Work and Codex is not changing. This is therefore a product-behavior update, not evidence of a new API-model release or a blanket change to every Sol deployment.

The company frames the change around conversational usefulness rather than a new capability claim. It says Sol should adapt detail to the question, reduce unnecessary formatting, and provide a correction when agreement would not be useful. OpenAI also reports that, in its internal evaluation of factual financial, medical, and legal prompts, responses with at least one factual error were 68% less common for Sol than for GPT-5.5 Instant. That result is useful directionally, but it is vendor-reported and tied to OpenAI's own evaluation set, so it should not be read as an independent measure of production accuracy.

A related OpenAI research report shows why the update cannot be separated cleanly from the surrounding harness. On ARC-AGI-3's public set, OpenAI says GPT-5.6 Sol rose from 13.3% with the benchmark's official harness to 38.3% after retained reasoning and context compaction were enabled. ARC Prize describes ARC-AGI-3 as an interactive benchmark for agents that must learn novel environments over time, with long-horizon planning and adaptation as central features. In that setting, preserving state is not cosmetic: it changes whether an agent can build on prior observations.

OpenAI attributes the gap to two mechanics in the generic benchmark harness: private reasoning was discarded after each action, and older actions disappeared as a rolling context window filled. Its Responses API guidance describes compaction as a way to reduce context size while carrying forward necessary prior state and reasoning into later turns. The comparison is a reminder that an agent benchmark measures a model-plus-orchestration system, not only the underlying weights. It also means the 38.3% figure is not directly comparable to a score produced under a different memory policy.

For technical users, the practical conclusion is narrower than a claim that Sol has become universally better. Do not extrapolate the ChatGPT tuning to Work, Codex, or an existing API integration, because OpenAI says those Sol deployments were not changed by this update. For long-running tool workflows, test retained state and compaction with the exact model, tools, prompts, and failure modes in use; OpenAI recommends the Responses API, reasoning retention, and compaction, but its ARC result is still a provider-authored experiment. Independent, like-for-like evaluations of these harness choices remain the missing evidence.