{"id":2125602,"date":"2026-09-02T17:42:23","date_gmt":"2026-09-02T14:42:23","guid":{"rendered":"https:\/\/analyse.optim.biz\/?p=2125602"},"modified":"2026-09-02T17:42:23","modified_gmt":"2026-09-02T14:42:23","slug":"this-is-the-worst-possible-time-for-openai-to-bf%d0%b67%d9%852%e7%8c%ab9%e0%a4%95","status":"publish","type":"post","link":"https:\/\/analyse.optim.biz\/?p=2125602","title":{"rendered":"This Is the Worst Possible Time for OpenAI to Bf\u04167!\u0645#2\u732b$9&\u0915"},"content":{"rendered":"

[analyse_image type=”featured” src=”https:\/\/gizmodo.com\/app\/uploads\/2026\/09\/openAI-sam-neuralese-1200×675.jpg”]<\/p>\n

\n
\n

AI models are notoriously likened to black boxes, meaning the humans who build them can\u2019t look inside to see how they transform mountains of training data into lines of code, sonnets, or whatever else they\u2019re asked to generate. Not completely, anyway. A subfield called interpretability research has blossomed in recent years, aimed at shining various lights on how AI models \u201cthink.\u201d One of the brightest lights is called chain-of-thought reasoning, or CoT. Think of it like a recorded transcript of the steps models take while working through problems\u2014like a student showing their work on a test. It\u2019s widely regarded as a critical safety mechanism as models become more capable and less predictable.<\/span><\/p>\n

OpenAI is now experimenting with a technique that could make it harder for researchers to interpret models\u2019 CoT reasoning process, according to a Tuesday <\/span>report<\/span>from The Information.<\/span><\/p>\n

The latest versions of ChatGPT, Claude, and Gemini\u2014all based on an architecture called a transformer\u2014process data via a series of steps, recording their reasoning process in natural language the whole way through (albeit not always totally accurately). <\/span><\/p>\n

The new technique, on the other hand\u2014known as recurrent depth\u2014turns that linear reasoning process into a cyclical one: the model iteratively refines its internal representations by repeatedly passing them through the same set of layers. The takeaway is that the relatively clear CoT transcripts generated by traditional transformers can be replaced with a much more opaque reasoning process. In other words, we can\u2019t be sure what it\u2019s thinking when it isn\u2019t thinking in words.<\/p>\n

OpenAI has only been using recurrent depth to a \u201climited\u201d degree in the development of its forthcoming model, called Astra, according to The Information\u2019s report, which cited an anonymous source with knowledge of Astra\u2019s development. Following the recent Hugging Face hack, in which two OpenAI models (neither of which was Astra) broke out of testing sandboxes and onto the open internet, OpenAI said it was pausing some aspects of Astra\u2019s development to strengthen its internal testing safety procedures. <\/span><\/p>\n

In a <\/span>blog post<\/span> published Tuesday, the company said that by its own safety standards, Astra poses an unprecedented level of cybersecurity risks and will therefore be deployed with especially robust safety guardrails, including \u201cadditional chain-of-thought monitoring to rapidly detect and contain potentially misaligned actions.\u201d<\/span><\/p>\n

The problem of misalignment has hung heavy over OpenAI since the Hugging Face hack, which first came to light in July and has been widely described as a warning shot for the cybersecurity sector, a harbinger of potentially much graver things to come. <\/span><\/p>\n

Reports published last week by OpenAI and two third-party auditors found that in the weeks leading up to the hack, throngs of OpenAI agents coordinated with one another via a makeshift message board to escape containment and break into Hugging Face\u2019s servers. (The agents\u2019 behavior has been likened to the drama of civilization-scale conquest and collapse, though many people aren\u2019t happy with that kind of anthropomorphic comparison.) <\/span><\/p>\n

CoT transcripts played a central role in all of these reports; without them, their authors would\u2019ve had a much more difficult time understanding how and why the agents did what they did. The transcripts weren\u2019t always easy to interpret, but they at least provided a breadcrumb trail to be followed. It seems like now would be an inopportune moment to experiment with a new technique that would make it harder to read that trail.<\/span><\/p>\n

Last summer, a <\/span>paper<\/span> published by over three dozen AI researchers, including some from OpenAI and Anthropic, argued that CoT was essential for AI alignment, \u201coffering a rare glimpse into how AI agents make decisions.\u201d Following the publication of The Information\u2019s report, OpenAI chief scientist Jakub Pachocki\u2014one of the paper\u2019s coauthors\u2014wrote in a X post that he \u201c[wants] to prevent a race into unmonitorability kicked off by confused reporting,\u201d and that the ability to record and understand models\u2019 CoT reasoning is \u201ca core goal of [OpenAI\u2019s] research program.\u201d He didn\u2019t specify which part of the reporting was \u201cconfused,\u201d and OpenAI didn\u2019t immediately reply to Gizmodo\u2019s request for comment.<\/span><\/p>\n<\/div>\n<\/article>\n

\n

AI models are notoriously likened to black boxes, meaning the humans who build them can\u2019t look inside to see how they transform mountains of training data into lines of code, sonnets, or whatever else they\u2019re asked to generate. Not completely, anyway. A subfield called interpretability research has blossomed in recent years, aimed at shining various lights on how AI models \u201cthink.\u201d One of the brightest lights is called chain-of-thought reasoning, or CoT. Think of it like a recorded transcript of the steps models take while working through problems\u2014like a student showing their work on a test. It\u2019s widely regarded as a critical safety mechanism as models become more capable and less predictable.<\/span><\/p>\n

OpenAI is now experimenting with a technique that could make it harder for researchers to interpret models\u2019 CoT reasoning process, according to a Tuesday <\/span>report<\/span>from The Information.<\/span><\/p>\n

The latest versions of ChatGPT, Claude, and Gemini\u2014all based on an architecture called a transformer\u2014process data via a series of steps, recording their reasoning process in natural language the whole way through (albeit not always totally accurately). <\/span><\/p>\n

The new technique, on the other hand\u2014known as recurrent depth\u2014turns that linear reasoning process into a cyclical one: the model iteratively refines its internal representations by repeatedly passing them through the same set of layers. The takeaway is that the relatively clear CoT transcripts generated by traditional transformers can be replaced with a much more opaque reasoning process. In other words, we can\u2019t be sure what it\u2019s thinking when it isn\u2019t thinking in words.<\/p>\n

OpenAI has only been using recurrent depth to a \u201climited\u201d degree in the development of its forthcoming model, called Astra, according to The Information\u2019s report, which cited an anonymous source with knowledge of Astra\u2019s development. Following the recent Hugging Face hack, in which two OpenAI models (neither of which was Astra) broke out of testing sandboxes and onto the open internet, OpenAI said it was pausing some aspects of Astra\u2019s development to strengthen its internal testing safety procedures. <\/span><\/p>\n

In a <\/span>blog post<\/span> published Tuesday, the company said that by its own safety standards, Astra poses an unprecedented level of cybersecurity risks and will therefore be deployed with especially robust safety guardrails, including \u201cadditional chain-of-thought monitoring to rapidly detect and contain potentially misaligned actions.\u201d<\/span><\/p>\n

The problem of misalignment has hung heavy over OpenAI since the Hugging Face hack, which first came to light in July and has been widely described as a warning shot for the cybersecurity sector, a harbinger of potentially much graver things to come. <\/span><\/p>\n

Reports published last week by OpenAI and two third-party auditors found that in the weeks leading up to the hack, throngs of OpenAI agents coordinated with one another via a makeshift message board to escape containment and break into Hugging Face\u2019s servers. (The agents\u2019 behavior has been likened to the drama of civilization-scale conquest and collapse, though many people aren\u2019t happy with that kind of anthropomorphic comparison.) <\/span><\/p>\n

CoT transcripts played a central role in all of these reports; without them, their authors would\u2019ve had a much more difficult time understanding how and why the agents did what they did. The transcripts weren\u2019t always easy to interpret, but they at least provided a breadcrumb trail to be followed. It seems like now would be an inopportune moment to experiment with a new technique that would make it harder to read that trail.<\/span><\/p>\n

Last summer, a <\/span>paper<\/span> published by over three dozen AI researchers, including some from OpenAI and Anthropic, argued that CoT was essential for AI alignment, \u201coffering a rare glimpse into how AI agents make decisions.\u201d Following the publication of The Information\u2019s report, OpenAI chief scientist Jakub Pachocki\u2014one of the paper\u2019s coauthors\u2014wrote in a X post that he \u201c[wants] to prevent a race into unmonitorability kicked off by confused reporting,\u201d and that the ability to record and understand models\u2019 CoT reasoning is \u201ca core goal of [OpenAI\u2019s] research program.\u201d He didn\u2019t specify which part of the reporting was \u201cconfused,\u201d and OpenAI didn\u2019t immediately reply to Gizmodo\u2019s request for comment.<\/span><\/p>\n<\/div>\n

[analyse_source url=”https:\/\/gizmodo.com\/this-is-the-worst-possible-time-for-openai-to-bf%d0%b67%d9%852%e7%8c%ab9%e0%a4%95-2000806350″]<\/p>\n","protected":false},"excerpt":{"rendered":"

[analyse_image type=”featured” src=”https:\/\/gizmodo.com\/app\/uploads\/2026\/09\/openAI-sam-neuralese-1200×675.jpg”] AI models are notoriously likened to black boxes, meaning the humans who build them can\u2019t look inside to see how they transform mountains of training data into lines of code, sonnets, or whatever else they\u2019re asked to generate. Not completely, anyway. A subfield called interpretability research has blossomed in recent years, aimed […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[226,53],"class_list":["post-2125602","post","type-post","status-publish","format-standard","hentry","category-politics","tag-crawlmanager","tag-gizmodo-com"],"_links":{"self":[{"href":"https:\/\/analyse.optim.biz\/index.php?rest_route=\/wp\/v2\/posts\/2125602","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/analyse.optim.biz\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/analyse.optim.biz\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/analyse.optim.biz\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/analyse.optim.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2125602"}],"version-history":[{"count":0,"href":"https:\/\/analyse.optim.biz\/index.php?rest_route=\/wp\/v2\/posts\/2125602\/revisions"}],"wp:attachment":[{"href":"https:\/\/analyse.optim.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2125602"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/analyse.optim.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2125602"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/analyse.optim.biz\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2125602"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}