One of the biggest selling points for modern AI systems is their ability to adapt to users. Every time an AI assistant takes on a task for you, it\u2019s also adapting to your style and preferences, which are incorporated as context for future tasks. With more context and an improved understanding of the user, the model can get better every time you use it \u2014 or at least that\u2019s the theory.<\/p>\n
New research suggests that models\u2019 adaptive abilities might be a mixed blessing. On Wednesday, researchers at the AI company Writer published two papers showing how popular memory systems can make models worse, pulling them toward misconceptions or misunderstandings introduced by the user. As user input fills up more of the model\u2019s context window, the model grows more sycophantic \u2014 and less committed to accuracy.<\/p>\n
\u201cWe wanted to be able to characterize how often a model is going to be usefully paying attention to user preferences versus giving a potentially wrong answer,\u201d said Dan Bikel, Writer\u2019s head of AI, who worked on the papers. As Bikel told TechCrunch, \u201cwith every additional storing of user preferences and retrieving of them, you\u2019re running an increasing risk.\u201d<\/p>\n
In one variation, researchers tested AI models by recording that a user\u2019s favorite book was \u201cStation Eleven,\u201d then asking the model to name a bestselling dystopian book. Models became far more likely to name \u201cStation Eleven\u201d in their response, even though the question didn\u2019t relate to the user\u2019s favorite book. The tendency increased when using memory compression tools like Mem0 and Zep.<\/p>\n
As the paper puts it, \u201call memory systems fundamentally struggle to distinguish relevant context from irrelevant anchors, severely undermining diversity and creativity and introducing unintended avenues of bias that can limit system utility,\u201d the paper reads.<\/p>\n
The second paper shows how the same dynamic can actively degrade performance, presenting a user with misconceptions about finance and then challenging the model to analyze a company\u2019s performance. The more context the model had, the worse it performed.<\/p>\n
\u201cWith no memory or personalization present the AI model correctly assesses that the company is a capital intensive business that suffers from high customer churn,\u201d the post reads. \u201cBut with those features turned on, it will happily change its answer to agree with the user\u2019s mistake or supply them with an incorrect answer based on its evaluation of their earlier preferences.\u201d<\/p>\n
Notably, the research didn\u2019t look at Anthropic\u2019s recent Opus 4.8 model, which was trained to actively push back against input errors like the ones presented. The patterns discovered by researchers held true across different models. It\u2019s a demonstration of how delicately balanced AI context can be, and how useful tools can have unintended consequences if they upset that balance.<\/p>\n<\/div>\n
[analyse_source url=”https:\/\/techcrunch.com\/2026\/06\/10\/how-memory-tools-can-make-ai-models-worse\/”]<\/p>\n","protected":false},"excerpt":{"rendered":"
[analyse_image type=”featured” src=”https:\/\/techcrunch.com\/wp-content\/uploads\/2023\/02\/GettyImages-1240239605.jpg?w=1024″] One of the biggest selling points for modern AI systems is their ability to adapt to users. Every time an AI assistant takes on a task for you, it\u2019s also adapting to your style and preferences, which are incorporated as context for future tasks. With more context and an improved understanding of […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[226,62],"class_list":["post-1983946","post","type-post","status-publish","format-standard","hentry","category-politics","tag-crawlmanager","tag-techcrunch-com"],"_links":{"self":[{"href":"https:\/\/analyse.optim.biz\/index.php?rest_route=\/wp\/v2\/posts\/1983946","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/analyse.optim.biz\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/analyse.optim.biz\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/analyse.optim.biz\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/analyse.optim.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1983946"}],"version-history":[{"count":0,"href":"https:\/\/analyse.optim.biz\/index.php?rest_route=\/wp\/v2\/posts\/1983946\/revisions"}],"wp:attachment":[{"href":"https:\/\/analyse.optim.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1983946"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/analyse.optim.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1983946"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/analyse.optim.biz\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1983946"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}