Question
Do AI systems “have memory” today? Is memory the main bottleneck limiting AI capability? I’m publishing a piece on what “AI memory” really means in practice (context window vs saved personalization vs retrieval/RAG). For experts: 1、How do you define “memory” in modern AI products? 2、Is memory the #1 current bottleneck? If not, what is (reasoning, reliability/hallucinations, planning, tool-use, cost/latency, data quality)? 3、One concrete example where better memory clearly improves outcomes. Please include your role/company. 120–180 words. No AI-generated responses.
Reviewed by
3 expert answers
- How do you define "memory" in modern AI products?
Memory in AI is how much context it holds during a conversation versus what it actually remembers between sessions. Tools like ChatGPT keep the thread going while you're chatting but forget everything the second you close it, unless you've set up custom instructions. Actual memory would mean the AI learns from what we've done before without me repeating myself constantly.
- Is memory the #1 current bottleneck? If not, what is?
For me, memory's a major pain but reliability's what really gets my goat. I can give AI all the context it needs, but that doesn't help when it just makes stuff up. I've had it spit out client proposals that referenced non-existent WordPress plugins or talked about design trends it just made up out of thin air. That kind of thing kills trust way faster than a tool forgetting what we were talking about 5 minutes ago.
3.One concrete example where better memory clearly improves outcomes:
If Claude could somehow keep track of our brand guidelines, client jargon & design systems from one session to the next I wouldn't have to spend 20 minutes a day re-explaining the same old stuff like why we prefer 'hero section' over 'banner'. Better memory would save me a solid 30% off setup time, keep our team from getting different answers & working on the same project only to find out we're all doing things different ways.
Name: Nirmal Gyanwali
Role/company: Founder/CMO of WP Creative
I believe there are three layers of memory in today's artificial intelligence. The first layer is the context window or what the AI has access to for reasoning in real-time. The second is persistent personalization. Persistent personalization includes everything from a user's saved preferences through to a history of past interactions with the AI. The third layer is retrieval memory. Retrieval memory utilizes retrieval-augmented generation techniques to allow the AI to retrieve the correct information at the appropriate time. While the majority of individuals consider these three layers to be the same, they serve very different purposes. Reliability, rather than memory, is currently the most significant bottleneck for AI systems and the greatest barrier to their widespread adoption. While AI systems are capable of recalling increasingly larger amounts of data through memory, they continue to be unable to reason consistently, execute complex multi-step plans seamlessly, and avoid confident yet incorrect responses. In the case where AI has been deployed into commercial applications, the impact of hallucination and the inability to leverage tools flexibly has caused greater risk than either of these three categories of potential risks.
A clear example of the value of using memory when creating a report is Search Engine Optimization (SEO) reporting. By using retrieval memory (the retrieval of data based on context), an AI analyst is able to compare multiple months worth of Google Analytics 4 (GA4) and Search Console data instead of relying on a single export. As such, the use of retrieval memory allows the AI to provide more context, and thus make better decisions, while reducing the likelihood of providing an erroneous recommendation. However, unless the AI produces an accurate recommendation based on retrieval memory through additional data validation, the AI will be able to supply a recommendation based on an increase in confident mistakes.
When people talk about "AI memory," they often lump very different things together: what the model can hold in its context window right now, what gets stored long-term as state or preferences, and what's retrieved on demand from external systems like RAG, CRMs, or databases. In real-world products, memory isn't a single capability it's a design and cost decision.
Memory isn't the biggest bottleneck today. The real issue is reliability in execution. Models still drift in their reasoning, struggle with consistent tool use, and fail quietly when conditions change. On top of that, memory has a real price. Pushing more history into prompts drives up token usage, latency, and cost, often with little practical benefit.
Where better memory clearly improves outcomes is in long-running workflows. An AI voice agent following up in finance or recruitment needs to remember what's been requested, what's outstanding, past objections, and when a human stepped in. That memory has to live outside the prompt, be structured, cheap to access, and easy to audit.
The future isn't bigger context windows. It's remembering less but remembering the right things.
Dave King
