This article was originally written in Japanese and translated into English with AI assistance. Please note that some expressions may carry nuances from the original Japanese.
🇯🇵 日本語版はこちら / Japanese version
Series: Special Edition (Part 8)
This article was written around April 2026 and kept in reserve until now.
This time the subject isn’t a technical implementation but an operational pitfall in working together with AI. Specifically, Claude’s project knowledge feature.
How It Started: Claude Began Talking About a Completely Different Project
In any session with a generative AI, there’s an upper limit on how much context can be handled, so once the work has progressed to a certain point you end up switching to a new session to continue. Because information is fundamentally cut off at each session boundary, the AI can no longer respond in a way that accounts for everything that came before — and there were plenty of times when things drifted in directions I hadn’t intended.
Based on that experience, at the end of each session I do the following: “Summarize how we got here. Then write an initial prompt that will let the next session start work without getting lost.” What was decided and how, what was actually done, how things should proceed from there, and what to watch out for — I have Claude compile all of that, and I use it as the input that opens the next session.
I register the resulting “initial prompt that reflects the history so far” in the project knowledge, and at the start of the next session I ask Claude to “refer to the initial prompt and begin work.” That has let me keep work continuous without veering off in unintended directions.
That day, I started the session as usual — but something about its behavior was off.
What came back had nothing to do with the topic I had in mind. I use Claude’s support not only for development work but also for writing blog articles, and for some reason the conversation started off about the website. It made no sense.
The initial prompt I had registered just beforehand was definitely giving instructions about a blog article, yet the discussion that started was about the homepage. This session-summary-and-initial-prompt scheme is something I run the same way not just for the blog and the website but for each product development project as well. A mix-up about a blog article is something I can notice and fix, but if the same mix-up happened in development code, I have no programming experience and couldn’t undo it by my own hand. Thinking this could turn into a disaster if left alone, I started digging into it.
Digging Into the Cause: Three Independent Factors Had Stacked Up
Factor 1: A Filename Collision
For a while, I was running the website implementation project and the blog strategy project in parallel. As sessions accumulated in both, each naturally reached the milestone of “session 15.” As a result, initial prompts with exactly the same filename had been created in both projects.
- Website side: SRW_次セッション用初期プロンプト_セッション15.md
- Blog side: SRW_次セッション用初期プロンプト_セッション15.md
The filenames match completely. The only clue that distinguishes them is inside the files themselves.
Factor 2: Project Knowledge Bloat
Before I knew it, 105 files had been registered in this blog project’s knowledge. When I classified them, 36 files (about a third) turned out to be related to the website implementation.
Why had it come to that?
It was the result of thinking “this might make good article material too” and registering one potentially relevant file after another into the knowledge. The struggles of the website implementation might get used as material for a blog article. So I’d register them. It was a simple motive.
But as a result, the purity of the context — “what is this blog project actually about?” — kept getting diluted inside the knowledge.
Factor 3: How the AI’s Search Behaves
Claude’s project_knowledge_search returns results in order of relevance to the query. When filenames are identical and the wording of the contents is similar as well (a lot of shared vocabulary like “session 15,” “initial prompt,” “tasks”), which one gets hit comes down to luck.
In this case, Claude read the website-side document that came up first and presented it to me as “this is the instruction sheet for session 15.” Claude had no ill intent. It simply processed the search results faithfully.
What Happened When the Three Factors Overlapped
Claude was presenting work instructions from a completely different project as the correct answer for this one. If I hadn’t noticed something was off and had said “okay, let’s go with option A,” I would have started work touching HTML files on the website side.
The website’s HTML — in the blog project.
Recovery would have been possible, but it could have meant anywhere from tens of minutes to several hours of wasted effort. And worse, new contamination of the knowledge would have occurred. The artifacts of the mistaken work (a website revision plan created inside the blog project, for instance) would have accumulated in the knowledge as well, producing a different kind of confusion in the next session.
Countermeasures: Preventing a Recurrence at Three Layers
You could also say I was simply lucky to notice something was off. To prevent a recurrence, I put countermeasures in place at three layers.
Countermeasure 1: Making Filenames Unique (A Naming-Convention Update)
The most reliable countermeasure was to make it physically impossible for filenames to collide.
The new naming convention:
Old: SRW_次セッション用初期プロンプト_セッション15.md
New: SRW_次セッション用初期プロンプト_20260419_1145_セッション15.mdAll it does is insert a timestamp in _yyyymmdd_hhmm_ format into the filename. Unless I create two files within the same minute, filenames cannot collide in principle.
I registered this rule in Claude’s Memory and made it the practice that whenever Claude creates a new file, it must check the current time with me before naming it. The time isn’t something Claude fetches automatically; the user’s clock is the single source of truth. That way there’s no discrepancy or drift in what gets retrieved.
Countermeasure 2: Taking Stock of the Project Knowledge (Periodic Deletion)
If you keep registering files by the loose standard of “this might make good article material,” the knowledge will inevitably bloat. Bloated knowledge lowers the AI’s search accuracy and raises the risk of misidentification.
As a countermeasure, I decided to periodically delete files that are clearly outside the project’s subject. If I want to keep something as article material, it goes somewhere other than the project knowledge (a local archive directory, for example). I now run the knowledge as a place that holds only “what is directly needed for the project currently in progress.”
Countermeasure 3: A Verification Process That Doesn’t Take the AI’s Answers at Face Value
When something Claude presents feels off, immediately ask “where is that from?” This time, my asking a single question — “was that really what it said?” — was what let Claude re-run the search and discover its own mistake.
AI is fluent when it’s wrong. Not getting carried along by that fluency, and voicing a sense of wrongness as a sense of wrongness, is the safety valve of the collaboration.
Operating Without Misunderstandings — As a Methodology
Let me organize the above countermeasures into a reproducible methodology.
Method 1: Make Timestamped Unique Naming the Rule
For newly created artifacts, session summaries, initial prompts, journals, and so on — every file generated by way of Claude — put a _yyyymmdd_hhmm_ format timestamp in the name. Register this in Memory so that Claude observes it on its own initiative.
Method 2: Be Conscious of the “Knowledge Boundary” of Each Project
One Claude.ai project corresponds to one purpose. “Blog strategy,” “website implementation,” and “other project A” are each run as independent projects. Even when I want to reference materials from another project as article material, I take the approach of bringing in that version manually at the point I actually need it. No mixing things in on a “might use it later” basis.
Method 3: Periodic Stocktaking of the Knowledge
Once a month, or at session milestones (every 10 sessions, say), list every file in the knowledge and sort through it from these angles:
- Is it directly related to the project’s current subject?
- Has it been superseded by a newer version, and is the old one safe to delete?
- If I want to keep it only as article material, should it be moved to the archive directory?
Method 4: Make “Re-checking the Context” at the Start of a Session a Habit
At the top of a session, before Claude starts on anything important, I ask once: “Summarize in one sentence what you’re about to do, including the target project and the target file.” If what Claude summarizes differs from what I had in mind, I stop right there. A check that takes tens of seconds prevents hours of wasted effort.
Method 5: The Small Habit of Putting a Sense of Wrongness Into Words
When a response from Claude makes me think “wait, what?”, I don’t swallow it — I ask a short question back. “Is this content correct?” “Which file are you looking at?” “Is that really about this project?” Short questions you can ask in three seconds are what stop a runaway.
Summary
This stumble occurred across the following three layers:
- The user’s own practice: I used the same naming convention in a different project + registered a large number of related files for article-material purposes
- The design of the filename system: no timestamp was included
- The AI’s search behavior: which of two identically named files gets picked can’t be controlled
I put countermeasures in place across three layers as well:
- Changing the file naming convention (introducing _yyyymmdd_hhmm_)
- Taking stock of the project knowledge (boundary awareness and periodic deletion)
- Turning session practice into habit (re-checking the context, putting a sense of wrongness into words)
Collaboration with AI tends to be discussed in terms of whether individual prompts are good or bad. But what actually stalls the work, I’ve come to feel, is more often outside the prompt — file management, knowledge management, the design of project boundaries. The unglamorous countermeasures that actually work are mostly found in that territory.
Today, having added just an eleven-character timestamp to my filenames, I feel like the project got one step easier to see clearly.
Addendum (July 2026)
The events in this article are from April 2026, back when semantic search over the project knowledge was the center of how I collaborated.
Afterward, in late June, while I was working on CubePlot (the first product from SRW), I noticed that Claude’s desktop app was asking for access to my local disk. That was the moment I first encountered Cowork — a new mode of operation that can reference local files directly. (I was away from June 20 to 27 during this period, so I only actually started using it after returning home.)
Once July arrived and a major revision of the website became necessary, my use of Cowork advanced another step. I’ve since moved to running three lines in parallel: Cowork gathers information across multiple directories — the website, CubePlot, and the blog — and acts as the “coordinator,” while the actual file changes are handled by separate Claude Code instances, one for the website revision and one for CubePlot development.
In other words, the accident this article covers — “mistaking one file for another via semantic search over the project knowledge” — no longer happens in my main mode of operation. When you reference local files directly, which of two identically named files gets hit isn’t “down to luck”: specify the path and it’s uniquely determined.
That said, I don’t think the lesson itself has been invalidated just because the search method changed. As long as I have the AI working across multiple directories, the risk that both AI and human lose track of “which project, and which file, am I looking at right now?” remains, and the habits of unique naming and file housekeeping still pay off. Above all, Countermeasure 3 — “when a response from the AI feels off, ask where it came from” — is universally effective no matter what the reference method is.
Reference methods change. But the underlying structure — that unless you make implicit assumptions explicit, neither AI nor human will get the message — is something I suspect won’t change from here on either.
About Soul Resonant Works
Soul Resonant Works is a solo venture developing seven local AI systems.
Starting from zero programming experience, the development is progressing through collaboration with AI.
🌐 Soul Resonant Works:
→ https://sr-works.net/en/index.html
📝 This blog publishes the entire development process as a serialized journal.
CubePlot (free version available)
CubePlot is the first product from Soul Resonant Works — it turns a CSV into a 3D scatter plot you can rotate and zoom. No install, no sign-up: it’s a single HTML file you open in your browser, and it works offline. CubePlot itself does not send the data you load outside your machine — external network traffic is blocked at the browser level (CSP).
Start with the free version.
▶ Product page: https://sr-works.net/en/cubeplot/
▶ Get it (commercial license, USD $39 + tax where applicable): https://soulworks8.gumroad.com/l/pzcij
If you found this article useful, please share it.
Leave a Reply