{"id":76,"date":"2026-07-21T22:00:00","date_gmt":"2026-07-21T13:00:00","guid":{"rendered":"https:\/\/sr-works.net\/blog-en\/?p=76"},"modified":"2026-07-21T22:00:00","modified_gmt":"2026-07-21T13:00:00","slug":"datamigrator-interim-recap-the-road-after-m2-and-a-detour-called-utf8conv","status":"publish","type":"post","link":"https:\/\/sr-works.net\/blog-en\/datamigrator-interim-recap-the-road-after-m2-and-a-detour-called-utf8conv\/","title":{"rendered":"DataMigrator Interim Recap \u2014 The Road After M2, and a Detour Called utf8conv"},"content":{"rendered":"\n
\n

This article was originally written in Japanese and translated into English with AI assistance. Please note that some expressions may carry nuances from the original Japanese.<\/em><\/p>\n<\/blockquote>\n\n\n\n

\ud83c\uddef\ud83c\uddf5 \u65e5\u672c\u8a9e\u7248\u306f\u3053\u3061\u3089 \/ Japanese version<\/a><\/em><\/p>\n\n\n\n

\n\n\n\n

Series: DataMigrator Development Journal (Part 7, Interim Recap)<\/p>\n\n\n\n

\n\n\n\n

Last time, M1 (the project-foundation milestone) wrapped up. By rights, this time I should be writing about M2 (implementing the scan engine). But to be honest, as of this writing, I haven’t started M2 yet. Instead, development of a different product (utf8conv) took off first, and DataMigrator has, for the moment, been set aside on the shelf.<\/p>\n\n\n\n

It feels a bit awkward to keep writing the series in this half-finished state, but letting the blog sit idle until the implementation is fully done doesn’t feel right either. So this time, I’m putting together “where DataMigrator currently stands” and “what I plan to do from M2 onward,” and using it as an article to close out the DM series for now. Next time, I’ll slot in a standalone review article, and after that I’ll move into the utf8conv series. Returning to DataMigrator will come later still.<\/p>\n\n\n\n

Where DataMigrator Currently Stands<\/h2>\n\n\n\n

Here’s a rough summary of where DataMigrator stands right now.<\/p>\n\n\n\n

First, the requirements definition is done. It went through revisions from Rev.0 all the way to Rev.7, ending up as a document of over thirty chapters (I wrote about that revision process in an earlier article). Next, all eight design documents are complete: architecture, source-code organization, GitHub repository structure, DB schema, class design, GUI design, test design, and the initial prompt for the implementation phase (I also wrote about that design phase in an earlier article). And Claude Code has been introduced, with M1 (project foundation + DBManager + CI\/CD) now complete (I wrote about that introduction and the road to finishing M1 in an earlier article, too).<\/p>\n\n\n\n

On the other hand, it’s not yet a “working tool” from a user’s point of view. Right now, what sits in the GitHub repository is a Python package that can’t even display an empty window. Only the sixteen tests are passing, so the DB layer is reliably working, but everything else is still ahead.<\/p>\n\n\n\n

In design-document terms, seven milestones remain in full: M2 (scan engine), M3 (hash engine), M4 (grouping + explore mode), M5 (simple mode), M6 (full-scrutiny mode), M7 (unwanted-file handling + output), and M8 (polish + release). M1 took two hours, so by simple arithmetic the remaining seven would take fourteen \u2014 I’d like to say that, but it probably won’t be that simple. M1 was mostly a matter of “laying things out roughly as the design document said,” but from M2 onward, there’s a growing element of “reconciling with reality” \u2014 does the code written actually get along with the real filesystem, does the performance hold up, does it break down at the edge cases.<\/p>\n\n\n\n

What I Want to Confirm From M2 Onward<\/h2>\n\n\n\n

There are a few points I’m concerned about for the milestones from M2 onward. I’m writing them down partly as notes to myself.<\/p>\n\n\n\n

The first is performance. What DataMigrator ultimately has to deal with is data on the scale of hundreds of thousands of files and multiple terabytes. The design calls for a three-stage approach \u2014 narrow things down fast with a partial hash, then run a full hash \u2014 but it’s a common story for things to turn out slower than expected once you actually run them. I expect to run into this first around M3.<\/p>\n\n\n\n

The second is filesystem traps. macOS is full of them: iCloud placeholders (files whose contents haven’t been downloaded), extended attributes, resource forks, invisible files, symlink loops. The design document lays out countermeasures, but there’s a good chance that, once I’m actually implementing, I’ll run into things like “huh, this wasn’t written down.”<\/p>\n\n\n\n

The third is whether the “never delete” design philosophy holds up consistently. DataMigrator never performs a “delete.” Even when it finds duplicate files, instead of deleting them it moves them into a quarantine folder called “removed,” so users can change their minds and get them back. Can this policy be kept consistent across every feature from M4 onward? The design document says it will be, but I have a feeling that implementation convenience will keep tempting me to make “just this one exception” \u2014 and whether I can hold the line when that happens is also a test of myself.<\/p>\n\n\n\n

The fourth is GUI usability. No matter how solid the internals are, nobody will use it if the screens are hard to use. The GUI design document lays out a structure of nineteen screens, but once I actually build it in PySide6, there are bound to be places where “this isn’t what I expected.” This is a part I can only judge by using it myself as a user \u2014 it’s not something the design document alone can settle.<\/p>\n\n\n\n

The fifth is whether I can actually make it to release. Turning it into a .app with py2app, signing it with an Apple Developer ID, getting it through Notarization, and distributing it as a DMG file \u2014 I’m planning to walk through this process first with utf8conv. By the time DataMigrator is ready to release, I should be able to reuse the release know-how from utf8conv, so I’m treating that as a kind of advance investment.<\/p>\n\n\n\n

Why utf8conv Cut In Line<\/h2>\n\n\n\n

Here, let me explain why development of a different product (utf8conv) started up right after DataMigrator’s M1 finished. This is a bridging note for readers following this blog in chronological order.<\/p>\n\n\n\n

I touched on this in an earlier blog post too: when you’re developing while conversing with Claude, you sometimes run into character-encoding problems. The AI tells you, “please re-save this file as UTF-8.” You re-save it as told. But the garbled text on screen doesn’t get fixed. VS Code itself says, “this is UTF-8.” And still it’s not fixed.<\/p>\n\n\n\n

This experience kept nagging at me. Digging into the cause eventually leads to the story of a mismatch between a file’s actual contents (its byte sequence) and the “encoding declaration” written at the top of the file \u2014 but I’ll save that story for the next series. What I want to say here is that the moment I thought “I should build a tool that solves this problem,” my head jumped from DataMigrator’s M2 straight over to utf8conv.<\/p>\n\n\n\n

Looking back now, this jump has both a good side and a bad side. The bad side is obvious: DataMigrator has stalled. As for the good side, let me note just this much: utf8conv is, in scale, much smaller than DataMigrator, and it’s easy to run straight through from requirements definition to implementation and release prep. So partway through, the calculation crept into my head that, as a candidate for “the first product SRW releases,” utf8conv is more realistic than DataMigrator.<\/p>\n\n\n\n

In other words, while keeping the ambition of finishing DataMigrator as “the first product,” I ended up swapping the order so that utf8conv gets released first as “the first-first product” (odd phrasing, I know). The idea that running the full release pipeline through utf8conv first means I can reuse that same pipeline when DataMigrator releases might be a rationalization that’s close to an excuse.<\/p>\n\n\n\n

What I Can Say, As an Interim Recap<\/h2>\n\n\n\n

Having gotten DataMigrator’s development this far, I want to close this article by writing down just three things I can say at this point.<\/p>\n\n\n\n

First: it was worth spending the time on requirements and design. All told, requirements definition and design probably took around twenty hours. The temptation of “wouldn’t it be faster to just start coding” came up more than once, but the reason M1’s implementation could proceed almost nonstop in the end is that I could hand Claude Code exactly what was written in the design documents. If the design had been vague, time spent agonizing over each implementation step would have piled up, and it probably would have ended up slower.<\/p>\n\n\n\n

Second: don’t leave everything entirely to the AI. The AI will write the design documents, it’ll write the code, it’ll even write the tests on its own. But the fundamental judgment calls \u2014 what to build, why to make a given choice, what to include in or cut from the MVP \u2014 those you have to think through yourself. Put another way, my sense is that what AI covers is the “work,” not the “decisions.” If there’s anyone reading this blog who’s trying to build something with AI, this is the one thing I think is worth being prepared for.<\/p>\n\n\n\n

Third: even if it stops partway, that has value of its own. As of this writing, DataMigrator still isn’t finished. Only M1 is done. It’s possible that, even after the utf8conv series, some other “unplanned” thing will cut in and it’ll never actually get finished. Honestly, that possibility isn’t zero. Even so, what I’ve gained along the way \u2014 how to run a requirements process, a feel for cycling AI reviews, the right granularity for design, how to work with Claude Code, how to use uv, setting up CI\/CD, Claude Code’s autonomous test generation \u2014 all of it is an asset I can reuse in the next product, whether that’s utf8conv or any of the seven AI systems. Not finishing doesn’t mean it adds up to zero.<\/p>\n\n\n\n

And with that, I’m closing out the DM series here for now. Once DataMigrator moves forward into M2 and beyond, I’ll pick up the story again.<\/p>\n\n\n\n

Next Time<\/h2>\n\n\n\n

Next time, I’m taking a bit of a detour to slot in a standalone review article. The theme is “Local LLM Benchmark.” It’s a record of actually running benchmarks to test the question of whether confidential data that can’t be sent to the cloud can be inspected using nothing but a local LLM running on my own Mac (64GB of memory). The utf8conv series starts after that.<\/p>\n\n\n\n

\n\n\n\n

About Soul Resonant Works<\/h2>\n\n\n\n

Soul Resonant Works is a solo venture developing seven local AI systems. Starting from zero programming experience, the development is progressing through collaboration with AI.<\/p>\n\n\n\n

\ud83c\udf10 Soul Resonant Works:
\u2192 https:\/\/sr-works.net\/en\/index.html<\/a><\/p>\n\n\n\n

\ud83d\udcdd This blog publishes the entire development process as a serialized journal.<\/p>\n\n\n\n

\n\n\n\n

If you found this article useful, please share it.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

This article was originally written in Japanese and translated into English with AI assistance. Please note that some expressions may carry nuances from the original Japanese. \ud83c\uddef\ud83c\uddf5 \u65e5\u672c\u8a9e\u7248\u306f\u3053\u3061\u3089 \/ Japanese version Series: DataMigrator Development Journal (Part 7, Interim Recap) Last time, M1 (the project-foundation milestone) wrapped up. By rights, this time I should be writing […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-76","post","type-post","status-publish","format-standard","hentry","category-ai-development-journal"],"_links":{"self":[{"href":"https:\/\/sr-works.net\/blog-en\/wp-json\/wp\/v2\/posts\/76","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sr-works.net\/blog-en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sr-works.net\/blog-en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sr-works.net\/blog-en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/sr-works.net\/blog-en\/wp-json\/wp\/v2\/comments?post=76"}],"version-history":[{"count":2,"href":"https:\/\/sr-works.net\/blog-en\/wp-json\/wp\/v2\/posts\/76\/revisions"}],"predecessor-version":[{"id":78,"href":"https:\/\/sr-works.net\/blog-en\/wp-json\/wp\/v2\/posts\/76\/revisions\/78"}],"wp:attachment":[{"href":"https:\/\/sr-works.net\/blog-en\/wp-json\/wp\/v2\/media?parent=76"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sr-works.net\/blog-en\/wp-json\/wp\/v2\/categories?post=76"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sr-works.net\/blog-en\/wp-json\/wp\/v2\/tags?post=76"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}