This article was originally written in Japanese and translated into English with AI assistance. Please note that some expressions may carry nuances from the original Japanese.
🇯🇵 日本語版はこちら / Japanese version
Series: utf8conv Development Journal (Part 7)
At the end of last time, I wrote this.
With this, the implementation of utf8conv is complete. The Mac app form (
.app), as I wrote two installments ago, has already been exported. What remains is not writing code but the procedures for “putting it out into the world.” The signing and notarization with an Apple Developer ID written under “Future Outlook” in the requirements document. And then deciding where to distribute it.
Last time’s story was a record from March. And now, as I write this, it is October. utf8conv has still not been distributed.
This time, I will write about what happened in that half year. Not a story about development, but a story about “why something I thought I had finished building did not go out.”
What Was Left When I Wrote “Just a Little More”
First, let me line up what was visible as of last time.
- The code is written. The tests pass too
- The
.appfor macOS has been built too - What remains is the signing and notarization with an Apple Developer ID
- And then, deciding where to distribute it
Written like this, it looks like a matter of a few more days. In fact, I thought so myself.
There is something that was not in this list. That I found this out in September is what this installment is about.
Why I Was Stopped for Half a Year
The reason I was stopped is a single one. It was because I could not decide whether or not to join the Apple Developer Program. To distribute a Mac desktop app to other people, the app has to be checked as to whether it was developed in a proper environment, and that requires joining the Apple Developer Program (99 dollars a year). What I was agonizing over was not the amount itself. “Am I even the kind of person who is at a level where I can join in the first place?” “If I pay 99 dollars a year, will I be able to get a return on it?” “Can software of mine, truly developed together with AI, really get through that process?” — I was extremely skeptical about these three. The payment itself is not an amount I could not afford, but if the certification and so on did not go through, it would be wasted, and I did not know whether I could recover from there.
Unable to reach conclusions on these, I decided to stop the work. I would resume once I had made up my mind — that is what it came to.
Also, a different idea had come up, and the urge to hand that one over to AI quickly and confirm whether it could be realized had come over me — that too was one reason I was stopped.
I Had Crossed the Wall While Building Something Else
For that half year, the work of distributing utf8conv was stopped. But that does not mean my hands were idle. I built and released CubePlot, a tool for drawing 3D graphs, and separately from that I was building one more desktop app, for Mac and Windows.
It was on August 11 that I joined the Apple Developer Program in order to distribute that desktop app.
What joining lets you do is “signing” and “notarization.”
- Signing: writing your name with a certificate, saying “this app is something I made”
- Notarization: sending it to Apple’s side and having them check that nothing bad is inside
An app that has not been signed and notarized shows the warning 「開発元が確認できません」 (the developer cannot be verified) when you try to open it on a Mac.
And this is the thing I most wanted to write this time.
The wall that had stopped utf8conv, I had crossed, before I knew it, while building a different app.
Joining the Apple Developer Program is per developer, not per product. Once you have joined, it can be used as is for signing and notarizing other products too. During the half year I was stopped, intending to cross it for utf8conv’s sake, I had crossed it for a different reason. That is what it was.
Rereading After Half a Year: The Design Was Right, and My Notes Were Wrong
Now, from here is the main subject.
In September, I checked the drafts of this series against the current implementation. Their content is the record from March, and half a year has passed. I needed to confirm that they did not conflict with the implementation as it is now.
There were conflicts. Only, in the opposite shape from what I had expected.
In the draft of the technical explanation installment, I had written this.
utf8conv opens a file, reads only the first few hundred to few thousand bytes, and determines which encoding it is written in. There is no need to read the whole file.
But when I open the code, this is what it says.
raw = path.read_bytes() # ← reads the whole file
It reads all of it. There is no processing anywhere that reads only the beginning.
(The technical explanation installment of this series has been corrected since this discovery. If you reread it now and it says it reads the whole file, that is why.)
At first I thought, “I must have changed the implementation partway through.” In half a year, that kind of thing happens. But as I followed the records, that was not it.
There was no record of changing the implementation partway, neither in the development journal nor in the instructions. When I went back and checked the code saved in March, as of the code saved on day two of development, that is, March 8, it was already reading all of it. Not once had it read only the beginning.
Then where did that description come from?
From One Line I Wrote to an Article
I searched for the origin. I searched the utf8conv repository — the place where this project’s files and the history of their changes are kept together — and the place where the blog manuscripts are kept, for the word 「先頭」 (the beginning), and lined up the results from oldest to newest.
- March 8, the requirements document Rev.2 says “determine the whole from the leading bytes”
- The instruction text handed to AI the next day, based on that requirements document, carries the same description
- It is carried over into the memo organizing the discussion points, written in May
- The blog-material memo recorded on March 8 has the concrete expression “reads only the first few hundred to few thousand bytes”
- That is taken into the blog’s manuscript collection
- It becomes the draft, before correction, of the technical explanation installment
Six documents. Counted as the number of times it was copied, five. And all of them were still on hand. Four in the utf8conv repository, two on the blog side. (The recorded date of the fourth memo is March 8, the same as the requirements document Rev.2. Which of these two came first cannot be told from the records.)
In other words, one mistaken line I had written in the specification had become an article I wrote, and was about to go out into the world. The implementation had been right from the start, and only the explanation of the implementation had taken on a life of its own over half a year.
This error was found and corrected in August. The requirements document Rev.3 of August 4 says this.
The description “determine by looking only at the leading bytes” was an error and was deleted in Rev3. In reality,
read_bytes()reads the whole file, and the whole is decoded
But what was corrected in August was only the requirements document and the document that instructed that correction. The material memo, the manuscript collection, and the article draft were all left as they were. What is more, the requirements document itself still had a sentence derived from this misstatement, “processing speed is proportional to the number of files,” and that disappeared only on September 9. Correcting upstream does not automatically correct the downstream copies. And where the copies are, you do not know until you look.
One more thing became clear. Among the tests, there is a test placed with a comment like this.
Detect here a regression to an implementation that “looks only at the first N bytes.”
It had been set up so that if I ever carelessly went back to an implementation that “reads only the beginning,” the test would fail. The implementation had been consistent from first to last. What wavered was the writing.
In That Half Year, the Design Had Changed in Three Ways
One more thing I learned from the comparison. During the half year that the distribution work was stopped, utf8conv’s design had changed in three ways. In terms of the revision history of the requirements document, Rev.3 of August 4 and Rev.4 of August 31.
First. The way character encodings are assigned changed from the ground up.
In the fourth installment of this series, I wrote this: try 16 encodings from the top in a fixed order, and adopt the first one that reads. How to decide that order was the theme of that installment.
Now it is different. Instead of guessing by order, it asks the person using it first: “What language is this file?” Choose Japanese and there is one candidate; choose Korean and there is one. The very idea of trying everything in turn is gone.
Moreover, Japanese alone is split into two choices: 「日本語（まずはこちら）」 (Japanese (try this first)) and 「日本語（EUC-JP：上でエラーや文字化けになる場合）」 (Japanese (EUC-JP: if the one above gives errors or mojibake)). The reason is written in a comment in the code. With a byte sequence that could be read either way, EUC-JP would succeed first, and things like Shift-JIS half-width katakana had been silently garbled. If the candidates are narrowed to one, that mix-up structurally cannot happen.
While doing this rewrite, I had also plugged a hole I had left noted in the technical explanation installment. The story of an ISO-2022-JP file looking like “a file with nothing but ASCII” and passing straight through. Now, the processing that looks for those signal bytes comes before the UTF-8 check. Something I had not noticed when I wrote the article had been fixed half a year later, with no connection to the article. The order is the reverse: after the fix, the article caught up.
Second. latin-1 was removed from the candidates.
That latin-1, which I had placed in the second installment of this series as “a catch-all.” Back then I wrote, “it can read any byte sequence, so the processing does not have to stop.” And I also wrote, “this decision gets overturned later.” Here I collect on that.
The problem was that the very state of “none of them could read it” disappears. No file being unreadable means that images, executables, everything becomes a target for conversion. The fortress meant to keep things from stopping was in fact breaking the safety valve. So I removed it.
Third. The way of saving after conversion changed.
Previously, the original file was moved aside into a backup folder. Now, the default is to put a mark at the front of the original file’s name and leave it where it is. The reason is simple: the file’s information, such as its creation date, is preserved. Moving it to a folder can lose that.
None of the three appear in the March articles. Because the design did not exist in March. The half year I had thought of as “stopped” had, on the record, been properly moving.
And There Was Not a Single Procedure for Building the Thing to Distribute
Last, the discovery that hit hardest.
In September, I ran an inspection of the product candidates. Not “do the files exist,” but an inspection of “actually build it, run the tests, and get one main feature working” (I did not go as far as opening the screen to check).
For utf8conv, the result was this.
- The tests pass (426 collected, 354 passed. The remaining 72 are skips, and zero failures)
- The character encoding conversion actually works too (★ but the screen is unconfirmed)
- There is not a single procedure in the repository for building a distributable (the thing bundled into a form you can hand to people)
And this despite my having written in the last article, “The Mac app form (.app), as I wrote two installments ago, has already been exported.”
To be precise, it is like this. PyInstaller, the tool that bundles a Python program into the form of an app, is in the list as a development tool. But there is not a single configuration file that says how to run it. There is no procedure written down, in other words.
What Is a Spec File?
A short detour, to write down what this “configuration file” is.
PyInstaller is a tool you run by typing a command in the terminal. When I built the .app in March too, the procedure left in the README was a single command.
Here is something I did not know at the time. When PyInstaller receives this command, it first writes, by itself, a blueprint called a “spec file.” And when it assembles the app, it executes not the command but that blueprint. The official documentation says this.
The spec file is actually executable Python code. PyInstaller builds the app by executing the contents of the spec file.
In other words, the act of typing the command was having the blueprint written automatically, and then having it executed.
And from the second time on, you can hand it that blueprint directly.
pyinstaller utf8conv.spec
That alone produces the same thing as before. No need to remember long options. What is more, things that cannot be fully written on the command line — bundling data files in, including the parts needed at run time — can also be written in this blueprint.
The Blueprint Had Been Set Not to Be Recorded, from Day One
So where is that blueprint kept?
When I looked, it had never once gone into the repository’s record. I went back through all 24 commits — a commit is the save operation of “keep the changes up to here, under this name” — and there were 0 spec files.
The reason was simple. The .gitignore — a file for writing down “do not record this kind of file” — had this in it.
dist/
build/
*.whl
*.spec
Both dist/ and build/ are things you can make by rebuilding. A spec file, too, is something you can remake by typing the command, so it was in the same row.
And this .gitignore was written at 23:40 on March 7, 2026, in the very first commit. It is 「U1: プロジェクト基盤セットアップ」 (U1: project foundation setup). (U1 is the number I had given to the first unit of work.) The .app was built about 12 hours later.
Here, there is something I have to write down. This .gitignore is not something I wrote with my own hands. It was created in the “project foundation setup” on day one, together with configuration files like pyproject.toml. As I wrote in the first installment of this series, that work was done by having AI move its hands.
The setting of not recording *.spec worked exactly as written. The blueprint never once remained in the record. The command to remake it did remain in the README, but the premise “it can be remade” stands on the premise “there is someone who knows the command to remake it.” I did not remember that command, and I did not even know that such a thing as a spec file existed, until half a year later.
A correct boilerplate that AI had put in place had hidden a single blueprint from the me of half a year later. Nobody made a mistake, and still this kind of thing happens.
This is different from “the build is failing.” There is no place where the build is written. What remained was not the work of fixing, but the work of writing.
In the last article, I had written that what remains is “not writing code but the procedures for ‘putting it out into the world.’” In reality, before the procedures, there was still something to build.
What Did “Complete” Mean?
In last time’s closing, I wrote, “With this, the implementation of utf8conv is complete.” That was not a lie. The code is written, and the tests pass.
Only, between “it works” and “it can be distributed,” there was a distance.
- It works (on my own computer)
- It can be bundled into a distributable form ← ★ this was missing
- Signing and notarization are done
- The place to put it is decided
When I wrote “complete” in the last article, what was done was the top line. The second line, I learned for the first time in September’s inspection, was not there. The membership the third line needs had been completed in August, for a different product.
Lining them up like this, I can see that what I had been calling “complete” was only the top line. For someone who started building with zero programming experience, the difference between these four lines was not visible until I had finished building.
And this distance did not close in a few days. Building took 4 days, but distributing is still not finished, even in October.
Where Things Stand as of October
The procedure for building the distributable was written up as a document in mid-October and handed to AI, and the first stage went through. I decided to make a Windows version as well; the materials for trying it out are consolidated into one, and it runs on Windows first. The signed and notarized version is scheduled to be tried from a download.
Next time, I will write about what I learned before writing that procedure.
About Soul Resonant Works
Soul Resonant Works is a solo venture developing seven local AI systems.
Starting from zero programming experience, the development is progressing through collaboration with AI.
🌐 Soul Resonant Works:
→ https://sr-works.net/en/index.html
📝 This blog publishes the entire development process as a serialized journal.
CubePlot (free version available)
CubePlot is the first product from Soul Resonant Works — it turns a CSV into a 3D scatter plot you can rotate and zoom. No install, no sign-up: it’s a single HTML file you open in your browser, and it works offline. CubePlot itself does not send the data you load outside your machine — external network traffic is blocked at the browser level (CSP). Start with the free version.
▶ Product page: https://sr-works.net/en/cubeplot/
▶ Get it (commercial license, USD $39 + tax where applicable): https://soulworks8.gumroad.com/l/pzcij
If you found this article useful, please share it.
Leave a Reply