(Almost certain at this point that this is going to be my personal hobby and obsession for this year.)
In today’s journal.
If I had to rebuild an iOS app, as an interface to my agents, purely for myself, what would it actually look like?
Much much much much thinner than I expected.
- The deeper big brain lives on my Mac mini. I’ll write about that separately.
- But on the app side, I realized I do not want menus, tabs, and a lot of UI.
- I want the thinnest possible interface between me and the thing that is actually doing useful work.
For me, that means voice.
Most of the time, I do not want to sit down and type into box into yet another app.
I want to talk while walking, at home, in between things, when I have an idea, or when I want something done quickly. In one APP.
So the final “product” constraint became very simple:
- I speak.
- My agent infers my intent.
- It does something useful in our (digital) world.
- It reports back. I approve, review or give more tasks.
- We keep moving.
Much like how I would behave if I had a personal human assistant for me.
So that constraint was good and clarifying.
- This is also my first real iOS app. Xcode always felt intimidating from the outside.
- But with Codex we are continuing to make great and fast progress and learning very fast!
And for engineering nerd in me, this has been especially fun.
- When you strip away 95% of the complexity and keep only the 5% you actually need. Then you really start obsessing over that 5% and go super deep in stack.
Some Selected Details.
(1) Apple exposes a voice isolation feature in SwiftAPI that runs directly on device in its DSP… what!!! This is surprisingly useful if the app is meant to work while I’m outside or in noisy environments (which I do on walk). Din’t know this!
(2) Also transcription - all locally on phone. I knew this. Both (1) + (2) run in few ms.
(3) Even with an app that only has three or four buttons, there is a lot to care about. How long should I hold the record button? What feedback makes it feel instantaneous instead of clunky? What do I like? Confronted with a lot of design decisions.
(4) I also used to think some of Apple’s newer liquid design thingy was a bit gimmicky. But once I started building with it - I started appreciating and understand the “why” of it more.
So overall the gist is.
Earlier, if you wanted something “tailored” exactly to you, you had to be rich. You needed a personal tailor.
We poor folks used to buy bought off-the-rack “clothes” and then cut themselves to the fit. Software has mostly worked like that too.
We all use the same interfaces for a common low denominator.
But now I can build more personal, more tailored software.
I kinda knew this was gonna happen from reading from how other people were already doing in their tech blogs. But it’s nice to live through that reality.
Good days 2-5. More to come!