Diary Day 6 to 8: Behind the scenes of building my personal agent from scratch

1 min read


Wired all of my health context into Dobby.

How my health reaches Dobby

In the last two days, I integrated:

  • Apple Health and Apple Watch data
  • body composition and weight data from my weighing scale
  • health records like vaccinations, blood tests, and medical reports

So now all of that lives in one place, inside Dobby’s brain, privately on my Mac mini, and it can use that context whenever needed.

Why this matters is simple.

I do not want six apps for different questions.

I want one assistant that knows my context and can reason across it.

So now I can ask, in plain natural language, usually by sending a voice note:

  • What did my workouts look like this week?
  • How does my body composition look right now?
  • What is my vaccination status?
  • Is there anything in my health records I should pay attention to?

And I get the answer in the same iPhone app I already use for everything else.

Workout summary from Dobby

Body composition snapshot from Dobby

Because it is built on top of Codex App Server, it also inherits the usual agent primitives: tool use, MCPs, skills, file search, and web lookup when needed.

More importantly, because it is part of a larger brain with more context about me, it can do more than just retrieve data. It can reason across it more holistically, in the context of my life.

Next steps

  1. Add a layer of gamification. If I hit a health goal, like reaching my target weight, Dobby should reward me somehow.
  2. Surface proactive insights from this data.