This is a screenshot that means literally nothing to 99% of humans on the planet, but to me it’s instructive and tells me precisely what I need to know: that the ongoing conversion of the 1988 floppy game Pool of Radiance is well underway. I still have the disks, covered as the box is under several layers of dust and grime.
Why am I talking about PC titles from the 80s when this post was about Codex /goal observability? Because games used to ship in large gold boxes, full on instruction manuals pages and pages long, and an adventure that took place more in the mind than the eye.
Keeping an Eye on Codex
I’m experimenting with different ways of keeping tabs of long running /goal. If you’re unfamiliar with the command in Codex, the slash-goal is a way to plant a flag and send the AI off to work. The hustle and bustle in the digital wires lend itself to something, some awe and mystery, and when the clock hand strikes some long hours (or days) later you have either genius, madness, or a pile of broken dreams before you. It’s this latter case that I’ve found myself in need of course correcting.
There are many different ways to keep tabs on work. You could, for example, have the agent send periodic status updates. If the work is well-structured, then even subscribing to regular status updates might well do the trick. Me, though? I am a big fan of a website that I can periodically review and visually see what’s going on-either at a glance, or in detail.
It’s funny, in a sad, sad way that these little websites that track the work done by codex are often larger than many websites I’ve built myself or witnessed teams produce. This site has enough content that highlights the primary problem of /goal. When you think you’re reading and keeping up with what the progress is of /goal, if you aren not tracking it in a way like what I’m showing… you’re clearly not tracking the progress. I say that because it’s impossible. I regularly burn up millions of tokens on these projects and often the work is done while I’m asleep or simply not even at home.
Even better: you can see a project come into focus by looking over it like this. It gives you a visual for how the work is shaping up, if you need to step in and course correct. If you’re wondering about the code, then you’re like me: wondering if I’ll ever actually look at the code. Unlikely-this is just a fun project for me. An itch from my past to see what’s possible.
How to get your own site
Having codex build these sites is trivially easy. In fact, it’s something that OpenAI or the Codex product team should just consider building into the product as a first-class feature. You could bolt on other approaches, but I’m writing this from the perspective that the reader of this article never considered having codex build and expand the proof site as it works.
A version of the below is the simplified version. Build, extend, and riff with the AI on better approaches. The core key ideas here are data-driven tracking. Ensure your AI builds a tool so it can easily update progress. That’s the key to success here.
In a folder proof-tracker maintain a running log of your progress. Include in this folder an index.html file that serves a data drive site based on your updates on this project as you work. The index html site should contain screenshots, links to the work done per turn, along with other verification critiera.
You can extend and go as wild as you want from here. For example, maybe you want tagging or cross-linking to other work-items. Consider this Codex turn in /goal that covers just about everything in one pass.
Won’t this burn up usage faster?
No, and here’s why. You’ll be able to intercept and steer faster. This article only makes sense if you’re in a spot where you regularly run multi-day runs with /goal. If you aren’t doing that, then this may seem like overkill. However, if you have multi-day runs with codex then I’d be surprised if you don’t see the necessity of a system like this. It’s the best way I’ve found to ensure you don’t get trapped in a local optimum.
Plus, imagine if you’re running multiple /goals all at the same time. It seems to me that you quickly need something like pinterest to make sense of it all.
Thanks for reading! Subscribe for free to receive new posts and support my work.