agents / 09

I’m done with Captain.

I started the project because booking a flight seemed like exactly the kind of problem an agent should solve.

Currently, our assistants help with travel. They search different websites, put the best options in a doc and send for review. The work is repetitive, the objective is clear and most of the process happens in a browser. I thought - surely, this can be automated.

I spent the last few months trying and didn’t succeed, but I got far enough and decided it wasn’t a problem worth solving.

Browser control

My first approach was browser control. I wanted Captain to visit the same website as we would, search for flights, compare the results and eventually complete a booking.

General-purpose coding agents can do some of this now, but I wanted to understand what it would take to build the capability into my own software. That took me through agent frameworks, tool interfaces, memory, long-running jobs and the different ways of getting structured information into a model.

Captain also became an experiment in how much software I could make with coding agents. I used Codex heavily, tried different models and kept increasing the complexity of the idea. I explored conversational interfaces, built a visual workspace to sit alongside the chat, and used HyperFrames while trying to make a film about the product.

The project was useful as a reason to touch all of these things. But every new capability also made Captain more complicated than the simple automation I had imagined.

Inventory

The first hard problem was inventory.

A person can search several travel websites and use judgment to reconcile the results. Software needs a dependable source of structured data. I tried browser searches, web search and a Codex worker that searched the internet in the background. Some of it worked, but none of it felt dependable enough to build a public product around.

Eventually I got good flight data from Duffel. The results were useful and the prices were competitive enough to keep going. Captain could search immediately with Duffel, then use a slower internet search to look for alternatives.

Getting reliable results did not make the product simple. It exposed the operating cost of continuously searching and storing them. In one early test, 16 trips created more than 11,000 flight records in 24 hours. I had built a system that could watch everything, so it tried to watch everything.

I redesigned the worker to search less aggressively. Matching trips shared results. Distant flights were checked less frequently. Captain kept a compact price history and only sent a message when something changed meaningfully.

One flight

I narrowed Captain down to a single job: track one flight until departure.

You could describe a trip in Telegram using text or voice. Captain would resolve the airports and dates, ask for whatever was missing and build a complete brief. A dashboard showed the search activity, preferences, options and price history. Every day, Captain checked the selected flight and returned a buy or wait verdict.

If you wrote in another language, Captain would notice and start replying in it — daily alerts included. There was nothing to configure. It picked the language up from the conversation itself.

The dashboard also kept a summary of the activity: tracking started, a flight selected, the plan changed. Meaningful checkpoints, not every chat message or search refresh.

This mix of conversation and interface was one of the most interesting parts of the project. Chat was good for expressing intent, especially for trips with several cities or fuzzy dates. The interface was better for comparing options and changing decisions. Neither was enough on its own.

I put the product into beta and made the Telegram bot public. Captain could talk about travel, plan a complex itinerary and track prices. It didn’t book flights or hold passenger records. Booking and payments would come later, once I trusted the recommendations.

To enable actual bookings, I’d have to reach out to Duffel. Until then I had mostly been asking whether I could make the software. Now I had to decide whether I actually wanted to turn it into a product.

Dependability

I underestimated how much work it would take to make a conversational agent feel dependable.

Captain often understood a request, but occasionally just did the wrong thing. It might ask for something the user had already provided, make a bad assumption about a date or fail halfway through creating a trip. Each mistake would send me back to Codex: read the conversation, work out what went wrong, change the prompts or tools, deploy, then try again.

I built an admin to watch conversations, jobs and model costs. I added automated conversation tests and a daily review that inspected failures. Both Captain and Pilot had repair loops: when a test failed, another agent investigated the problem and proposed a fix.

This was fascinating, but also frustrating. I could keep improving individual failures without feeling like I was in control of the whole system. A repair loop did not remove maintenance. It created another system that needed to be watched.

The trial-and-error process is manageable while learning. It is less appealing when the result handles real itineraries, changing prices and eventually people’s money.

The product question

Captain didn’t have proprietary flight data. It was taking information available elsewhere and placing a conversational layer over it. Meanwhile, the general-purpose chat and coding tools I was using to build Captain were getting better at searching the web and controlling browsers themselves.

That made the value difficult to defend. Captain was not as broad or as capable as a general chat app. Its specific utility was checking a price every day and telling me when something changed. That was useful, but not useful enough for me to keep operating the product.

More importantly, it was not better than the human experience I already had. I still preferred sending a trip to my assistant, reviewing a considered set of options and knowing that a person could use judgment when something unusual happened.

Yesterday, it got another conversation wrong and I decided to shut it down.

Pilot

With Captain gone, I can focus on my personal agent, Pilot. It’s the bookmark-reading, Telegram-speaking, Pi-hosted result of all this exploration.

If I was working on a team paid to build a travel agent, I’m sure we’d be able to get Captain to work. But as a side project, I couldn’t hack it.

Was fun though.