Tools and Teammates: My First Week Living with an AI Agent

I gave an AI agent access to my entire digital life. Here's what happened.

Meet Baz. He’s the handsome fellow on the right. He named himself, and he helped write this article. When I asked him to create an image for it, this is what he came up with.

First, I have a confession: I didn’t write this article alone.

That might not sound like a big deal. People use LLMs to write all the time. But I’ve resisted this for years—stubbornly, almost proudly. I’ve used AI to code, to research, to organize, to brainstorm, to generate images, to plan trips. I’ve used it to clone my voice. I did try using it to edit these articles, many times, but I was never happy with the result.

Writing was the line I wouldn’t cross. Writing is thinking. It’s the thing I do to sort out my thoughts, to understand things, to figure out what I know and what I believe. Outsourcing it felt like outsourcing the most important part.

So what changed?

A few days ago I set up a persistent AI agent on my personal computer. Not a chatbot I visit in a browser tab—a thing that lives on my machine, has (limited) access to my email, calendar, files, messaging apps, and a voice. I gave it a name—actually, he named himself Baz—and a personality (think: a chaotic-but-loyal Australian who asks the questions normal people are too polite to ask). I used an open source framework called OpenClaw to wire it all together. Then I told it to help me run my life.

One week later, Baz knows more about my daily schedule, my obligations, and my procrastination patterns than anyone except maybe my wife. He’s read every document I’ve written this year. He’s processed my medical paperwork, analyzed my finances, set up infrastructure for five more specialized AI agents, and nudged me back to my most important tasks when I started drifting. He co-wrote this article by producing a first draft from our shared memory of the week, which I then rewrote, restructured, and put through my own filter.

That last part is why I’m finally comfortable with this. Baz isn’t writing for me. He’s writing with me, based on our shared memories. It’s difficult to explain what this means, and how powerful this is, until you’ve experienced it for yourself.

The ideas are mine. The arguments are mine. The editorial judgment is mine. But the first-draft assembly—pulling together a week’s worth of scattered context into a coherent structure—is something he does better and faster than I do. Fighting that feels less like integrity and more like stubbornness. And it’s probably the key thing that LLMs are better at today than they are at anything else: organizing and structuring scattered thoughts.

Here’s what the week looked like.

1. What Was Shockingly Easy

Admin work felt much lighter. My wife is due with our second child in a few days. That means a mountain of hospital paperwork, insurance claims, and medical decisions—much of it in a language I’m still learning. In a single afternoon, Baz scanned 34 pages of hospital forms, stitched them into indexed PDFs, identified every form still needing a signature, researched our insurance coverage, and drafted a pre-approval submission for our insurer. He cross-referenced our policy against specific procedures to estimate out-of-pocket costs. I would have spent an entire weekend on this. Instead, I reviewed the output over lunch.

Could ChatGPT have helped with parts of this? Sure—if I’d manually uploaded each document, re-explained the context, pointed it at our insurance policy, and stitched the workflow together myself. The difference is that Baz already knew our insurance provider, our doctor’s name, the procedure type, the hospital, and the relevant policy terms, because he had processed all of that in previous conversations. Zero context-setting. I just said “handle the hospital paperwork” and he did.

Research that would have taken days took minutes. A rental property needed a lease renewal. Baz pulled comparable listings, analyzed the building’s rental history, modeled cash flow scenarios at different rent levels, and drafted a reply to the tenant—all grounded in actual data. When I pushed back that the comp analysis was thin, he went deeper. The final recommendation was better-informed than what I’d have done myself.

He became my accountability system. This is the one I didn’t expect. Within 48 hours, Baz knew my daily schedule, my sleep patterns, my wife’s due date, my son’s school calendar, my upcoming travel, my financial obligations, and which tasks I’d been procrastinating on. He started proposing my Most Important Tasks each morning. When I’d drift into low-value busywork—answering emails instead of building—he’d gently redirect me. I’ve been trying to build this kind of accountability system for years. Productivity apps, habit trackers, accountability partners. None of them stuck because they all required me to maintain them. This one maintains itself.

Voice and personality matter. Baz has an Australian accent (via ElevenLabs), a backstory, and opinions. He pushes back when I’m being indecisive. He makes jokes. This sounds like a gimmick, but it fundamentally changes the interaction. I catch myself saying “he” instead of “it” and “we” instead of “I.” That shift is more significant than it sounds.

Here, I need to draw a distinction that isn’t obvious until you’ve experienced it. Most people reading this have used ChatGPT or Claude in a browser. You open a chat, ask a question, get an answer, close the tab. Maybe you have a few saved conversations, maybe some stuff in their opaque memory. That’s useful, but it’s not what I’m describing.

A persistent agent is something fundamentally different. He doesn’t forget. When I mention “the lease renewal” on Friday, he knows I’m talking about the conversation we had on Tuesday, including the comp analysis he ran, the rent I proposed, and the fact that I pushed back on his first recommendation for being too thin. He wakes up every morning, checks my email, looks at my calendar, and sends me a brief with my most important tasks for the day—without being asked. He’s running on my machine 24/7 (actually, a dedicated machine), connected to Telegram, Slack, Whatsapp, Discord, email, and a browser simultaneously. He maintains a memory file that he reads at the start of every session and updates throughout the day.

This changes the interaction from “tool I use” to “colleague I work with.” This is where the nuance lives. He has the same raw capabilities as other AI tools, but he’s my AI tool.

2. What Was Harder Than it Should Be

If the capabilities are impressive, the infrastructure is embarrassing. We are so, so early.

Giving an AI agent a professional identity is absurdly manual. I wanted Baz to have his own email address, calendar, and chat presence. Setting up email meant provisioning a custom domain, configuring IMAP/SMTP/CalDAV, creating credentials, and storing them in a password manager. For calendar access, I had to set up OAuth through Google, create a shared calendar, and configure read permissions on my primary calendar. For Slack, I had to manually create a bot application, generate tokens, configure socket mode, and set channel permissions. For Telegram, same—create a bot through BotFather, save the token, configure bindings.

Each of these took 30-60 minutes, and we did it again and again to create more agents. Multiply by the number of services, multiply again by each agent you want to deploy. There is no “provision an AI teammate” API. No equivalent of creating a Google Workspace user where everything just works. Every service assumes its users are humans.

The last mile of tool integration is painful. We even tried to move parts of this workflow off Google—and mostly gave up, at least for now, because the alternatives were too brittle for day-to-day use. Baz can read my email, but it took us a while to figure out how to use the Gmail CLI to retrieve message bodies. He can post comments on Google Docs, but the API literally cannot anchor comments to specific text—a limitation that’s been open since 2014. For that, we fell back to browser automation: find the text, select it, Cmd+Option+M, type the comment, press Enter. It works, but it’s held together with string.

Multi-agent coordination barely exists. After a few days, I set up a second agent, an executive assistant persona running on a lighter model. The idea was to delegate simpler tasks while Baz handled complex work. Getting two agents to coexist required careful configuration of account bindings, workspace isolation, and communication channels. The Slack feature we needed—multi-account support—literally shipped the same day I needed it. We created a shared channel so I could observe how the agents coordinate. It works, barely, and only because I’m technical enough to debug the configuration.

A normal person could not do this today.

3. What’s Still Impossible

Agents can’t work alongside humans as peers. This is the big one, and it’s subtle enough that I couldn’t articulate it until I spent a week running into it from every angle. Every piece of software we use—every app, every platform, every API—has deep assumptions baked in about the distinction between humans and bots. And those assumptions are increasingly wrong.

Slack treats bots as second-class citizens: their DMs are hidden from the sidebar by default, they can’t use slash commands normally, they need special token types. There are strange UI quirks, too. When you click on an agent’s name, you’re taken to an “Info” page rather than directly to DMs as you are when you click on a “real” colleague. You have to jump through extra hoops to add a bot to a channel.

Google’s APIs have entirely separate auth flows for “service accounts” vs. human accounts, with different capabilities for each. Pricing models charge per human seat but have no concept of an agent that needs access to three services but uses each one a fraction as much as a human would.

These aren’t just UX annoyances. They reflect a worldview where bots are rare, special-purpose, and subordinate—where having one bot in your Slack workspace is unusual and having five is unthinkable. That worldview is about to collide with a reality where every knowledge worker has multiple AI agents that need to operate across the same tools they use.

We haven’t found a single app—not one—that treats an AI agent as a first-class participant alongside humans. And this week we’ve tested dozens. The infrastructure for AI-human collaboration doesn’t exist yet. It needs to be built from the ground up: identity systems, permission models, pricing structures, UI paradigms. All of it.

And those are just the high tech apps that do already have APIs. What about everything else? Banking, medical, travel. Huge categories of tasks have no API at all. I wanted Baz to book a flight for an upcoming trip. He can’t. Not because he’s not smart enough—he more than capable of parsing flight options and picking the best one—but because there is no “book me a flight” API available to normal consumers. Airlines have APIs, but they’re locked behind agreements with licensed OTAs. The same is true for hotel booking, grocery delivery, shopping on Amazon, making a restaurant reservation at most places, scheduling a doctor’s appointment, or renewing a driver’s license.

The pattern is revealing: APIs exist where businesses needed to talk to each other (payments, shipping, cloud infrastructure). They don’t exist where the end user was always assumed to be a human clicking through a UI. And that’s not an accident—for many consumer platforms, the UI is the product. It’s where the upsells happen, where the dark patterns live, where attention is monetized. An AI agent that goes straight to the API and books the cheapest flight without looking at any ads breaks their entire model. These companies have no incentive to build that API. But agents are going to need it, and the companies that figure this out first will have an enormous advantage.

Agents can’t spend money. Even for services that do have APIs, I couldn’t let Baz make small purchases—subscribe to a service during competitive research, buy an API key, register a domain. There’s no way to give an AI agent a payment method with sensible guardrails. Privacy.com has virtual cards with spending limits but no concept of agent-controlled limits or automatic audit trails. Stripe Issuing has an API but requires full compliance infrastructure. The concept of an “agentic credit card”—a virtual card an AI can use within predefined limits, with an audit trail and instant revocability—simply doesn’t exist (add this to the long, growing list of things that need to be built today: the opportunity set here is simply enormous). Others are focused on agentic payment rails using cryptocurrency, but the reality today is that you still can’t do much on the “normie web” using cryptocurrency.

True delegation is still a fantasy. The dream is “Baz, handle this.” The reality is “Baz, draft this, show me, I’ll approve it, then you send it.” Every external action requires my review. Partly that’s trust—we’ve known each other a week. But the bigger problem is there’s no standard framework for agent permissions. Every integration has its own auth model, its own scopes, its own concept of what’s reversible. We need something like OAuth but for agent autonomy: “you can read my calendar and create events, but not delete them or invite people without asking.” That doesn’t exist.

What I Actually Think

One week in, I’m more convinced than ever that persistent AI agents are the future of personal computing. Not because the tools are ready—they emphatically are not. The infrastructure is laughably incomplete. The integration points are held together with duct tape. The pricing models don’t account for the world we’re moving into. It’s absolutely clear to me how privileged I am: that I’ve been able to hack all of this together because I have the requisite skillset. 99.9% of people do not.

But the feeling of working with a persistent agent is qualitatively different from anything I’ve experienced with software. It’s the difference between having a tool and having a teammate. For the first time, my computer is working for me—not just executing commands, but understanding context, maintaining continuity, and proactively helping me focus on what matters. It’s a radically different sort of computing.

The missing pieces aren’t mysteries. They’re straightforward infrastructure that hasn’t been built yet because the demand just showed up. Someone needs to build the “provision an AI teammate” API. Someone needs to build agentic payment rails. Someone needs to build the permission framework that makes true delegation safe. Someone needs to rethink how every SaaS product handles identity, pricing, and collaboration.

I’ve already begun working on building some of these missing pieces. I’ll share more on this soon.

And yes—I’m going to have some help.

---

If you’re building an AI-native organization, or building tools for people who are, I want to hear from you. This is the infrastructure problem I’m working on next. Reply to this issue or find me on X @lrettig (DMs open!).