Hermes Agent: The Complete Self-Hosted Guide — Desktop Install to a Local-Model Fleet

I've written this whole series one piece at a time — what an agent even is, how to install one, how to give it a phone, eyes, a local brain, and a whole team of siblings. This is the map that ties it together. Each section is a stage of the journey, told in a few sentences, with a link to the post that actually walks you through it. Read it top to bottom to see the shape of the thing, or jump to the stage you're on.

An agent uses your tools to finish jobs — that's what Hermes is

You mostly use ChatGPT one question at a time: you ask, it answers, you do the next step yourself. An agent is different — it holds a goal, reaches for tools, and finishes the job, running on your side and plugging into apps you already use. Once you accept that you want an agent, the next move is not to build one; the hard parts (memory, tool-calling, a loop that doesn't fall over) are already packaged in a framework, so you install one and go. The framework I use is Hermes, and the mental model that makes the rest of the guide click is brain plus body: your ChatGPT account is the brain that thinks, Hermes is the body that acts, and that's one fixed combo with nothing to shop for.

Start here if the whole idea is new: what an agent is versus a chatbot, why you install a framework instead of rolling your own, and the brain-plus-body combo this guide uses.

Your first working agent takes about 15 minutes, on Mac or Windows, no terminal

This is the stage where it stops being theory. Hermes ships a real desktop app — a .dmg on Mac, an .exe on Windows — that you install like any normal program, and on first launch it auto-installs everything it needs (Python, Node, Git, ffmpeg) so you never open a terminal or edit a config file. For the brain, you pick "OpenAI OAuth (ChatGPT)" and sign in with your existing ChatGPT account in a browser window — no API key, no separate bill; the credentials just get stored locally. "Done" looks unremarkable on purpose: you type a message, it replies and can use a tool, and that's your first agent running on your own machine.

Full walkthrough with screenshots: install Hermes Desktop on Mac or Windows.

Run it from your phone, then hand it a task that works while you sleep

An agent stuck on your laptop isn't much of an assistant, so the first daily-use move is to reach it from your phone. You create one bot with Telegram's official @BotFather, hand Hermes the token, and message the bot — no public URL, webhook, or tunnel, because Hermes polls for messages itself. (If you're in Taiwan or Japan, LINE connects the same way; the English walkthrough uses the Telegram path.) Then comes the payoff that makes it feel alive: you describe a recurring job in plain words — say, research a topic each morning and summarize it — and Hermes runs it on a schedule and pings your phone with the result while you're doing something else.

The two posts for this stage: connect Hermes to Telegram for phone control and set up a self-running daily task that messages you.

Give it eyes and ears, a local brain, and hands on your own tools

Once the basics work, the advanced stage is a set of jobs you hand the agent, each of which upgrades what it can perceive or touch. Give it senses: a text-only brain can't read a photo or a voice clip, so Hermes bolts on a small vision model as a perception side-car and faster-whisper for speech, and now a screenshot or a voice memo just works. Give it its own brain: swap the cloud ChatGPT model for a local one — Ollama with Gemma for something light, or a bigger ds4 — and you get full autonomy with no provider and no caps, at the honest cost of speed. Give it hands: wire in your own tools over MCP, the "universal outlet" standard, so it can read your folders, run your commands, and call services you wrote.

The runbooks: vision and voice as a perception side-car, swapping in a local brain for full autonomy, and connecting your own tools via MCP.

Raise a whole fleet — and watch it from your phone

Eventually, one agent becomes a team. Hermes lets you run multiple "sibs," each with its own config, memory, personality, and even its own model, so you can split work across specialists (most people only need one — this is for when you like to tinker). Operating a fleet means you'll hit weird behavior, and the debugging lesson worth internalizing early is that eight times out of ten a broken agent is the harness around the model — tools, config, memory — not the model itself. To keep tabs on the fleet without living in a terminal, the Muninn iOS app reaches your home agent from anywhere over an encrypted iroh P2P tunnel (no cloud in the path) and pulls Hermes's Kanban board onto your phone so you can see who's on what. And the fleet actually does work: I gave three sibs the same one-line "make Tetris" spec and each shipped a playable game in one shot, then had a local 27B model use Hermes's /learn to turn a finished task into a reusable skill it wrote itself.

The fleet posts: run multiple sibs, each its own agent, why 8 of 10 failures are the harness, not the model, reach your home agent anywhere with Muninn over iroh P2P, watch the fleet on a phone Kanban board, one spec, three sibs, three Tetris games, and /learn, where an agent authors its own skill.

Where to start

If you don't have Hermes running yet, ignore everything past the install stage and go straight to the desktop install section above — get one agent talking to you first, and add the rest later. If it's already running and you want more out of it, jump to the advanced runbooks and the fleet stage: give it senses, a local brain, and your own tools, then scale it out and watch it from your phone.

If you want the broader, tool-agnostic version of this topic, see How to Run an AI Agent from Your Own Desktop.