Most AI tools are a website you visit. You open a tab, you type, you get an answer, you close the tab and it’s gone.

I wanted something different: an agent that lives on my own server, is always running, remembers things between conversations, and that I can reach from my phone without opening a browser at all.

That’s Hermes, and it’s been running on the Fedora box for a few months now.

What it is

Hermes is an agent framework you install on a machine. It has a dashboard, a command-line interface, a memory store, and gateways that connect it to things like Telegram.

The distinction that matters: it isn’t a chat window. It’s a process on a server with its own state. You talk to it, it does something, it remembers. Next week you refer back to that and it knows what you meant.

Install is one command and it runs on Linux and macOS.

The model question, and a surprise

This is the part I’d most like to pass on, because it saved me real money.

An agent needs a model behind it. The obvious route is an API key, where you pay per token. That works and it adds up quickly, especially with an agent that runs in a loop and uses a lot of context without you watching.

But you can also point it at a subscription you already pay for. If you have a flat monthly plan with one of the big providers, the agent can run against that instead of a metered key. Same model, no per-token meter.

For what I do, that changed the economics completely. I stopped rationing. I’ll let it work through something long and clumsy, because a slow answer costs the same as a fast one, which is nothing extra.

Two honest caveats. There are rate limits, and you will meet them if you leave something grinding away. The first time I hit one it wasn’t obvious what had happened, because the error came back looking like a generic failure rather than “you’ve used your allowance.” And you’re tied to that provider’s models. An API key gives you more choice.

You can also point it at a local model via Ollama, which costs nothing at all and is noticeably less capable. I use that for the small stuff. One gotcha there: Hermes wants a large context window and the default Ollama models don’t offer enough, so you have to build a variant with a bigger context before it’ll accept it.

Mostly I run on the subscription.

Getting to the dashboard

Hermes has a web dashboard, and it deliberately refuses to listen on anything but localhost. No flag, no config, it just won’t, unless you explicitly tell it to run insecurely.

Which is the right default and briefly annoying.

The clean answer is an SSH tunnel. Start the dashboard on the server, then from your laptop:

ssh -L 9119:127.0.0.1:9119 you@your-server -N

and open localhost:9119 in your browser. The dashboard thinks it’s talking to something local. It is. The tunnel does the rest.

It took me embarrassingly long to work out why this wasn’t connecting, and the answer was that the dashboard process wasn’t actually running on the server. Start it there first, then tunnel. In that order.

Telegram, which is the bit I use daily

Hermes can run a Telegram gateway, which means I have a bot in my Telegram contacts that is my agent. Message it from anywhere, get an answer, carry on.

This turned out to be the feature I use most, by a long way. Not because Telegram is special but because it means the agent is in the same place as my messages rather than behind a login on a website.

One thing that will bite you if you run Hermes in two places: only one instance can use a given bot token. Telegram allows exactly one poller per bot. Start a second one and you get conflict errors that don’t explain themselves.

I have Hermes on both the server and my laptop. The server runs the Telegram gateway. The laptop is command-line only and never starts a gateway. Worth deciding this deliberately rather than discovering it.

If you do want both machines set up the same way, hermes backup on one and hermes import on the other will seed it. Keys don’t come across in the export, so you re-enter those.

Talking to it on my phone

The last piece is an Android app called Hermes-Relay, which gives you a hands-free voice client pointed at your own instance over Tailscale. Talk to it in the car, get an answer back.

This took more fiddling than it should have. I’d put the dashboard behind Tailscale’s TLS proxy, and the app was sending plain HTTP, so everything came back as a confusing 400 that mentioned nothing about certificates. The fix was to stop wrapping it in TLS and let the app talk plain HTTP across the tailnet, which is already encrypted anyway.

That’s worth generalising: on a tailnet, adding TLS on top is often just a second thing to go wrong. The transport is already private. I keep reaching for a certificate out of habit and then removing it.

Why this instead of a chat window

Three reasons that have held up.

It remembers. Not just within a conversation. Across them. I can refer to something from weeks ago and it’s there.

It’s reachable. Telegram from my phone, voice from the car, command line from any machine on the tailnet, dashboard when I want to see what it’s doing. Same agent behind all of them.

It’s mine. It runs on hardware in my house. The memory is on my disk. When a provider changes something, I change a config line rather than waiting for a product decision.

It’s more setup than opening a website. Considerably more, and some of it was genuinely irritating. But what you get at the end isn’t a better chat window. It’s a thing that lives somewhere and is there when you want it, which turns out to be a different kind of useful.