Every other backend in this series is a company's API: you paste a URL, a key, and a model name, and you pay or hit a cap. KoboldAI is the exception. KoboldCpp runs an open-weights model on your own computer, serves it on an OpenAI-compatible endpoint, and charges nothing per token, because there is no company in the loop. There is no key to leak, no daily cap, no privacy trade, and no bill. It is the most self-contained way to run Janitor AI.
It is also the one with a catch that none of the others have, and the catch is not the model. It is that Janitor AI runs in the cloud and your model runs on localhost, and the two live in different places. This guide is honest about that from the start, because a KoboldAI page that skips it sends people to paste localhost into a website and watch it fail.
Why localhost does not work, and what does
KoboldCpp, once running, serves its OpenAI-compatible API at an address like http://localhost:5001/v1. On your own machine that address means your machine. But Janitor AI is a website running on Janitor's servers, and when you paste localhost into a website's settings, the website looks for that address on its own computers, where nothing is listening. The request never reaches you.
The fix is to give your local endpoint a public address that Janitor can reach over the internet, which is what a tunnel does. A tunnel opens a connection from your machine out to a public HTTPS URL and forwards anything that arrives there back to your local server. Cloudflare Tunnel and ngrok are the common ones; a reverse proxy on a server you own works too. The public URL the tunnel gives you, ending in /v1/chat/completions, is what goes in Janitor's Proxy URL field, not localhost.
KoboldCpp on your PC
OpenAI endpoint at localhost:5001/v1
A tunnel
Cloudflare Tunnel, ngrok, or your own reverse proxy
A public HTTPS URL
this is what Janitor can reach
Janitor's Proxy URL
the public URL, ending in /v1/chat/completions
Your model answers
no key, no token cost, chats stay on your machine
The values
| Field | Value |
|---|---|
| Config Name | Local Kobold |
| Proxy URL | your tunnel's public URL, ending in /v1/chat/completions |
| API Key | blank, or a placeholder, or your tunnel password if you set one |
| Model | the model KoboldCpp loaded (its name as the endpoint reports it) |
KoboldCpp has no API key by default, which is why the field can be empty. If you protected the tunnel with a password, put that password in the field. Save, refresh, and test as with any backend; a network error here almost always means the tunnel is down or the URL is missing /chat/completions, per the pattern in our network error guide.
What you get, and what you give up
The appeal is real and worth stating plainly.
- No cost per token. Nothing to fund, no cap to hit, no 429 from a provider. The only meter is your electricity bill.
- Privacy. Your messages go from Janitor to your own machine and back. No provider stores them, and there is no key that a leak could expose. This is the strongest privacy position of any backend, stronger than a paid API, because there is no third party at the model at all.
- No provider policy. An open-weights model runs what it runs; there is no external content policy declining scenes. What the model was trained to do is what it does.
The costs are equally real.
- Setup is work. You install KoboldCpp, download a model file, and stand up a tunnel. SillyTavern's own documentation, describing the same class of local backend, notes that the installation can be complex and that its team does not provide support for it. Budget an afternoon the first time.
- Your hardware is the ceiling. The model you can run is the one your GPU or RAM can hold, which is smaller than the frontier models on OpenRouter. For many roleplayers a good mid-size local model is plenty; for others the quality gap is the reason they pay. That is a taste call, and your machine decides the menu.
- The machine has to stay on. Close the laptop or lose the connection and the tunnel drops and the endpoint dies mid-chat. A desktop left running is the natural home for this.
Where a network proxy fits, which is almost nowhere
This is the backend where our own product has the least to do. The model is on your machine; the bridge is a tunnel; neither is something an IP-changing proxy touches. There is exactly one case, and it is the generic one shared by every backend: if janitorai.com itself is unreachable from your network or country, a network proxy reaches the site, and once the site loads, your tunnelled model answers as normal. Janitor AI blocked UK addresses from July 2025 until it switched to age verification in June 2026, which is the kind of wall that case describes. For that, a static residential or ISP address in a served country, from $0.44 per GB on a usage meter, is the tool; for the model, the tunnel and the cost, it is irrelevant, and we would rather say so than imply a local setup needs us.
If you want the frontier-model quality that a local model cannot match, the free hosted routes are the alternative, at the cost of a cap and a provider policy: OpenRouter's :free models and Google's Gemini free tier, both in best free Janitor AI proxy. The local route trades that quality for cost, privacy and no cap, and for a lot of people that is the trade worth making.
Sources
- SillyTavern documentation, introduction and API connections: local backends including KoboldCpp, the localhost endpoints, and the note that local-backend installation can be complex and is unsupported by the dev team.
- Janitor AI Help Center, The Absolute Beginner's Guide to Using a Proxy with Janitor, Part One: the proxy field and the menu path.
- Janitor AI, Tough news for our UK users and An update for users in Brazil, Australia, and the UK: the UK block and the move to age verification.