Jaisiu

Local GGUF agent gateway with optional cloud fallback

Jaisiu is a local GGUF agent gateway. Your agents run on your machine by default — same inference engine Ollama uses, the same GGUF models you already know. The Pryzm broker is an opt-in fallback for tasks that need a bigger model, with a hard token cap so agents pause instead of running up a surprise bill.

How routing works

  1. Local first. If a task is within your model's context and capability, it runs locally. No token spend, no network call.
  2. Broker fallback. If the task is too large or too hard, the agent hands it to the broker. The broker uses a pooled token cap; once the pool is empty, agents pause.
  3. No silent cloud. A task never leaves your machine unless it goes through the broker, and the broker log shows exactly what was sent.

Which models work

Anything that loads in llama.cpp: Qwen, Gemma, DeepSeek, Mistral, Phi, Llama, GLM, and the small Mistral-derivative set Jaisiu ships by default. Drop the GGUF file into ~/.jaisiu/models/, restart the gateway, and the next task can use it. The broker is API-compatible with the OpenAI chat completions shape, so any model the broker supports (Qwen, Claude, GPT-4 family) works the same way.

The token pool

The pool is part of your license. Basic is 200M tokens / month, Fleet is 4B / month. Add-on packs (100M for $29, 400M for $69) extend the pool without a subscription change. Hard cap — agents pause at zero, never overage. You can see the current pool balance in Jaisiu → Settings → Usage.

Get started

Install: one command, real installer script, no Docker required. Read the script before running it. See docs/models for the routing config and CLI flags.