How I built a private LLM router: one boring lookup table decides what leaves my network

My cloud LLM tier is net zero dollars. Groq handles the quick sorting-and-labelling work on a free quota I have never exhausted, Cerebras runs the cloud reasoning lane on another one, and Codex through my ChatGPT login gives me a top-tier coding model with no metered bill. The work that genuinely needs Anthropic’s Claude still goes to Anthropic. Everything else runs on what AI providers give away to court developers.

A private LLM router holds it together. I built it in about three hours, in one coding session, and hardened it over a few follow-ups. It now sees every LLM request across four machines and a dozen-plus backends. One day after it went live I tore the privacy model out and rebuilt it, because the first version leaked.

If you read the manual routing playbook I published earlier, this is the sequel. That post was about picking the model by hand for each job. This is the machine that does the picking.

What an LLM router actually is (and why I wanted one)

By March 2026 my daily setup had grown to roughly a dozen AI endpoints (individual model connections) across four machines and five providers: two local Qwen models (an open-weight family I run on my own hardware) on a Mac Studio and an Nvidia 3090 Ti PC, a vision model on the Mac Studio, several Gemini accounts, plus keys for Groq, Cerebras, and Mistral, and Codex through my ChatGPT login. Every endpoint had its own little script, its own retry logic, its own quota counter. None of them could see the others.

A router ends that arrangement. It sits between the things asking questions (apps, automated scripts, AI agents) and the AI models that answer them, reads each request, and decides which model should handle it. From the asking side it is a single web address, hiding the zoo behind it.

The sprawl I could live with. What cost me money was that nothing knew what the whole setup was doing, and four separate jobs all needed that missing view.

Four jobs one router has to do at once

Spreading load across consumers. Coding sessions, agents, scheduled jobs, and tools all hit the same models with no coordination between them. One runaway batch of agents drained my paid Anthropic allowance by Wednesday. I wanted to shift load with a single setting, not a code rewrite.

Free tiers. Cerebras and Groq both give away real daily capacity to court developers, and Codex is effectively free inside the ChatGPT subscription. A paid Anthropic token spent on a yes-or-no question that a free model answers in a fifth of a second is money spent for nothing.

Frontier models, when the task earns one. Multi-step reasoning, novel code review, planning: that work wants a frontier model (the largest and most capable ones, like the models behind ChatGPT or Claude). Rare enough, but I want it reachable without editing a config file.

Privacy. A lot of what I work on I will not send to an outside AI lab: meeting notes, internal documents, draft strategy, customer data, signed NDAs. The router has to let me say “keep this on my own network” and honour it every single time, with no quiet fallback to the cloud when the local machine is busy.

How I built this LLM router in three hours

Two structured plan reviews ran before any code landed: one in “CEO mode” to pressure-test scope, one in “engineering mode” to hunt for gaps, both inside a single Claude Code session where the spec, the reviews, and the parallel coding all lived.

The CEO review made the call that mattered. It flagged the request journal (a running log of every request the router handles) as the highest-value feature and forced it into the first phase instead of “later.” The engineering review caught safe quota writes that cannot corrupt under load, defensive parsing for the Gemini login process, and three providers that all speak the same API dialect and could share one client.

Then the build ran as parallel work, split into phases and handed to multiple coding subagents (separate AI coding sessions working at the same time on disjoint files).

Phase	Work	How	Tests
1	Core logic (config, router, quota tracking, journal)	5 coding subagents in parallel	78
2	Backend clients (the connectors to each provider)	Codex + Opus subagents	48
3	The web app with task routing + streaming	One Opus subagent	14
4	Deploy: run as a service, health checks, live smoke test	Manual	–
5	A monitoring tab on my ops dashboard	One Opus subagent	–
6	A replay tool to re-run old traffic against new models	One Opus subagent	8

Total wall-clock from “let’s spec this” to “it is live”: about three hours. Phase 1’s five parallel subagents all came back with passing tests in under ten minutes, because the plan had already drawn clean enough lines that they never collided.

Later sessions added the Codex connection, automatic retries on flaky failures, health checks every 60 seconds across all backends, a tool that lets my AI agents call the router directly, and a batch helper. The test suite is now roughly 1,500 unit tests plus 70 live integration tests.

Every request carries a job label, and the label picks the model

The caller attaches a task_type to each request: a plain string saying what the request is for, like classify, extract, write, reason, or generate code. The router reads that string and sends the request to the model best suited to that kind of work. Nothing inspects the prompt itself.

The obvious alternative is a cascade: try the cheapest model first, climb the price ladder only when it fails. My local Qwen scores 40% on debugging tasks (full numbers in the 38-task benchmark post), so sending every debugging prompt to Qwen first buys a wasted round trip and a garbage answer before the model that can actually do the work ever sees it.

One benchmark finding drove the whole design: free local models hold their own on classification, extraction, code generation, and writing, and they lose meaningfully on multi-step reasoning and debugging. Here is the table the router runs today. Where a lane lists more than one target, the request goes to whichever is least busy, and the others stand behind it as fallbacks.

task_type (the label)	Eligible backends, least-busy wins
cloud_research	Gemini gemini-2.5-flash (web grounding)
cloud_reasoning	Cerebras gpt-oss-120b, then Groq gpt-oss-120b
cloud_code	Codex gpt-5.4, then Codestral
cloud_writing	Mistral mistral-small-2603
cloud_classifyextract	Groq llama-3.1-8b-instant, then Cerebras gpt-oss-120b
cloud_other	Escape hatch: the caller names the model
local_classifyextract	Ministral-8B on the Quadro sidecar, Ministral-8B on the M4000, or the 3090 Ti (qwen3.6-35b-a3b)
local_code	Mac Studio (qwen3.6-27b-mtp), or the 3090 Ti (qwen3.6-35b-a3b)
local_reasoning	3090 Ti (qwen3.6-35b-a3b)
local_reasoning_nothink	3090 Ti, thinking forced off, or the Mac Studio (qwen3.6-27b)
local_reasoning_secondary	Mac Studio (qwen3.6-27b-mtp)
local_research	3090 Ti
local_writing	3090 Ti (qwen3.6-35b-a3b)
local_synthesize	Mac Studio (qwen3.6-27b-mtp), over retrieved context
local_image	Mac Studio (qwen3-vl-4b-instruct)
local_promptinjecttest	Mac Studio (hermes-3-llama-3.2-3b)

One bit of vocabulary from that table. A mixture-of-experts model, or MoE, keeps a large pool of weights on hand but only runs a small slice of them for each token it produces. That slice is the model’s active parameters. The 3090 Ti’s qwen3.6-35b-a3b holds 35 billion parameters and activates about 3 billion per token, which is why it answers at closer to the speed of a much smaller model.

The single-target lanes fail closed. If the one machine that serves local_image is busy or down, that request errors out rather than quietly going somewhere else, and it never crosses to the cloud looking for a home. I would rather see an error than a prompt on someone else’s server.

Privacy-by-prefix: the pattern that keeps sensitive prompts off the cloud

Every label in that table starts with either local_ or cloud_, and the router makes its entire trust decision from that prefix before it looks at anything else. A local_ request goes to one of my own machines and never leaves my private network. A cloud_ request may go to an outside provider. The caller decides by choosing the label. There is no separate setting to forget and no default that could quietly leak something.

My machines are stitched together by Tailscale, a private network mesh that puts all of my own hardware on one secure tunnel, as if they shared an office LAN even when they are scattered across the city. A local_ request rides that tunnel and never touches the public internet.

The mistake that forced a rebuild

The first version of the router worked the category out for you. That is how it leaked.

Callers passed an unprefixed label like classify or writing, and the router decided per-model from quality scores and available headroom. If no label came in at all, the router read the prompt text and guessed the category from keywords.

I caught the guessing problem while reading the journal one afternoon. Some prompts about internal documents had matched the word “research” in their text, been auto-labelled as research, and routed to Gemini in the cloud. The leak itself was small. Nothing bounded it, though: once you trust a keyword-matcher with your privacy policy, you have trusted it with every prompt it has not seen yet.

The second flaw was quieter. The old router had a rule that if a prompt is too big to fit in the local model’s memory, skip the local machine and use the cloud. That reads as sensible engineering right up until the oversized prompt is also a sensitive one. The request goes to Gemini, and nothing in the response mentions it.

Both behaviours got deleted in a single patch. The label is now required, and a missing or unrecognized one returns an error. The local tier never falls back to the cloud, so if no local machine has room the request fails rather than escaping to a provider. Across a trust boundary, every convenience that guesses on your behalf is a way over the line that nobody signed off on.

Journal entries document themselves now, too. A line reading task_type: cloud_classifyextract records that someone declared this prompt safe for the cloud at that moment.

What the LLM routing daemon looks like under the hood

The entire routing table is one dictionary: a lookup that maps each label to the model tier it belongs to, with no clever rules and no pattern matching.

_ROUTES: dict[str, Tier] = {
    # cloud-only categories
    "cloud_research":           Tier.GEMINI,    # web grounding
    "cloud_reasoning":          Tier.CEREBRAS,  # gpt-oss-120b
    "cloud_code":               Tier.CODEX,     # gpt-5.4 (ChatGPT auth = effectively free)
    "cloud_writing":            Tier.MISTRAL,   # Gemini Pro quota is zero on current API keys
    "cloud_classifyextract":    Tier.GROQ,      # llama-3.1-8b-instant
    "cloud_other":              Tier.CLOUD_ANY, # caller names the model

    # local categories
    "local_classifyextract":    Tier.LOCAL,     # ministral-8b sidecars, 3090 Ti behind them
    "local_code":               Tier.LOCAL,     # Mac Studio qwen3.6-27b-mtp, 3090 Ti behind it
    "local_reasoning":          Tier.LOCAL,     # 3090 Ti
    "local_reasoning_nothink":  Tier.LOCAL,     # same, thinking forced off
    "local_reasoning_secondary":Tier.LOCAL,     # Mac Studio qwen3.6-27b-mtp
    "local_research":           Tier.LOCAL,     # 3090 Ti, caller pre-fetches context
    "local_writing":            Tier.LOCAL,     # 3090 Ti
    "local_synthesize":         Tier.LOCAL,     # Mac Studio, RAG over retrieved context
    "local_image":              Tier.LOCAL,     # Mac Studio qwen3-vl-4b-instruct
    "local_promptinjecttest":   Tier.LOCAL,     # Mac Studio hermes-3-llama-3.2-3b
}

Sixteen entries, no precedence rules. When I am not sure where a request will go, I read one screen of code.

The request flow has the same shape: validate the label, route by prefix, pick a machine on that side, adjust the prompt if needed, dispatch, log.

  Request arrives at /v1/chat/completions
          |
          v
  ┌─ task_type in body? ── no ──> HTTP 400 (explicit required)
  │         yes
  │         v
  │  task_type valid? ── no ──> HTTP 400
  │         yes
  │         v
  │  route by prefix:
  │    local_*  -> Tailscale-only backends (Mac Studio / 3090 Ti PC)
  │    cloud_*  -> external (Groq / Cerebras / Codex / Gemini / Mistral)
  │         |
  │         v
  │  within-tier selection:
  │    least-loaded backend that can serve this task
  │    ties broken round-robin
  │    pinned tasks (writing, image, judge) go to their one machine
  │         |
  │         v
  │  prompt adaptation:
  │    local+classifyextract:  disable thinking (saves ~2950 tokens)
  │    local:                  cap max_tokens, lower temp
  │    cloud_research:         enable Google Search grounding
  │         |
  │         v
  └──> dispatch, journal, update quotas

Some task types are pinned to a single host and a single model, because those jobs only run in one place. Image work always goes to the Mac Studio, the only machine in the house with a vision model loaded, and the prompt-injection judge is pinned to its own small model on the same box. Neither lane has a second machine, so neither has a failover. The pins used to be two hand-maintained dictionaries that drifted every time I moved a model; they are now derived from one table that names the backend and the model together, so a lane cannot end up pointing at a host that no longer runs it.

The pins exist because of a problem you only find by running real mixed traffic.

Sidebar: the model-swap traffic jam

The Mac Studio runs a tool called LM Studio with two models defined, a text model and a vision model. To save memory, LM Studio loads only one at a time and swaps on demand. That swap takes 10 to 30 seconds, and every request waiting on that machine stalls for the whole window. Mixed traffic (a text job, then an image job, then a text job) made it ping-pong, and throughput collapsed. Anything unlucky enough to arrive mid-swap went from under a second to 30-plus seconds.

My fix was to pin a default text model for the quick-labelling jobs. The router still spreads that work across machines, but the Mac Studio now stays loaded with its text model whenever no one is doing vision, so image traffic pays the swap cost only when an image actually arrives. The name for an on-demand load like this is a JIT swap (just-in-time), and it is convenient until two kinds of work share one machine.

Sidebar: the 502 that was three silent failures stacked

A routine job started failing with 502 errors on every classify request. The journal showed empty responses coming back from the 3090 Ti. The router’s silent-failure detector caught them, retried on the Mac Studio (same empty result), and reported the failure cleanly to the caller.

The real bug was buried deeper. The local backend had been prepending a text instruction (/no_think) to tell the model to skip its reasoning step, assuming the newer Qwen model obeyed the same directive the older one did. It did not. The new model ran its full hidden reasoning anyway, spent its entire output budget thinking, and returned an empty answer with nothing left to say out loud. The proper fix sent that “skip thinking” instruction through the official API setting instead of as a text token in the prompt. Model pinning sidestepped the whole thing anyway, by routing those jobs to a smaller text model with no reasoning step at all.

Three silent choices, one for the tier, one for the machine, one for the model, produced three silent failures stacked on each other, and the 502 on top said nothing about any of them. Pinning each choice explicitly turns the quiet failures into loud ones.

Sidebar: proving the fix actually reached the model

I did not trust the “skip thinking” fix until I had watched it work, so I ran a test across every task type to confirm the instruction travelled all the way through the router, the backend, and the model’s own template. A representative sample:

task_type              expect  actual  finish   content  reasoning  notes
─────────────────────────────────────────────────────────────────────────────────
local_classifyextract  OFF     OFF     stop     908      0          no-think fires
local_writing          ON      ON      stop     912      394        thinking preserved
local_code             ON      ON      stop     1000     797        needs max_tokens >= 1024
cloud_classifyextract  N/A     n/a     stop     44       0          cloud passthrough clean

Zero tokens on hidden reasoning for the quick-labelling task types, scratchpad intact on the reasoning-on routes. One calibration gotcha worth writing down: local_code silently failed at a small output budget because thinking ate the first few hundred tokens before any code appeared. Any thinking-on route needs max_tokens >= 1024 if you expect a real answer back.

Picking a machine: least-loaded beat round-robin

The first version spread local work with round-robin: one request to the Mac Studio, the next to the 3090 Ti, then back again, strictly taking turns. It is the simplest thing that works, and it works badly the moment your machines are not identical. Taking turns hands the next request to a box that is already three deep in a queue while the other one sits idle.

The router now uses least-loaded selection. Each backend reports a load number, which is just the requests it is working on plus the requests waiting in its queue, and the next request goes to whichever eligible backend has the smallest number. Round-robin survives only as the tie-break. On a mixed workload, where a long reasoning job and a one-line classify request take wildly different amounts of time, the least-loaded queue drains while the round-robin queue piles up behind the slowest thing in it.

It took an afternoon. The table had already narrowed the candidates, so the only thing that changed was how one function chose among them.

The free tiers, lane by lane

The cloud tier costs nothing because the work that would justify a paid frontier model is either sensitive enough to stay on my own hardware or trivial enough that a free tier handles it.

• Groq. The quick sorting-and-labelling lane. Free quota, and fast: a classify request measured end to end through the router, including my own network hops, comes back in 180 to 220 milliseconds. When labelling and extraction dominate a job (one cleanup pipeline fires roughly 750 such requests per run), that is the difference between an 8-minute job and a 2-minute one.
• Cerebras. Carries the cloud reasoning lane on gpt-oss-120b, and stands behind Groq as the classify fallback. The free quota is generous enough that I have not hit the daily cap on real workloads.
• Codex. My ChatGPT login gives me the gpt-5.4 coding model with no metered API key. The catch: it launches as a separate process rather than a fast streaming connection, carrying a few seconds of startup overhead per request, so the quick jobs never go here. Using a ChatGPT login this way for automated pipelines also sits in a grey zone of OpenAI’s terms, which were written for interactive use. I respect the rate limits, do not industrialize it, and assume the policy could change.
• Gemini. Web-grounded research only, and the thinnest lane in the stack. The API key is configured at 5 requests per minute and 100 per day, enough for research questions and nothing like enough for bulk work.
• Mistral. Long-form cloud writing, after Gemini Pro’s quota went to zero on my keys.

What changed when I migrated everything over

Most callers were mechanical swaps. A few forced real decisions about where work belongs.

Public-data classifiers moved to the cloud. Pipelines that classify already-public content (press releases, saved social posts, scraped listings) had defaulted to the local Qwen out of habit. They switched to cloud_classifyextract, and response time dropped from about 800 milliseconds to about 200. A 500-item job that used to take 8 minutes now runs in under 2.

Sensitive reasoning moved to local. Some tools had been hitting the old cloud research route while reasoning over private material I had already fetched. I rewrote them to local_research. Slower, smaller model, no web search, but the data stays on my own network, which is the trade I wanted.

Image work got its own route. Vision tasks had been bolted onto a generic fallback. They got a proper local_image route with an explicit model, and callers now declare at the call site whether they are sending an image.

The agent tool stopped guessing. The tool my AI agents use to call the router used to fall back to keyword matching. Now it requires an explicit task type and shows the full vocabulary in its description. An agent reading the prompt knows whether it is sensitive far better than a keyword-matcher ever could.

Six weeks later: everything moved, and the table absorbed it

I wrote most of this post at the end of May. The stack underneath it has churned harder since then than in any six weeks before it, and every one of those changes landed as an edit to a lookup table.

Cerebras deleted a model out from under me. The classify lane ran on Cerebras llama-3.1-8b-instant, and Cerebras pruned that model from my account entirely. I found out while researching the free LLM API teardown, and would otherwise have found out as a wall of failures in a batch job. The repair was one line: Groq took over as primary for cloud_classifyextract, with Cerebras gpt-oss-120b behind it as the fallback. No caller changed.

Three Gemini accounts went unhealthy and quietly left. The router had six Gemini lanes, three API keys and three login-based accounts. The login-based ones have all gone unhealthy, and the 60-second health check pulled them out of rotation without my involvement. The remaining lanes absorbed the traffic.

Round-robin lost to least-loaded, as described above. The table already knew which backends were eligible, so the selection function was the only thing that had to learn anything new.

Classify and extract merged into one lane. They had always routed to the same model with the same settings, so cloud_classify and cloud_extract became cloud_classifyextract, and the local pair merged the same way. This is the only change in the six weeks that broke callers, and it broke them the right way: every stale call site returned HTTP 400 with a list of the valid labels, loudly, on the first request.

The daily driver got bigger and faster at the same time. When I built the router, the 3090 Ti ran a dense qwen3.6-27b at about 42 tokens per second (tokens per second, or tok/s, is simply how fast a model emits text). It now runs qwen3.6-35b-a3b, the mixture-of-experts model from the table above. I measure that at about 136 tok/s end to end over five runs today, with a tight spread. Excluding the roughly two seconds it takes to produce the first token, the decode rate is about 212 tok/s. A bigger model, three times faster, on the same card. The dense 27b did not disappear; it moved to the Mac Studio and now serves the code and synthesis lanes from there. Both models changed machines underneath the router, and the router did not care.

Routing changes stopped requiring a rebuild. The router now reads a runtime configuration overlay, so retargeting a lane is a file edit and a reload.

None of this was in the plan in May, and six weeks of churn is the honest argument for the router. I picked several of the models wrong. A provider deleted one out from under me, three accounts died, and my load-spreading strategy turned out to be the wrong one. The blast radius of all of it stayed inside one file I can read in twenty seconds.

What I would do differently

I should have caught the privacy boundary at the planning stage. The original spec covered routing strategy, authentication, streaming, and logging, and it never asked what happens when a private prompt gets classified into a cloud route. “What is the worst-case behaviour at the trust boundary?” is now a line on my planning checklist.

Without the request journal I would never have found the leak at all. When a router logs nothing, there is nothing to read on the afternoon you want to know where your prompts have actually been going.

I had also worried the privacy tier would feel slow. It does not. The Qwen MoE model on the 3090 Ti, at about 136 tok/s, feels comparable to the cloud on most routes, and the latency tax is 1 to 3 seconds on typical reasoning work rather than the 10-plus seconds I had budgeted for.

What is live today

• 16 task types, routed by prefix, validated on every request, so no task crosses the local/cloud boundary by accident.
• 14 backends, split between my own machines (the 3090 Ti, two Ministral sidecars on old Quadro cards, and the Mac Studio’s text, vision, and prompt-injection judge servers) and the cloud (Groq, Cerebras, Mistral, Codestral, Cohere, the Gemini API, and Codex). Each cloud key pools its own independent quota.
• Least-loaded backend selection, with round-robin only as the tie-break.
• Network-only access over Tailscale. No password layer; the private network is the security boundary.
• Streaming responses and full compatibility with the standard OpenAI API format, so existing tools talk to it without changes.
• Every request logged to a file (locked down to my user only), written when the response finishes.
• Continuous health checks every 60 seconds, all backends probed at once.
• Automatic retries on temporary failures, with a guard against being billed twice for the same request.
• A runtime configuration overlay, so a lane can be retargeted without a rebuild.
• An ops dashboard tab: per-backend health, quota bars, response-time percentiles.
• A replay tool that samples old logged requests and re-runs them against any model I name.
• Roughly 1,500 unit tests plus 70 live integration tests.
• A tool my AI agents call directly, plus a batch helper for agent workflows.

What is next

• Caching answers for the quick-labelling jobs, since those outputs are deterministic. Still not done.
• More dashboard panels for recent requests and task distribution. Partly built.
• A “what would this have cost on Anthropic?” column in the journal viewer. Still open.

The longer game is the journal plus the replay tool, which together turn this from a router into a continuous model-evaluation system. Every day of logged traffic is benchmark material derived from real work rather than synthetic tests. When the next frontier model drops, I will replay last week’s actual traffic against it and see how it does.

FAQ

What is an LLM router?

A service that sits between LLM consumers (apps, agents, scripts) and one or more backends (cloud APIs, local models) and decides where each request goes. It picks based on task type, cost, speed, remaining quota, model capability, and in this design a privacy level the caller declares. From the consumer’s side it is one OpenAI-compatible web address hiding the backend zoo behind it.

What is the difference between an LLM router and an LLM gateway?

A gateway handles transport: authentication, rate limiting, logging, retries. A router adds backend selection on top of that. Tools like LiteLLM, OpenRouter, and the NVIDIA Inference Blueprint blur the line. This project is firmly router-side: every request gets a task type and a privacy tier before any backend is picked.

Why not just use OpenRouter or LiteLLM?

OpenRouter is a hosted reseller of cloud models. LiteLLM is a local proxy that normalizes roughly 100 providers. Both are good tools. Neither routes local-versus-cloud as a per-request privacy decision, neither handles Codex through a ChatGPT login (it is not a normal API), and neither logs every request for replay benchmarking. I built my own to get those three behaviours.

Why prefix the task type instead of adding a separate privacy field?

A separate field can be forgotten, and any default it carries is how leaks happen. The prefix makes the trust tier visible at every call site, log line, error, and code review. local_classifyextract reads as “I want this private.” A privacy field two lines later reads as ceremony. The prefix costs a few characters at every call site, and paying that cost is what makes an accidental leak hard.

What happens when a private prompt accidentally gets routed to the cloud?

It cannot. Keyword auto-classification was deleted in the privacy rebuild. Every caller passes an explicit task type, and unknown values are rejected with an HTTP 400 before the prompt is even inspected. An earlier version did silently re-route private prompts to Gemini through keyword overlap; I caught it by reading the journal and rebuilt the API the next day.

Is task-based routing the same as semantic routing?

No. Semantic routing reads the prompt’s content with a small classifier model and picks a backend based on what the prompt looks like it is doing. Task-based routing makes the caller declare the work category up front. Semantic routing is more flexible at the cost of a runtime classifier and silent-misclassification bugs. Task-based routing has zero ambiguity at the boundary, which is exactly the trade the privacy tier needed.

The LLM router stack: tools and pricing

• Groq (groq.com | console) – Very fast inference for open-weight models, serving llama-3.1-8b-instant in about two-tenths of a second end to end; generous free daily quota, paid plans by token.
• Cerebras Inference (cerebras.ai | docs) – Fast inference for large open-weight models such as gpt-oss-120b; free tier with a daily quota, paid by token.
• Codex / gpt-5.4 (openai.com/codex | CLI docs) – OpenAI’s reasoning-tuned code model, reachable through a ChatGPT login in the Codex command-line tool; included in a ChatGPT Plus/Pro/Business subscription, no metered API.
• Gemini (ai.google.dev | studio) – Google’s multimodal model with a very large context window and native Google Search grounding; free tier, rate limited per key, paid by token above quota.
• Mistral (mistral.ai) – Handles the cloud long-form writing lane, plus Codestral as the code fallback; free tier, paid by token.
• Anthropic Claude (anthropic.com | docs) – Used outside the router for paid frontier work; token-based, with a weekly quota on Pro plans.
• LM Studio (lmstudio.ai) – Desktop app for running local models with an OpenAI-compatible interface; free for personal use.
• llama.cpp (github) – The open-source engine that runs the qwen3.6-35b-a3b model on the 3090 Ti; free, MIT licensed.
• Tailscale (tailscale.com) – A private network mesh (built on WireGuard) connecting my own machines into one secure tunnel, the boundary the local routes never cross; free for personal use, up to 100 devices.