# Local models in Kaitoi Code

> Set up Ollama, LM Studio or an OpenAI-compatible model server for Kaitoi Code, with no Kaitoi login or Kaitoi inference charge for local-model turns.

Source: https://kaitoi.io/docs/cli/local-models/
Section: Kaitoi CLI / Getting started
Written for: Anyone who wants Kaitoi Code to use a model they run themselves.
Prerequisites: Kaitoi CLI installed; A local model server, such as Ollama or LM Studio
Last reviewed: 2026-10-23

---

Kaitoi Code can run on a model you serve yourself instead of a hosted one.
Code talks to your model server directly from your computer: those turns never
go through Kaitoi and incur no Kaitoi inference charge. Your model server or
hosting provider can have its own costs. For the request-route diagram, model
selection and fallback behavior, see [Hosted and local models](/docs/cli/models/).

## Ollama and LM Studio

Code looks for both on their usual addresses when it starts:

- **Ollama** at `127.0.0.1:11434`, or wherever `OLLAMA_HOST` points. Its
  provider is `local:ollama`.
- **LM Studio**'s local server at `127.0.0.1:1234`. Its provider is
  `local:lmstudio`.

Start your model server, then pick a model:

```bash
kaitoi code --provider local:ollama --model qwen3.6:27b
```

In a session, `/providers` lists the local models it found under
**On this machine (not billed)**, and `/provider local:ollama` and
`/model <name>` switch to one.

A model must be able to call tools and needs at least 16,384 tokens of
context. `/providers` marks a model that is too small. Code uses a window of
32,768 tokens, or the model's own window when that is smaller; set
`contextTokens` for more.

## No account needed

You can use Code with a local model without a Kaitoi account. When you are not
signed in and Code finds a local model, the sign-in card offers to use that
model without an account, or you can start Code with `--provider` set to a
local provider.

Without an account, Code still reads and edits files in your trusted folder
and runs the commands you approve. Kaitoi's own tools, such as projects and
graphs, web search, media and Open in Studio, and the hosted models need a
login. Run `/login` in the session, and they are added from your next message.

Web reads and media runs you make while signed in are still charged, even when
the model itself is local; the turn line shows "model not billed" and the
credits for the rest.

## vLLM and other servers

Any server that speaks OpenAI's chat completions API, such as vLLM or
llama.cpp, is added in the `localModels` section of `agent.json`:

```json
{
	"localModels": {
		"endpoints": [
			{
				"id": "vllm",
				"kind": "openai",
				"baseUrl": "http://10.0.0.5:8000/v1",
				"apiKeyEnv": "VLLM_API_KEY",
				"models": ["Qwen/Qwen3-32B"],
				"contextTokens": 65536
			}
		]
	}
}
```

Its provider is then `local:vllm`, after its `id`.

| Key             | What it sets                                                                                                      |
| --------------- | ----------------------------------------------------------------------------------------------------------------- |
| `id`            | The name after `local:`. Lower-case letters, digits, `-` and `_`.                                                 |
| `kind`          | `openai` for an OpenAI-compatible server, or `ollama` for Ollama's own API.                                       |
| `baseUrl`       | The server's address.                                                                                             |
| `apiKeyEnv`     | The environment variable that holds the server's API key, if it needs one. The key itself never goes in the file. |
| `models`        | The model names to offer, for a server that cannot list them.                                                     |
| `contextTokens` | The model's context window, from 4,096 to 1,048,576 tokens.                                                       |

At the top of `localModels`, `"autoDetect": false` stops Code looking for
Ollama and LM Studio, and `contextTokens` sets the window for every endpoint
that does not set its own. Code never looks for other servers by itself: an
OpenAI-compatible server is only asked once you add it.

## Subagents on local models

A subagent role can run on a local model too, with `engine` set to `local` in
the `agents` section of `agent.json`, or with
`/agents engine explore local local:ollama`. See
[Claude Code and Codex engines](/docs/cli/engines/) for the other engines.

## Current limitations

- Reasoning effort settings do not apply to local models.
- How well Code works depends on the model: a small model can miss tool calls
  or stop early.