Local models in Kaitoi Code.

Kaitoi Code can run on a model you serve yourself instead of a hosted one. Code talks to your model server directly from your computer: those turns never go through Kaitoi and incur no Kaitoi inference charge. Your model server or hosting provider can have its own costs. For the request-route diagram, model selection and fallback behavior, see Hosted and local models.

#Ollama and LM Studio

Code looks for both on their usual addresses when it starts:

• Ollama at 127.0.0.1:11434, or wherever OLLAMA_HOST points. Its provider is local:ollama.
• LM Studio's local server at 127.0.0.1:1234. Its provider is local:lmstudio.

Start your model server, then pick a model:

kaitoi code --provider local:ollama --model qwen3.6:27b

In a session, /providers lists the local models it found under On this machine (not billed), and /provider local:ollama and /model <name> switch to one.

A model must be able to call tools and needs at least 16,384 tokens of context. /providers marks a model that is too small. Code uses a window of 32,768 tokens, or the model's own window when that is smaller; set contextTokens for more.

#No account needed

You can use Code with a local model without a Kaitoi account. When you are not signed in and Code finds a local model, the sign-in card offers to use that model without an account, or you can start Code with --provider set to a local provider.

Without an account, Code still reads and edits files in your trusted folder and runs the commands you approve. Kaitoi's own tools, such as projects and graphs, web search, media and Open in Studio, and the hosted models need a login. Run /login in the session, and they are added from your next message.

Web reads and media runs you make while signed in are still charged, even when the model itself is local; the turn line shows "model not billed" and the credits for the rest.

#vLLM and other servers

Any server that speaks OpenAI's chat completions API, such as vLLM or llama.cpp, is added in the localModels section of agent.json:

{
	"localModels": {
		"endpoints": [
			{
				"id": "vllm",
				"kind": "openai",
				"baseUrl": "http://10.0.0.5:8000/v1",
				"apiKeyEnv": "VLLM_API_KEY",
				"models": ["Qwen/Qwen3-32B"],
				"contextTokens": 65536
			}
		]
	}
}

Its provider is then local:vllm, after its id.

Key	What it sets
id	The name after local:. Lower-case letters, digits, - and _.
kind	openai for an OpenAI-compatible server, or ollama for Ollama's own API.
baseUrl	The server's address.
apiKeyEnv	The environment variable that holds the server's API key, if it needs one. The key itself never goes in the file.
models	The model names to offer, for a server that cannot list them.
contextTokens	The model's context window, from 4,096 to 1,048,576 tokens.

At the top of localModels, "autoDetect": false stops Code looking for Ollama and LM Studio, and contextTokens sets the window for every endpoint that does not set its own. Code never looks for other servers by itself: an OpenAI-compatible server is only asked once you add it.

#Subagents on local models

A subagent role can run on a local model too, with engine set to local in the agents section of agent.json, or with /agents engine explore local local:ollama. See Claude Code and Codex engines for the other engines.

#Current limitations

• Reasoning effort settings do not apply to local models.
• How well Code works depends on the model: a small model can miss tool calls or stop early.