Hosted and local models.

Kaitoi Code can use a hosted model through your Kaitoi account or talk directly to a model server you configure. The choice changes where the conversation is sent and how model usage is charged. Your workspace, mode, and tool permissions still determine what Code can do.

This guide covers CLI 0.9.0 and its matching backend. See Install and update for version and download availability. Commands beginning with kaitoi run in your terminal; slash commands run at Code's You > prompt.

#Follow the model request

Hosted requests pass through Kaitoi to the selected provider. Local-model requests go directly from Code to your configured endpoint. A local: provider can point to this computer, a server on your network, or a remote host; the address determines where the data goes.

The hosted model route is Kaitoi Code, then Kaitoi, then the hosted model. It requires a Kaitoi login and model usage is billed through Kaitoi. The local model route is Kaitoi Code directly to your configured model server, without passing the model request through Kaitoi. A provider such as local:ollama identifies this direct route. The server can be on this computer, your network, or a remote host. No Kaitoi login is needed for that conversation, and Kaitoi charges zero credits for its local-model inference; your server or hosting provider may have its own costs. Both routes can use tools, subject to mode, workspace, and permission checks. Tool calls may access workspace files, Kaitoi services, or external services separately. A local model therefore does not make every tool local, offline, or free. This diagram is a conceptual route map, not a terminal screenshot, and sends no requests.

One Code conversation fans out into the hosted or direct model path. A separate tool branch retains its own destinations and authority whichever model route is selected.

Choosing a local model does not grant it unrestricted access to your computer. File reads, edits, and command execution still follow Code's permission rules. Conversely, information supplied in your prompt or returned by an allowed file read can become part of the conversation sent to the selected model server.

#Find and choose a model

Inside Code, list the providers and models available to this session:

/providers

The listing marks the current choice and separates local servers from hosted providers. For hosted models, it can show input, output, and cache-read rates in credits per million tokens, plus a context window. Missing pricing or context information means the catalog did not supply it; it does not mean zero cost or unlimited context.

Use IDs from that list:

/provider <provider-id>
/model <model-id>
/status

Choose the provider first, then a model from that provider. /provider selects its default model; /model selects a specific model within the current provider. /status confirms the active provider and model. Model lists can change, so copy an ID from your own listing instead of relying on a model name from an old example.

When a writable user config is loaded, these choices can also become your defaults for future sessions. Read the save message. Use /config init to create the optional defaults file first; see Sign-in and configuration.

#Hosted models and account defaults

Hosted models require a Kaitoi login. Use /login inside Code, or kaitoi connect login in your terminal, then inspect /providers.

The hosted list comes from your connected Kaitoi environment. OpenRouter uses your saved model list when you have one. If you have never saved a list, the administrator's OpenRouter defaults are used. A saved empty list remains empty; it is not replaced by those defaults. This behavior requires the matching backend version as well as the CLI.

Provider IDs matter: ollama is the hosted Ollama Cloud provider; local:ollama is your directly connected Ollama server. They are separate routes with different authentication and billing.

#Choose for one launch

Startup flags override saved defaults without rewriting them:

kaitoi code --provider <provider-id> --model <model-id>

Replace both placeholders with IDs from /providers. To follow a provider's default, omit --model; Code does not carry over a saved model when you explicitly choose a provider at startup.

If an unavailable choice came from your saved defaults, Code may start with a fallback and print a notice. In a signed-in session, that can mean switching from a saved local model to a hosted model. Check the startup notice and /status before sending your first message. Use an explicit local provider flag when the conversation must use that route: a failed explicit local selection reports an error instead of taking this saved-default fallback.

#Start with Ollama or LM Studio

First install and start your model server, and make a suitable model available through it. Code discovers models; it does not download their weights for you. A model must be able to call tools and have enough context for an agent session.

With auto-detection enabled, Code checks these addresses:

Provider ID	Default address	Server interface
local:ollama	http://127.0.0.1:11434	Ollama's native API
local:lmstudio	http://127.0.0.1:1234/v1	OpenAI-compatible Chat Completions

For Ollama, OLLAMA_HOST can change the automatically detected address. Other ports and servers are not scanned; configure those explicitly.

Start with one of these commands, depending on which server you have ready:

kaitoi code --provider local:ollama

kaitoi code --provider local:lmstudio

You do not need to sign into Kaitoi for the local-model conversation. Use /providers to refresh local discovery, /model <model-id> to select an available model, and /status to confirm the choice. A stopped auto-detected server may be omitted from the listing unless it is your current provider.

The available tools depend on sign-in too. Signed-out sessions can use the local workspace tools and configured extensions that do not need Kaitoi. Kaitoi account tools and Crew require sign-in. You can use /login later; it keeps the selected local model and enables account tools from the next message.

#Connect another model server

Put local endpoints in the user-level agent.json, beside Code's user config.json. On macOS and Linux, this is normally ~/.config/kaitoi-code/agent.json (or under $XDG_CONFIG_HOME when set). On Windows it is %APPDATA%\Kaitoi Code\agent.json. See configuration file locations for the full settings distinction.

Merge the following localModels section into that JSON object, preserving any other settings. This example connects an OpenAI-compatible server that you have already started on port 8000:

{
	"localModels": {
		"autoDetect": true,
		"contextTokens": 32768,
		"endpoints": [
			{
				"id": "workstation",
				"kind": "openai",
				"baseUrl": "http://127.0.0.1:8000/v1"
			}
		]
	}
}

Restart Code after editing the file, then select the new provider:

kaitoi code --provider local:workstation

The endpoint ID becomes local:<id>. Use a short lowercase ID containing letters, digits, hyphens, or underscores, starting with a letter or digit. Choose kind: "ollama" for the native Ollama API or kind: "openai" for an OpenAI-compatible Chat Completions API, such as one served by LM Studio, vLLM, or llama.cpp. Here, “OpenAI-compatible” describes the server's API format; it does not sign you into an OpenAI account.

Use the server's actual base URL, including /v1 where required. URLs must use HTTP or HTTPS and must not contain credentials, query strings, or fragments. A non-loopback address sends the conversation to that host; plain HTTP does not encrypt it.

#Optional endpoint settings

Setting	Use it for
apiKeyEnv	Name an environment variable holding the server's bearer token, such as MODEL_SERVER_KEY. Set it in the environment that launches Code. Store only its name in this file.
models	For kind: "openai", supply exact model IDs as a JSON array when discovery is unavailable or you want a specific list. The server must still be reachable, accept authentication, and serve those models.
contextTokens	Override the shared context budget for this endpoint. Match the window your server actually supports.

At the localModels level, autoDetect: false disables the implicit Ollama and LM Studio endpoints; explicitly configured endpoints remain available. An explicit endpoint with the ID ollama or lmstudio replaces the implicit one, so you can change its address or settings without creating a second provider.

#Match the context window and tool support

Local models need tool calling and at least 16,384 tokens of context. Code refuses a model when the server reports that it cannot call tools or its context is below that minimum. Incomplete server metadata is not proof of compatibility: a model may appear in the list and still fail on its first request.

The default local context budget is 32,768 tokens. Code limits its budget to the configured window and the server's reported maximum, while keeping its minimum agent window. For an OpenAI-compatible server that does not report a window, /providers labels the context as assumed. Ollama may show the configured size without that label when its metadata is unavailable. Configure at least 16,384 and ensure the server really runs the model with that much context.

For native Ollama, Code sends the selected context size in the request. For OpenAI-compatible servers, contextTokens controls Code's client-side budget; it does not increase the server's loaded context. Set that window in the server too. Image input also depends on the model and the capabilities the server exposes; a text model does not gain vision from being local.

The /effort preference applies to supported hosted models. Local models use their server's defaults; changing /effort does not tune a local server's reasoning settings.

#Separate model cost from tool cost

Kaitoi charges zero credits for direct local-model inference. Hardware, electricity, and any charges from your own server or hosting provider are separate. Hosted inference is billed through Kaitoi.

Tools have their own destinations. A local-model session that signs into Kaitoi can still use chargeable account tools, graph runs, or media generation. Your configured MCP servers, hooks, and commands can also contact other services. The model selection alone therefore does not make the entire session offline or free. See Costs and budgets for session spending controls.

#When a model is missing or a request fails

What you see	What to check
Local server absent or not answering	Start the server, check its address, and run /providers again. Configure nonstandard ports in agent.json.
No models listed	Make a model available in the server. For OpenAI-compatible endpoints, models can supply exact IDs when the listing API is unavailable.
Cannot call tools	Choose a tool-capable model and enable tool support on the server. Plain text that looks like a tool call is not a working tool call.
Context too small or context-length error	Increase the actual server window to at least 16,384 tokens and align contextTokens. For a long conversation, /clear starts fresh context.
Authentication rejected by your server	Check that apiKeyEnv names an environment variable available to the Code process and that its value is accepted by that server.
Hosted provider unavailable	Check /status and /login. Review your enabled model list, especially an intentionally empty OpenRouter list.
Catalog could not be verified	Read the warning: hosted selection may be resolved by the backend. An unverified choice is not saved as a new default.

After a startup fallback or a deliberate model change, check /status again before continuing with sensitive material or chargeable work.