# Hosted and local models

> Choose a hosted model or connect your own model server in Kaitoi Code. Learn model selection, local setup, context limits, and where requests go.

Source: https://kaitoi.io/docs/cli/models/
Section: Kaitoi CLI / Getting started
Written for: Anyone choosing a model for Kaitoi Code.
Prerequisites: Kaitoi CLI installed; A Kaitoi account for hosted models, or a running local model server
Last reviewed: 2026-10-24

---

Kaitoi Code can use a hosted model through your Kaitoi account or talk directly
to a model server you configure. The choice changes where the conversation is
sent and how model usage is charged. Your workspace, mode, and tool permissions
still determine what Code can do.

This guide covers CLI **0.9.0** and its matching backend. See
[Install and update](/docs/cli/install/) for version and download availability.
Commands beginning with `kaitoi` run in your terminal; slash commands run at
Code's `You >` prompt.

## Follow the model request

Hosted requests pass through Kaitoi to the selected provider. Local-model
requests go directly from Code to your configured endpoint. A `local:` provider
can point to this computer, a server on your network, or a remote host; the
address determines where the data goes.


The hosted model route is Kaitoi Code, then Kaitoi, then the hosted model.
It requires a Kaitoi login and model usage is billed through Kaitoi. The local
model route is Kaitoi Code directly to your configured model server, without
passing the model request through Kaitoi. A provider such as local:ollama
identifies this direct route. The server can be on this computer, your network,
or a remote host. No Kaitoi login is needed for that conversation, and Kaitoi
charges zero credits for its local-model inference; your server or hosting
provider may have its own costs. Both routes can use tools, subject to mode,
workspace, and permission checks. Tool calls may access workspace files, Kaitoi
services, or external services separately. A local model therefore does not
make every tool local, offline, or free. This diagram is a conceptual route
map, not a terminal screenshot, and sends no requests.

One Code conversation fans out into the hosted or direct model path. A separate tool branch retains its own destinations and authority whichever model route is selected.

*Figure: Model-request routes. Tool calls have their own destinations and costs.*

Choosing a local model does not grant it unrestricted access to your computer.
File reads, edits, and command execution still follow Code's permission rules.
Conversely, information supplied in your prompt or returned by an allowed file
read can become part of the conversation sent to the selected model server.

## Find and choose a model

Inside Code, list the providers and models available to this session:

```text
/providers
```

The listing marks the current choice and separates local servers from hosted
providers. For hosted models, it can show input, output, and cache-read rates
in **credits per million tokens**, plus a context window. Missing pricing or
context information means the catalog did not supply it; it does not mean
zero cost or unlimited context.

Use IDs from that list:

```text
/provider <provider-id>
/model <model-id>
/status
```

Choose the provider first, then a model from that provider. `/provider` selects
its default model; `/model` selects a specific model within the current provider.
`/status` confirms the active provider and model. Model lists can change, so
copy an ID from your own listing instead of relying on a model name from an
old example.

When a writable user config is loaded, these choices can also become your
defaults for future sessions. Read the save message. Use `/config init` to
create the optional defaults file first; see
[Sign-in and configuration](/docs/cli/configuration/).

### Hosted models and account defaults

Hosted models require a Kaitoi login. Use `/login` inside Code, or
`kaitoi connect login` in your terminal, then inspect `/providers`.

The hosted list comes from your connected Kaitoi environment. OpenRouter uses
your saved model list when you have one. If you have never saved a list, the
administrator's OpenRouter defaults are used. A saved empty list remains empty;
it is not replaced by those defaults. This behavior requires the matching
backend version as well as the CLI.

Provider IDs matter: **`ollama` is the hosted Ollama Cloud provider**;
**`local:ollama` is your directly connected Ollama server**. They are separate
routes with different authentication and billing.

### Choose for one launch

Startup flags override saved defaults without rewriting them:

```bash
kaitoi code --provider <provider-id> --model <model-id>
```

Replace both placeholders with IDs from `/providers`. To follow a provider's
default, omit `--model`; Code does not carry over a saved model when you explicitly
choose a provider at startup.

If an unavailable choice came from your **saved defaults**, Code may start with
a fallback and print a notice. In a signed-in session, that can mean switching
from a saved local model to a hosted model. Check the startup notice and
`/status` before sending your first message. Use an explicit local provider
flag when the conversation must use that route: a failed explicit local
selection reports an error instead of taking this saved-default fallback.

## Start with Ollama or LM Studio

First install and start your model server, and make a suitable model available
through it. Code discovers models; it does not download their weights for you.
A model must be able to call tools and have enough context for an agent session.

With auto-detection enabled, Code checks these addresses:

| Provider ID      | Default address            | Server interface                   |
| ---------------- | -------------------------- | ---------------------------------- |
| `local:ollama`   | `http://127.0.0.1:11434`   | Ollama's native API                |
| `local:lmstudio` | `http://127.0.0.1:1234/v1` | OpenAI-compatible Chat Completions |

For Ollama, `OLLAMA_HOST` can change the automatically detected address.
Other ports and servers are not scanned; configure those explicitly.

Start with one of these commands, depending on which server you have ready:

```bash
kaitoi code --provider local:ollama
```

```bash
kaitoi code --provider local:lmstudio
```

You do not need to sign into Kaitoi for the local-model conversation. Use
`/providers` to refresh local discovery, `/model <model-id>` to select an
available model, and `/status` to confirm the choice. A stopped auto-detected
server may be omitted from the listing unless it is your current provider.

The available tools depend on sign-in too. Signed-out sessions can use the
local workspace tools and configured extensions that do not need Kaitoi.
Kaitoi account tools and Crew require sign-in. You can use `/login` later;
it keeps the selected local model and enables account tools from the next message.

## Connect another model server

Put local endpoints in the **user-level `agent.json`**, beside Code's user
`config.json`. On macOS and Linux, this is normally
`~/.config/kaitoi-code/agent.json` (or under `$XDG_CONFIG_HOME` when set).
On Windows it is `%APPDATA%\Kaitoi Code\agent.json`.
See [configuration file locations](/docs/cli/configuration/#find-the-right-settings-file)
for the full settings distinction.

Merge the following `localModels` section into that JSON object, preserving
any other settings. This example connects an OpenAI-compatible server that
you have already started on port 8000:

```json
{
	"localModels": {
		"autoDetect": true,
		"contextTokens": 32768,
		"endpoints": [
			{
				"id": "workstation",
				"kind": "openai",
				"baseUrl": "http://127.0.0.1:8000/v1"
			}
		]
	}
}
```

Restart Code after editing the file, then select the new provider:

```bash
kaitoi code --provider local:workstation
```

The endpoint ID becomes `local:<id>`. Use a short lowercase ID containing
letters, digits, hyphens, or underscores, starting with a letter or digit.
Choose `kind: "ollama"` for the native Ollama API or `kind: "openai"` for an
OpenAI-compatible Chat Completions API, such as one served by LM Studio,
vLLM, or llama.cpp. Here, “OpenAI-compatible” describes the server's API
format; it does not sign you into an OpenAI account.

Use the server's actual base URL, including `/v1` where required. URLs must
use HTTP or HTTPS and must not contain credentials, query strings, or fragments.
A non-loopback address sends the conversation to that host; plain HTTP does
not encrypt it.

### Optional endpoint settings

| Setting         | Use it for                                                                                                                                                                                                 |
| --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `apiKeyEnv`     | Name an environment variable holding the server's bearer token, such as `MODEL_SERVER_KEY`. Set it in the environment that launches Code. Store only its name in this file.                                |
| `models`        | For `kind: "openai"`, supply exact model IDs as a JSON array when discovery is unavailable or you want a specific list. The server must still be reachable, accept authentication, and serve those models. |
| `contextTokens` | Override the shared context budget for this endpoint. Match the window your server actually supports.                                                                                                      |

At the `localModels` level, `autoDetect: false` disables the implicit Ollama
and LM Studio endpoints; explicitly configured endpoints remain available.
An explicit endpoint with the ID `ollama` or `lmstudio` replaces the implicit
one, so you can change its address or settings without creating a second provider.

## Match the context window and tool support

Local models need **tool calling** and at least **16,384 tokens of context**.
Code refuses a model when the server reports that it cannot call tools or its
context is below that minimum. Incomplete server metadata is not proof of
compatibility: a model may appear in the list and still fail on its first request.

The default local context budget is **32,768 tokens**. Code limits its budget
to the configured window and the server's reported maximum, while keeping
its minimum agent window. For an OpenAI-compatible server that does not report
a window, `/providers` labels the context as assumed. Ollama may show the
configured size without that label when its metadata is unavailable. Configure
at least 16,384 and ensure the server really runs the model with that much context.

For native Ollama, Code sends the selected context size in the request.
For OpenAI-compatible servers, `contextTokens` controls Code's client-side
budget; it does **not** increase the server's loaded context. Set that window
in the server too. Image input also depends on the model and the capabilities
the server exposes; a text model does not gain vision from being local.

The `/effort` preference applies to supported hosted models. Local models use
their server's defaults; changing `/effort` does not tune a local server's
reasoning settings.

## Separate model cost from tool cost

Kaitoi charges **zero credits for direct local-model inference**. Hardware,
electricity, and any charges from your own server or hosting provider are
separate. Hosted inference is billed through Kaitoi.

Tools have their own destinations. A local-model session that signs into
Kaitoi can still use chargeable account tools, graph runs, or media generation.
Your configured MCP servers, hooks, and commands can also contact other services.
The model selection alone therefore does not make the entire session offline
or free. See [Costs and budgets](/docs/cli/prices/) for session spending controls.

## When a model is missing or a request fails

| What you see                              | What to check                                                                                                                                  |
| ----------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| Local server absent or not answering      | Start the server, check its address, and run `/providers` again. Configure nonstandard ports in `agent.json`.                                  |
| No models listed                          | Make a model available in the server. For OpenAI-compatible endpoints, `models` can supply exact IDs when the listing API is unavailable.      |
| Cannot call tools                         | Choose a tool-capable model and enable tool support on the server. Plain text that looks like a tool call is not a working tool call.          |
| Context too small or context-length error | Increase the actual server window to at least 16,384 tokens and align `contextTokens`. For a long conversation, `/clear` starts fresh context. |
| Authentication rejected by your server    | Check that `apiKeyEnv` names an environment variable available to the Code process and that its value is accepted by that server.              |
| Hosted provider unavailable               | Check `/status` and `/login`. Review your enabled model list, especially an intentionally empty OpenRouter list.                               |
| Catalog could not be verified             | Read the warning: hosted selection may be resolved by the backend. An unverified choice is not saved as a new default.                         |

After a startup fallback or a deliberate model change, check `/status` again
before continuing with sensitive material or chargeable work.