AI Provider & Model Management

Atamaia's AI routing layer is a multi-provider, multi-model orchestration system. It is not a thin proxy. Agents, chat sessions, and councils ask “give me a model for this role” and get a working connection — or a loud refusal — whether the answer is Claude via OpenRouter, a local llama.cpp model, or a floating always-latest alias.

A registered model is not automatically a usable one. Conflating the flags below caused silent failures. They are separate on purpose.

Canonical host: https://api.atamaia.ai. All endpoints require authentication. Responses wrapped in ApiEnvelope<T>.

The four fields, and who owns each

Field	Owner	Question it answers
`enabled`	Human	Should this model be routable at all?
`available`	Sync	Is the provider actually serving it right now?
`approvedForAgent`	Human	May autonomous agent runs use it?
`approvedForChat`	Human	May interactive chat use it?

Dispatch requires enabled AND available. Agent role resolution additionally requires approvedForAgent. The locked default must be all three, plus an enabled provider.

available defaults to true, including for providers that never push a roster (OpenRouter, Anthropic, OpenAI). Absence of a roster is not an assertion of absence.

enabled is never written by roster sync, in either direction. A returning model becomes routable again on its own. A human disable survives the next regeneration.

Live example: local:preview-27b is enabled=false, available=true. Resolve:

GET /api/ai/resolve/local:preview-27b

→ HTTP 400, errorCode: INVALID_OPERATION:

Model 'local:preview-27b' is not routable: it is disabled in Atamaia (a deliberate, human-owned setting — re-enable it to route here).

Previously both “switched off” and “not loaded” produced the same upstream 400, and a disabled model still resolved.

Architecture

Request: ChatRequest { modelId: "local:main-instruct", message: "..." }
    │
    ▼
AIRouterService
    ├── Resolve model (by modelId or role route)
    ├── RejectIfNotRoutable (enabled ∧ available)
    ├── Find provider (priority-ordered, health-checked)
    ├── Get credentials (tenant-specific or provider-level)
    ├── Build HTTP request (OpenAI-compat format)
    ├── Stream or synchronous call
    ├── Failover on retriable errors (429, 5xx, timeouts)
    ├── Record success/failure for circuit breaker
    └── Return ChatResponse with usage + cost

Fully qualified model ID is {provider.Prefix}:{modelId} — e.g. local:main-instruct, openrouter:~vendor/family-latest. Live OpenRouter prefix is openrouter, not or.

Providers

A provider is an LLM endpoint — cloud API, local inference server, or meta-router.

Provider types (`ProviderType`)

Type	Description
`OpenRouter`	OpenRouter meta-router
`LocalLlamaCpp`	Direct llama.cpp server
`Anthropic`	Anthropic API
`OpenAI`	OpenAI API (also used as the compat type for Gemini, Groq, …)
`Custom`	Any OpenAI-compatible endpoint
`AnthropicAgentSdk`	Claude Agent SDK (subprocess; no API key)
`Vllm`	vLLM
`LiteLlm`	LiteLLM

API

GET    /api/ai/providers
GET    /api/ai/providers/{id}
POST   /api/ai/providers
PATCH  /api/ai/providers/{id}
DELETE /api/ai/providers/{id}
GET    /api/ai/providers/{id}/catalog
POST   /api/ai/providers/{id}/models/sync
POST   /api/ai/providers/{id}/sync-key

Create body: name, type, prefix, baseUrl, apiKey, defaultModel, priority, timeoutSeconds, stripPrefixInRequests, configJson.

List DTO includes enabled, priority, timeoutSeconds, modelCount, isGlobal, hasCredential (never the raw key).

Live this tenant: 21 providers (local the primary inference host / the secondary inference host / a compact local model family, OpenRouter, Anthropic, a set of global catalog rows, …). Do not treat that count as a contract.

Models

GET    /api/ai/models
GET    /api/ai/models/{id}
POST   /api/ai/models
PATCH  /api/ai/models/{id}
DELETE /api/ai/models/{id}

What the DTO carries

Field	Notes
`enabled` / `available`	See above. PATCH can set `enabled`. Sync sets `available`.
`approvedForAgent` / `approvedForChat`	Human trust flags
`isLockedDefault`	At most one per tenant. PATCH can set it (clears the previous).
`contextLength` / `maxCompletionTokens`	From GGUF / catalogue
`isEmbedding` / `specDecode`	Capability (`mtp`, `eagle3`, or null)
`temperature`, `maxTokens`, `topP`, `topK`, `minP`, `repeatPenalty`, `frequencyPenalty`, `presencePenalty`	Sampler defaults. Null means “upstream did not say”, not zero.
`isLocal` / `localModelPath`	Local weights
`enableHydration` / `hydrationIdentityId`	Auto-hydrate before chat
`inputCostPer1M` / `outputCostPer1M`	USD

Create does not take enabled / available / sampler extras / isLockedDefault. Those are PATCH (human) or sync (roster). Sync must not send approvedForAgent / approvedForChat — approval is an Atamaia-side trust decision, not a fact about the file.

Live roster this tenant: 29 models. Snapshot, not a contract. Examples: the primary instruct model (local:main-instruct, enabled+available+both approvals); a disabled preview model as above; locked default as below.

Route configuration

Routes map a role (builder, scribe, tester, planner, researcher, reviewer, designer, orchestrator, plus live extras chat, coding, council, editor) to a preferred model.

GET    /api/ai/routes
POST   /api/ai/routes
PATCH  /api/ai/routes/{id}
DELETE /api/ai/routes/{id}

PATCH body: providerId, modelId, clearModel, role, priority, notes.

A route may legitimately have no model, meaning “use the provider’s default.” That is different from a route whose chosen model was deleted. Those two states used to look identical (isBroken did not exist); three roles silently resolved to an external paid default for five weeks (#476).

Now:

isBroken + brokenReason are set when a model is updated, deleted, or changes availability — and cleared when it recovers.
Resolution refuses a broken route. It does not quietly substitute.
Repointing is a human decision (PATCH exists as of this probe).

Live (2026-08-14): 12 routes, all isBroken=false. Builder / scribe / tester were added the same day after falling through to a previously-broken coding route.

Doc 95 listed “no PATCH/DELETE for route configs (#477)” as a known gap. The live API has both. Prefer the probe.

The locked default

One model per tenant may be isLockedDefault: the last-resort fallback when a role’s own model is unusable.

Refused deletion and disabling. The net that catches every broken role must not itself vanish.
Setting it on a new model clears it from the previous one.
It must be enabled, available, and approvedForAgent (and its provider enabled) to serve. If configured but unusable, that is logged at Error.

Live: openrouter:~vendor/family-latest (id 111). Point it at a floating always-latest alias so the safety net follows versions.

Model resolution

GET /api/ai/resolve/{modelId}

Exact match on modelId or prefix:modelId
Provider prefix split on :
For a role: highest-priority non-broken route whose model is routable
Locked default (if usable)
Otherwise: refuse. No silent paid-provider substitute.

Agent runs: explicit run modelId → role route → locked default.

Chat and broadcast

POST /api/ai/chat

{
  "modelId": "local:main-instruct",
  "message": "Explain Hebbian learning in 3 sentences",
  "systemPrompt": "You are a neuroscience expert.",
  "temperature": 0.7,
  "maxTokens": 500,
  "tools": [],
  "toolChoice": "auto"
}

Response: success, modelId, reply, usage, responseTimeMs, toolCalls, isRetriable.

POST /api/ai/broadcast

Same message to multiple models. Returns per-model timing and usage.

Streaming

Open Responses is the canonical stream. Upstream providers are normalized into it (response.created, output_text.delta, function_call.arguments.delta, response.completed, …). Clients do not need to know whether the model is on OpenAI, Anthropic, OpenRouter, or llama.cpp.

Provider health and circuit breaker

ProviderHealthTracker (singleton):

State	Behaviour
Healthy	Fewer than 3 consecutive failures
Open	3+ failures. Provider skipped for 5 minutes.
Recovery	After 5 minutes the circuit closes; the next request tests it.

Retriable: HTTP 408 / 429 / 5xx, TaskCanceledException, HttpRequestException.

GetHealthyFallbackModelsAsync filters on enabled + available + provider.enabled + approvedForAgent + circuit health. Failover candidates match what the roster is actually serving.

Roster push (self-hosted boxes)

Local inference hosts push their model list. Atamaia does not poll them. A box behind NAT needs no inbound reachability.

POST /api/ai/providers/{providerId}/models/sync
Header: X-Provider-Sync-Key: <key>

Auth: the sync-key header or a bearer token with AIModelManage. Issue a key with POST /api/ai/providers/{id}/sync-key.

Body:

Field	Default	Meaning
`models`	—	List of `ModelSyncItem`
`additive`	`false`	`false` = declarative full replace
`dryRun`	`false`	Plan only (`added` / `updated` / `deactivated` / `restored` / `unchanged`)

additive: false + { "models": [] } marks every model for that provider available=false. That is not a no-op. Probe with dryRun: true.

The publisher should send modelId, contextLength, localModelPath, isLocal, capabilities, sampler settings. It should not send approval flags.

Model groups

Named rosters so a caller does not re-paste a panel.

GET    /api/ai/model-groups
POST   /api/ai/model-groups
GET    /api/ai/model-groups/{id}
GET    /api/ai/model-groups/by-name/{name}
PATCH  /api/ai/model-groups/{id}
DELETE /api/ai/model-groups/{id}
POST   /api/ai/model-groups/{id}/members
DELETE /api/ai/model-groups/{id}/members/{modelId}

Live: security-panel (4 member models). Snapshot, not a contract.

Credentials, catalog, pricing

GET    /api/ai/credentials
POST   /api/ai/credentials
DELETE /api/ai/credentials/{id}
POST   /api/ai/credentials/{id}/validate
GET    /api/ai/catalog
GET    /api/ai/catalog/{id}
POST   /api/ai/sync-pricing

Global providers (isGlobal=true) are the shared catalog. Tenants supply their own keys. Keys are encrypted at rest; the raw value is never returned. Validate hits the provider’s /models endpoint.

POST /api/ai/sync-pricing pulls OpenRouter per-token prices onto models under OpenRouter providers.

Cost per call:

cost = (promptTokens * inputCostPer1M / 1_000_000) + (completionTokens * outputCostPer1M / 1_000_000)

Agent runs roll this up across the tree. Local models typically have zero cost.

Local models

The primary inference host and the secondary inference host (roster push) are LocalLlamaCpp providers. The host:port list in the 2026-03 page is stale. Read the live providers:

GET /api/ai/providers
GET /api/ai/models?providerId={id}

Verify a sync by reading the model list back against the box’s /v1/models. A sync that reports success while the roster still disagrees is the failure mode this subsystem exists to end.

Audit

Model updates, deletions, and roster syncs write audit_events (ai.model.updated, ai.model.deleted, ai.model.disable, ai.provider.models.synced), including refused attempts (succeeded = false). A no-op sync is specified to write nothing (not re-probed).

API catalog

Method	Endpoint	Permission	Description
POST	`/api/ai/chat`	ChatSessionCreate	Send chat message
POST	`/api/ai/broadcast`	ChatSessionCreate	Broadcast to multiple models
GET/POST/PATCH/DELETE	`/api/ai/providers`…	AIProviderView / Manage	Providers
GET	`/api/ai/providers/{id}/catalog`	AIProviderView	Browse that provider’s catalogue
POST	`/api/ai/providers/{id}/models/sync`	AIModelManage or sync key	Roster push
POST	`/api/ai/providers/{id}/sync-key`	AIProviderManage	Rotate sync key
GET/POST/PATCH/DELETE	`/api/ai/models`…	AIModelView / Manage	Models
GET/POST/PATCH/DELETE	`/api/ai/routes`…	AIRouteView / Manage	Role routes
GET	`/api/ai/resolve/{modelId}`	AIModelView	Resolve or 400 why not
POST	`/api/ai/sync-pricing`	AIModelManage	Sync OpenRouter prices
GET	`/api/ai/catalog`	(any auth)	Global provider catalog
GET/POST/DELETE + validate	`/api/ai/credentials`…	(any auth)	Tenant keys
GET/POST/PATCH/DELETE	`/api/ai/model-groups`…	(same family)	Named panels

Known gaps (as of this probe)

No periodic reconcile sweep (doc 95 #359 proposal 3). Flagging happens on write and on sync. A provider that quietly stops serving a model without a sync event is not detected until something touches it.

Doc 95’s “no PATCH/DELETE for routes” is closed on the live API.