Logging & Observability

Atamaia records significant actions in PostgreSQL and exposes them on three queryable surfaces. Agent runs add a fourth, denser trail.

Surface	Job	Permission
`SystemLog`	Platform telemetry — who called what, how long, what failed	`SystemViewAuditLog`
`AuditEvent`	Evidence — before/after, succeeded/failed, category	`SystemViewAuditLog`
`LogSubscription`	Who is told about which domain at which minimum level	`SystemViewLogs`
`AgentEvent`	Sequenced execution trace inside one run	`AgentRunView`

These are not the same table. A system log is operational. An audit event is evidence. An agent event is a step in a loop. Confusing them is how an indicator can read “all clear” because it queried the wrong store.

Canonical host: https://api.atamaia.ai. All listed routes require authentication except where noted. Responses wrapped in ApiEnvelope<T> except file downloads.

System logs

The SystemLog row is the platform-wide operational trail.

DTO (`SystemLogDto`)

Field	Type	Notes
`id`, `guid`	long, UUID	D3
`level`	`SystemLogLevel`
`entityType`	`LogEntityType`
`entityGuid`	UUID?
`action`	string	e.g. `memory.created`, `agent.run.started`
`source`	string?	Component that wrote the row
`userId`	long?	From `ClaimTypes.NameIdentifier`
`identityId`	long?	Not written by `SystemLogService.LogAsync` (#403) — filter exists, matches nothing from this path
`apiKeyId`	long?	Set only for API-key principals
`correlationId`	UUID?	From `HttpContext.Items["CorrelationId"]`
`httpMethod`, `httpPath`, `httpStatusCode`, `durationMs`		Request context
`clientIp`	string?
`detailsJson`	string?	Structured payload
`createdAtUtc`	datetime

TenantId lives on the entity and is applied by the global query filter. It is not on the DTO.

Levels (`SystemLogLevel`)

Level	Value	Purpose
Debug	1	Diagnostic detail
Info	2	Standard operations
Success	3	Confirmed successful actions
Warning	4	Anomalies, degraded operations
Error	5	Failures requiring attention
Critical	6	Failures that must not be silent (backup/alerter path)

Critical was added after the original page. The backup-failure alerter writes level 6, domain Data. See Deployment.

Entity types (`LogEntityType`)

None · Memory · Identity · Project · Task · Doc · Fact · Message · User · OrgUnit · AgentRun · ChatSession · Reflection · Connector · Role · Session

Automatic context

SystemLogService.LogAsync pulls from HttpContext on every write:

User ID — ClaimTypes.NameIdentifier (issued by both JWT login and API-key auth). A previous user_id claim name was never issued; that left user_id null on historical rows.
API key ID — AtamaiaClaims.ApiKeyId, API-key principals only.
Correlation ID — HttpContext.Items["CorrelationId"].
Client IP, HTTP method, path.

Identity ID is deliberately not set (#403). It is derivable from user_id via the unique identity↔user mapping. Snapshotting it on the log would create a second truth that drifts when the mapping changes. The column and the identityId filter remain; they match nothing written by this service.

List

GET /api/system-logs

Auth: SystemViewAuditLog.

Parameter	Type	Notes
`level`	`SystemLogLevel?`
`entityType`	`LogEntityType?`
`entityGuid`	UUID?
`userId` / `identityId` / `apiKeyId`	long?	`identityId` filter is live; writes do not populate it
`correlationId`	UUID?	Follow one request
`from` / `to`	datetime?
`page` / `pageSize`	int	Default 1 / 50. Not `limit`/`offset`
`sort` / `dir`	string	Whitelist: `createdAtUtc` (default desc), `level`, `action`, `source`, `entityType`, `httpMethod`, `httpPath`, `httpStatusCode`, `durationMs`
`filter`	bag	Same fields as named params plus `source` contains, `httpMethod` exact

Get one

GET /api/system-logs/{id}

Summary (SPA indicator)

GET /api/system-logs/summary?windowHours=24

A count endpoint, not a page of rows. The header indicator polls this. Fetching rows to count them would move real volume every 30 seconds to render one number.

Field	Meaning
`windowHours`, `since`	Window
`critical`, `error`, `warning`	Counts in window
`actionable`	Combined trouble count

It does not swallow its own failures. If this throws, the client must show UNKNOWN, not a reassuring zero (#225).

Audit events

A separate table. GET only — that is the design. There is no POST, PUT, PATCH, or DELETE. Rows are written by the service layer at the moment of the action. An audit trail with a delete endpoint is not evidence.

GET /api/audit-events
GET /api/audit-events/{id}

Auth: SystemViewAuditLog.

Categories (`AuditCategory`)

None · CrossIdentityAccess · Authorization · TenantPolicy · Credential · Deletion · Authentication · Administrative

List filters

category, entityType, entityGuid, userId, apiKeyId, correlationId, succeeded, from, to, plus paging/sort/filter bag.

DTO (`AuditEventDto`)

Field	Notes
`category`, `action`, `entityLabel`
`entityType`, `entityGuid`, `entityId`
`userId`, `username`, `apiKeyId`
`succeeded`, `reason`, `failureReason`
`correlationId`, `clientIp`
`beforeJson`, `afterJson`	State snapshots

Use this store when the question is “was this allowed, and what changed.” Use system logs when the question is “what did the request do and how long did it take.”

Log subscriptions and system health

GET    /api/log-subscriptions
GET    /api/log-subscriptions/{id}
POST   /api/log-subscriptions
PUT    /api/log-subscriptions/{id}
DELETE /api/log-subscriptions/{id}

Auth: SystemViewLogs on the controller (all five verbs).

Create body (CreateLogSubscriptionRequest):

Field	Type	Notes
`identityId`	long?	Who is subscribed
`domain`	`LogDomain`	Required
`minLevel`	`SystemLogLevel`	Required
`sourceHost`	string?	Optional host filter

Domains (`LogDomain`)

Platform · Agent · Memory · Embeddings · Channels · Auth · Billing · Inference · Sync · Web · Data

Hydration’s systemHealth section is subscription-driven. Zero subscriptions → the section is null. A Critical row in a domain nobody subscribes to is written, stored, and seen by no one. That is worse than no alerter: it converts “we have no alerting” into “we think we have alerting.”

SystemViewPlatformLogs additionally lets a hydration identity see cross-tenant platform rows. Separate from SystemViewLogs because it crosses the tenant boundary.

Agent execution tracing

For agent runs, AgentEvent is a second, denser layer. System logs capture API-level actions. Agent events capture every internal decision, tool call, and failure inside one loop. See Agents for the run lifecycle.

Event types (`AgentEventType`)

52 values (the original page said 37). Grouped by the enum’s numbering:

Category	Events
Planning	PlanCreated, PlanRevised, StepStarted, StepCompleted
LLM	LlmRequest, LlmResponse
Tool use	ToolCallRequested, ToolCallResult, ToolCallBlocked
Workspace	FileCheckedOut, FileCommitted, FileConflict
Decisions	Decision, Observation, Reasoning
Failures	EmptyResponse, PrematureIntent, StaleLoop, ContextOverflow, ToolTimeout, DependencyBlocked, BudgetExceeded
Loop detection	DuplicateRead, RepeatedToolCall
Context	ContextWarning50, ContextWarning75, ContextWarning90, ContextCompacted, ContextFlushed
Lifecycle	Checkpoint, Paused, Resumed, BudgetWarning, BudgetExtended
Escalation	EscalationCreated, EscalationResolved, EscalationNotificationFailed
Children	ChildSpawned, ChildCompleted, ChildFailed, ChildMessage, ParentResponse, SystemMessage
Audit	RunStarted, RunCompleted, RunFailed
Interaction	InteractionMessageReceived, InteractionMessageSent, PauseChatLinked, PauseChatSummarized, ApprovalRevision
Other	AssertionWithoutEvidence

File checkout/commit/conflict and SystemMessage are not on the original page.

Each event carries sequence (monotonic within the run), type, summary, dataJson, tokensUsed, durationMs, createdAtUtc.

Query

GET /api/agent/runs/{runId}/events?sinceSequence=50&limit=200

Auth: AgentRunView.

sinceSequence lets a UI poll for new events without replaying the whole trail.

Analytics

GET /api/agent/analytics?modelId=&role=

Aggregates: total/completed/failed runs, success rate, avg iterations/tokens/duration, total cost, avg feedback score.

Correlation across layers

X-Correlation-Id header — caller may supply one; otherwise the middleware generates a UUID. Always echoed on the response.
System logs — store that UUID as correlationId.
ApiEnvelope.requestId — same UUID.
Agent events — linked to their run, which is linked to the creating request.

CorrelationIdMiddleware is the stamp. ApiEnvelopeMiddleware copies HttpContext.Items["CorrelationId"] onto requestId.

Request / response envelope

All JSON API responses use ApiEnvelope<T>:

{
  "ok": true,
  "requestId": "550e8400-e29b-41d4-a716-446655440000",
  "data": { },
  "count": 42,
  "error": null,
  "errorCode": null,
  "hint": null
}

On error, ok is false and error / errorCode / hint are populated. requestId matches the correlation header and the system-log correlationId.

Soft delete alignment

D15: soft delete only, never hard delete.

System logs reference entities that still exist (IsDeleted = true). A GUID on a log still resolves.
There is no delete endpoint on system logs or audit events.
Agent events are append-only.

What this is not

Not Redis / Prometheus / Loki. Those appear in the old deployment sketch. Prod observability for Atamaia itself is these PostgreSQL tables plus hydration systemHealth. See Deployment.
Not the code-graph drift report. That is a different store. See Code graph.

Endpoint catalog

GET    /api/system-logs
GET    /api/system-logs/summary
GET    /api/system-logs/{id}
GET    /api/audit-events
GET    /api/audit-events/{id}
GET    /api/log-subscriptions
POST   /api/log-subscriptions
GET    /api/log-subscriptions/{id}
PUT    /api/log-subscriptions/{id}
DELETE /api/log-subscriptions/{id}
GET    /api/agent/runs/{runId}/events
GET    /api/agent/analytics
GET    /api/agent/runs/{runId}/feedback