Observability for Placement Decisions

In Admission Control for AI Workloads, I narrowed admission to three outcomes: admitted, queued, or rejected. That made the capacity decision explicit, but it left an operational question unanswered.

After a workload moves through placement, admission, execution, and inference, how do I explain why it ran where it did?

A model request can succeed while the surrounding system makes the wrong operational decision. A background task may wait behind lower-priority work. A local model may load even though a qualified runtime is already warm elsewhere. A remote fallback may keep latency low while quietly increasing cost. A queue may accept work it cannot complete before the deadline.

Request logs alone do not explain any of that. The useful trace connects the infrastructure decision to what happened afterward.

The signals exist, but the workload is missing

My current system already records pieces of this path.

The inference proxy returns the selected backend, model, route reason, inferred complexity, and request latency. It exposes request and token counts per backend. Queue-backed workers and scheduled jobs publish run and audit records to a shared ledger. The fleet view shows job state across several execution lanes.

Those signals remain component-level. The proxy knows a request happened. The queue knows an item was claimed. The worker knows a process ran. GitHub knows a pull request appeared. The ledger knows a job succeeded. Correlating them still requires operator knowledge.

There is no shared identifier connecting the original placement intent, proxy route, queue record, worker attempt, inference requests, and final artifact.

That is the gap I want to close. I do not need a new observability stack. I need a small contract that lets the systems I already operate describe one workload lifecycle.

One workload needs a hierarchy of identities

A single workload may contain several placement decisions, admission attempts, execution attempts, and inference requests. workload_id is the root of that hierarchy, not the only identifier in it.

workload_id
  ├─ decision_id
  │    └─ admission_attempt_id
  │         └─ claim_id
  ├─ execution_attempt_id
  │    └─ inference_request_id
  └─ artifact_id

The workload identifier should be created at the first stable boundary and propagated through every component that supports it. More specific identifiers preserve the relationships underneath it.

That distinction matters during retries. If a worker restarts or a provider call is repeated, I need to tell a new workload from another attempt to complete the same one. A caused_by or parent reference can connect each event to the decision or attempt that produced it.

The decision record is the center

The placement layer should write a decision record before execution begins. For work admitted immediately, it can remain small.

{
  "workload_id": "work_01J...",
  "decision_id": "decision_01J...",
  "admission_attempt_id": "attempt_01J...",
  "profile": "private-interactive",
  "version": 3,
  "policy_version": "2026-08-18",
  "execution_target": "local-worker-pool",
  "inference_target": "local-mlx",
  "admission_outcome": "admitted",
  "claim_id": "claim_01J...",
  "decision_reason": "warm capacity available",
  "excluded_targets": [
    {"target": "cloud-inference", "reason": "locality policy"},
    {"target": "local-llamacpp", "reason": "context requirement"}
  ],
  "fallback_applied": false,
  "decided_at": "2026-08-18T13:42:11Z"
}

A queued outcome needs a different record. Queue acceptance reserves a place to wait, not execution capacity on the eventual worker. The initial decision can name the queue and the eligible targets considered, but execution and inference targets are recorded only when a worker claims the task and admission runs again.

That preserves the boundary from the previous article. Redirection is another admission attempt against another eligible target, not a terminal outcome. A fallback or degraded path is an attribute of the path eventually used.

The record captures the policy result without copying prompts, transcripts, repository content, or tool output into the placement layer. It also creates a stable join point for runtime metrics, queue events, worker logs, provider usage, and artifacts.

Record exclusions, not only selections

Most routing systems tell me where a request went. Fewer explain why other targets were excluded.

That omission becomes expensive during incidents. A workload may use remote inference even though local infrastructure appears healthy. The missing fact could be that the local model did not meet the context requirement, its observed startup time exceeded the deadline, or its capacity claim was already committed to interactive work.

A sudden rise in one exclusion reason can reveal operational drift:

• insufficient_memory may signal a larger model or a competing workload
• queue_deadline may show that worker capacity no longer matches demand
• unqualified_runtime may expose an incomplete rollout or validation gap
• policy_locality may identify a profile with no viable fallback
• cost_budget may show that remote fallback is being used too often

Selection tells me what happened. Exclusion often tells me what needs to change.

Exclusion records can also support policy testing, but only when the original inputs remain available. Replaying a decision requires references to the profile and policy revisions, candidate inventory, qualification state, capacity observations, and the time each observation was made. The outcome alone is not enough to reconstruct the choice.

Time has several components

End-to-end latency is useful, but it is too coarse to diagnose placement behavior.

A workload can spend time in placement and admission, wait in a queue, prepare a workspace, load a model, execute tools, generate inference, run verification, and deliver an artifact. Those stages should not be collapsed into one duration.

A slow result caused by queue depth needs a different response from one caused by cold model loading. A fast model response followed by slow artifact delivery points somewhere else entirely.

The same breakdown makes local and remote paths easier to compare. Tokens per second may make a local runtime look competitive while its cold-start cost makes it unsuitable for short interactive work. A queue-backed worker may execute quickly while still missing the workload deadline because it waited too long to start.

Queue acceptance is not completion

Asynchronous execution often hides behind a successful enqueue response. The caller sees that the task was accepted, but the system may still fail to claim it, start it, complete it, or deliver the result.

I need queue lifecycle events connected to the original decision: enqueued, claimed, started, heartbeat, completed, failed, cancelled, expired, and artifact delivered.

Queue depth alone is not enough. Oldest-item age, claim delay, execution duration, retry count, and deadline risk are often more useful. The record should also distinguish queue wait from worker startup. If a task is claimed quickly but spends several minutes preparing a workspace or loading a model, adding queue consumers will not help.

Cancellation and expiry need records too. A task that completes after the caller no longer needs it still consumed capacity and cost. Without that signal, the system appears successful while doing useless work.

Measure the workload contract

I would start with a small set of measurements that can test placement and capacity policy:

• admitted, queued, and rejected outcomes by profile
• selection and exclusion counts by target and reason
• fallback rate, destination, and policy compliance
• placement, admission, queue, startup, execution, and delivery time
• deadline misses and completion rate by selected path
• capacity claims, target saturation, and oldest queued item
• estimated cost by profile, target, and outcome

The point is not to collect every possible measurement. It is to make policy and capacity decisions falsifiable.

If a profile says interactive work should prefer a warm local target, the metrics should show whether that happens and whether the workload meets its latency objective. If admission reserves local capacity for interactive work, the system should show what background work waited and whether the reservation improved user-visible performance.

Service-level objectives belong at the same boundary. Useful examples include the percentage of workloads receiving an admitted, queued, or rejected result within the decision window; the percentage of queued workloads starting before their deadline; the percentage of private workloads staying inside their permitted locality; and the percentage of fallback executions remaining within declared policy.

A remote fallback can keep the completion SLO green while exhausting the cost budget. That does not make the policy healthy. It means reliability needs more than one objective.

A shared event envelope is enough to start

The next step in my system is a shared envelope rather than a shared backend.

event_id
workload_id
decision_id
admission_attempt_id
execution_attempt_id
inference_request_id
caused_by
claim_id
profile
version
policy_version
execution_target
inference_target
admission_outcome
stage
started_at
finished_at
result

Not every event needs every field. A queue event has no inference request. An inference event may not own an artifact. The common identifiers and causal references are what allow the records to be joined without pretending each component has the same lifecycle.

Existing logs, metrics, and traces can remain where they are. The contract can be represented as OpenTelemetry spans, events, or log records and exported through an OpenTelemetry pipeline, written to the existing ledger, or stored locally. Placement observability should not depend on one vendor or storage system.

Preserve the data boundary

More visibility is not always better.

The placement layer needs enough evidence to explain and evaluate an infrastructure decision. It does not need the full prompt, transcript, email body, repository content, or tool output. Those belong to the harness or a protected audit system with their own access and retention rules.

Classifications and identifiers can still be sensitive. They need access control, bounded retention, and redaction or pseudonymization where an identifier could expose a person, repository, customer, or workload purpose.

The event should retain policy inputs, target state, outcomes, timing, and resource impact while keeping workload content out of the central record.

Evidence changes governance

A shared record would let me follow one workload from intent to artifact, explain why each target was selected or excluded, and see where time, capacity, or cost changed the result. It would also expose whether the policies described in Placement Profiles and the capacity claims from Admission Control behave as intended after they meet a real queue, worker, runtime, and provider.

Observability does not make an autonomous system safe by itself. It makes the system answerable.

Coming next

The final article in this series moves from evidence to authority.

• From Model Routing to Workload Placement: why task execution and inference are separate placement decisions
• Placement Profiles: how workloads describe durable requirements without naming infrastructure
• Admission Control for AI Workloads: how capacity claims decide whether eligible work can run
• Observability for Placement Decisions: how to connect placement, admission, execution, and outcome records
• Governance for AI Workloads: how authority, verification, rollback, and human control bound automation