In From Model Routing to Workload Placement, I separated two decisions that are often treated as one: where a task executes and where its inference requests are served.
That distinction helped, but it exposed another problem.
I still needed a way for a workload to describe what it required without knowing the current topology.
A coding task may need access to a private repository, a controlled workspace, and frontier reasoning. A scheduled maintenance job may need local-only execution but tolerate a long queue. An interactive request may allow approved cloud inference while keeping its tools and artifacts local.
Those requirements are more durable than the hostnames, provider aliases, queue names, and model endpoints used to satisfy them.
I have been using the term placement profile for that boundary.
The request should not contain the route
One early version of my system encoded placement decisions close to the caller.
A scheduled job knew which worker to use. A coding workflow knew which provider alias to call. A private task depended on a particular local endpoint.
Timeouts and fallback behavior were spread across shell scripts, prompts, scheduler entries, environment variables, and service configuration.
That worked until the infrastructure changed.
A worker moved. A model was replaced. A local runtime became unavailable. A task that had been synchronous moved behind a queue because it regularly exceeded the request window. Each change leaked back into the workload definition.
The route had become part of the request.
A placement profile separates those concerns: the workload states what must be true, and the placement layer works out which current targets can satisfy it.
The profile carries the requirements that decision needs, independent of whichever host, runtime, model, or queue ends up chosen.
What belongs in a profile
The exact schema is still evolving, but the useful categories are becoming clear.
profile: private-interactive
version: 3
task_execution:
locality: private-ok
mode: synchronous
workspace: controlled
required_capabilities:
- repository-access
- bounded-tools
inference:
locality: cloud-ok
reasoning: frontier
latency_class: interactive
fallback:
allow_queue: false
allow_remote_execution: false
on_no_target: rejectThe task-execution section describes where the harness and tools may run and where the artifact may be created. Inference is separate: what may serve the model request. Fallback covers what the system is allowed to do once the preferred path isn't available.
Keeping those parts separate matters. A workload may allow remote inference without allowing the repository, tools, or execution environment to leave private infrastructure. Another may allow the entire task to move to a qualified remote worker. A local-only workload may prefer rejection over either option.
Some requirements are not negotiable
Not every field should influence placement in the same way.
Some are hard constraints.
- data must remain inside an approved boundary
- task execution must use a controlled workspace
- remote execution is prohibited
- a required tool or accelerator must be available
- work that cannot meet the request window must use an asynchronous path
- inference may use only qualified targets
Others are preferences.
- prefer a target that is already warm
- prefer local capacity when the expected quality is similar
- prefer the lowest-cost qualified path
- avoid displacing interactive work
- prefer a target in the same data or failure domain
A preference may be relaxed. A privacy boundary should not be.
This is why fallback behavior has to be explicit. Without it, the platform eventually invents a fallback through retries, timeout handling, or provider configuration. That is usually where the least visible policy ends up living.
A background workload may wait, while an interactive request may redirect to approved cloud inference. A private task may reject immediately, or a long-running task may move from a synchronous path to a queue-backed worker.
The system should not discover those choices by accident.
One profile, two placement decisions
A placement profile should preserve the distinction from the first article rather than collapsing the task and the model into one destination.
Consider a repository review.
The task needs a private workspace with Git and repository credentials. The reasoning quality needed for the review may justify an approved frontier model. The correct placement may therefore be:
task execution → self-hosted worker
inference → approved cloud model
artifact → draft pull requestAnother review may remain local end to end. A simple summarization request may have no separately placed task at all. The same policy can still place them differently.
A model name is therefore a poor placement profile. It says nothing about where tools execute, what data may leave the environment, whether the task belongs on a queue, or what should happen when the model is unavailable.
A profile narrows the choices
The profile does not make the final decision by itself.
A useful placement flow looks more like this.
Policy first removes targets the workload is not allowed to use. Qualification removes targets that have not been validated for that class of work. Admission checks whether the remaining targets can accept the workload now. Preferences then help choose among the valid options.
The order matters.
A low-cost target does not become valid if it violates locality. Responding to SSH doesn't make a worker qualified. And fitting in memory says nothing about whether a model is ready, approved, or fast enough to meet the deadline.
The final result should explain exclusions as well as selection.
{
"execution_target": "self-hosted-coder",
"inference_target": "approved-frontier-api",
"excluded_targets": [
{ "target": "local-standard-model", "reason": "reasoning requirement" },
{ "target": "managed-sandbox", "reason": "remote execution prohibited" }
]
}Selection tells me what happened. Exclusion tells me whether the system behaved as intended.
This does not replace a scheduler
Schedulers already provide resource requests, node selectors, affinities, priorities, runtime classes, and admission policies.
Those mechanisms should remain where they already work.
The gap appears when one workload crosses boundaries that no single scheduler owns: a laptop trigger, a self-hosted worker, a queue, a local model runtime, an approved cloud API, and a code-review system.
A placement profile gives those systems a shared statement of workload intent. Kubernetes, systemd, a queue, a model gateway, or a managed sandbox can each enforce the part they own, without each one inventing its own version of the same policy.
What exists today
The current system already carries parts of this contract.
Scheduled workloads record locality, duration, reasoning, runtime, and placement metadata. The inference proxy applies a narrower set of runtime rules based on caller location, request complexity, isolation, backend health, and WAN availability.
The important gap is that those requirements are not yet expressed through one reusable placement-profile schema. Hosts are still commonly chosen ahead of execution, and the proxy and worker paths interpret policy separately.
I do not think this article needs to turn the package tooling into the main subject. It is one implementation surface for the profile, not the architectural idea itself.
The useful next step is smaller: allow a workload package or caller to reference a versioned profile, then resolve that profile against the targets available in the current environment.
Profiles can become another form of drift
Centralizing intent does not automatically make it clear.
A profile named private, interactive, or batch can become another opaque alias if the underlying requirements are not visible.
Two workloads may both call themselves private while assuming different data boundaries. Interactive is weak without a latency expectation. Batch still needs a maximum queue time, retry behavior, artifact destination, and cost boundary.
Profiles also need versioning. Changing a fallback rule or locality boundary in place can alter the effective authority of every workload that references it. A breaking change should create a new profile version rather than silently reinterpret old work.
Workload-specific constraints still belong with the request. A shared profile should supply defaults without hiding what changes the decision.
The goal is to remove infrastructure coupling, not detail.
Eligibility is not admission
A profile can describe exactly what a workload needs and still produce several eligible targets.
That does not mean any of them can accept the work now.
The preferred local runtime may be short on memory. A qualified model could be cold, or a worker might have no free workspace. Sometimes the queue is already past deadline, or the remaining cost budget just doesn't cover the approved cloud path.
That is the next boundary.
Admission control turns a valid placement option into an operational result: admitted, queued, redirected to another eligible target, or rejected by capacity or policy.
The profile describes the work. Admission protects the system that has to run it.
Coming next
This article stops at eligibility. The next post goes into admission itself.
- Admission Control: how capacity, readiness, deadlines, and policy determine whether eligible work can run
- Observability for Placement Decisions: how to record why work ran, waited, moved, or was rejected
- Governance for AI Workloads: how authority, verification, rollback, and human control bound automation