Job Workflow

Jobs, Polling, and Not DDoSing Your Own Shoebox

The API is job-based because local AI work takes time, memory, and basic manners.

By Eric Rhea Last updated: June 2026 Status: Working research note
A cyberpunk job queue gate holding glowing request cubes before a small edge AI board.
A tiny queue protecting a tiny machine from very large ambitions.

The standard flow

Every generation or analysis operation should be treated as a job. The result is not guaranteed to exist immediately after POST. This is not a chat completion endpoint. It is a local machine doing local work, and sometimes local work involves fans.

  1. Create a job with POST /jobs/{kind}.
  2. Read the returned job_id.
  3. Poll GET /jobs/{job_id} until terminal status.
  4. Read outputs or call /jobs/{job_id}/outputs.
  5. Clear old completed work when useful.

Status values

Jobs move through queued, running, and sometimes stopping. Terminal states are succeeded, failed, stopped, and cleared. Agents should poll for those states, not sleep for a random amount of time and call it orchestration. That is not orchestration. That is a nap with confidence issues.

A copyable text-job pattern

API=http://127.0.0.1:8899

JOB_ID="$(curl -sS -X POST "$API/jobs/text" \
  -H 'Content-Type: application/json' \
  -d '{"prompt":"hello local API","mode":"fake"}' \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["job_id"])')"

curl -sS "$API/jobs/${JOB_ID}"
curl -sS "$API/jobs/${JOB_ID}/outputs"
curl -sS "$API/jobs/${JOB_ID}/events"

Use fake or benchmark replay modes for route mechanics. Save the real GPU time for when the route has proven it can walk across the room without falling into the cable drawer.

Heavy throttling is a feature

The API intentionally limits heavy domains: heavy_image, heavy_audio, heavy_video, and heavy_3d. It reports active jobs, pending jobs, per-domain limits, memory floors, cooldowns, admission status, and retry_after_seconds through GET /throttle.

curl -sS "$API/throttle"

Real field note from building this series: when I generated the article art, the first image succeeded, the next request hit 429 with heavy_cooldown, and the API told me how long to wait. That is not failure. That is a small machine demonstrating boundaries. Honestly, inspirational.

Stop and clear

Long-running jobs should be stoppable when supported. Completed jobs can be cleared as housekeeping.

curl -sS -X POST "$API/jobs/${JOB_ID}/stop" \
  -H 'Content-Type: application/json' \
  -d '{}'

curl -sS -X POST "$API/jobs/clear" \
  -H 'Content-Type: application/json' \
  -d '{}'

curl -sS -X DELETE "$API/jobs/${JOB_ID}"

The API records events, so stopping is not just "the agent got bored." You can inspect the lifecycle and see what happened.

Agent etiquette

  • Check /health and /throttle before launching heavy work.
  • Respect retry_after_seconds on 429. It is not a suggestion from a timid librarian.
  • Prefer benchmark_replay when testing route mechanics.
  • Poll job status and fetch outputs. Do not assume synchronous completion.
  • Do not expose this API to the public internet. It is intended for trusted LAN use.