Reference Images: Diffusion with Adult Supervision

The current verified path

The previous reference-image gap is closed with a hostable SD1.5 ComfyUI image-to-image path. The main route is POST /jobs/image with mode: "img2img". The engine is ComfyUI, the checkpoint is CyberRealistic_V9_FP16.safetensors, and outputs are returned under /assets/{filename}.

The source docs call out a practical limit: the img2img path is intended for images with max side around 768. This is local edge hardware, so "make it enormous and see what happens" remains a valid way to find out that you are the problem.

Reference modes

Mode	Engine	Use it when
`img2img`	ComfyUI diffusion	You want the default verified reference-image path.
`reference_edit`	ComfyUI diffusion alias	You need compatibility wording for an edit-style request.
`reference_generation`	ComfyUI diffusion alias	You need compatibility wording for generation from a reference.
`reference_diffusion`	ComfyUI diffusion alias	You want the route name to say exactly what it does.
`reference_stylize`	Local Pillow fallback	You only need a quick local pixel transform. It is not diffusion and not model-backed identity transfer.

Copyable img2img call

The input image is sent as base64. The output job includes a generated image path and a normalized uploaded reference path.

API=http://127.0.0.1:8899

IMAGE_B64="$(python3 - <<'PY'
import base64, sys
from pathlib import Path
print(base64.b64encode(Path(sys.argv[1]).read_bytes()).decode())
PY
/path/to/reference.png)"

curl -sS -X POST "$API/jobs/image" \
  -H 'Content-Type: application/json' \
  -d "{
    \"prompt\":\"turn this reference character into a polished cinematic poster while preserving the composition\",
    \"mode\":\"img2img\",
    \"denoise\":0.35,
    \"steps\":6,
    \"image_base64\":\"${IMAGE_B64}\"
  }"

Then poll the returned job_id just like any other job. The output contains outputs[0].path, outputs[0].reference_path, engine, and model_id.

Denoise is the steering wheel

denoise controls how hard the diffusion process is allowed to depart from the reference. Lower values preserve composition and identity better. Higher values give the model more creative room, which is polite phrasing for "it may wander off and return wearing a cape."

How these article images were made: I drew simple 16:9 seed compositions locally, then submitted them through /jobs/image with mode: "img2img". One large route-map seed ran too long and was stopped. Smaller seeds succeeded quickly. The API taught the lesson while providing the syllabus.

The honest limitations

The current path is useful, not omnipotent. It is SD1.5-class reference diffusion, so it can produce handsome artifacts but it is not a perfect brand-preserving product editor. The local reference_stylize mode is even more explicit: it preserves and stylizes pixels locally and says plainly that model-backed identity transfer is not installed.

ControlNet Canny is the next optional upgrade because it would help preserve structure. IP-Adapter is not the first target on this hardware profile because it would require extra custom nodes plus a large CLIP vision encoder. "But what if we installed everything?" is how small machines become decorative heaters.