DEVELOPER REFERENCE

Inference API

Submit single or batch text inference, poll or stream progress, cancel unfinished work, and retrieve structured results.

Open Dashboard https://pool.rungpu.io/api/v1

Authenticate

Use a project key as a bearer token. Project keys are restricted to inference and inference-result operations.

Authorization: Bearer tk_proj_...
Content-Type: application/json

Operations

POST/inferSubmit one job

GET/infer/{job_id}Poll one job

GET/infer/{job_id}/streamStream one job with SSE

POST/infer/{job_id}/cancelCancel one job

POST/infer/batchSubmit independent batch items

GET/infer/batch/{batch_id}Poll a batch

GET/infer/batch/{batch_id}/streamStream a batch with SSE

POST/infer/batch/{batch_id}/cancelCancel unfinished batch items

GET/infer/resultsList retained media results

GET/infer/results/{asset_id}/downloadCreate a two-minute download grant

DELETE/infer/results/{asset_id}Delete a retained result

Submit

curl -X POST https://pool.rungpu.io/api/v1/infer \
  -H "Authorization: Bearer $RUNGPU_PROJECT_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "llama3.2",
    "input": { "prompt": "Summarize this document" }
  }'

A 202 response includes status_url, stream_url, and cancel_url. Batch submission uses /infer/batch with a modality and ordered items.

Poll

GET the returned status URL until the job reaches completed, failed, or cancelled.

Stream

SSE emits status, complete, timeout, and error events. Complete contains the terminal result.

Cancel

POST the cancellation URL. Batch cancellation leaves already-terminal children unchanged.