Batch processing

Batch moves big, delay-tolerant work off the realtime path — thousands of translations, summaries, or OCR pages processed for about half the price.

In this guide

• Know when batch beats realtime
• Walk the submit, poll, and results_url lifecycle
• Poll jobs reliably from Python
• Mix chat, audio, OCR, and Helios calls in one account

Choose batch when waiting saves money

Batch costs roughly 50 percent less than the same realtime call and supports much higher queue limits. The trade-off is latency: results arrive in minutes to hours, not seconds.

Good batch fits: Dashain catalog translation, nightly ticket triage, invoice backfill for NPR ledgers. Keep live support, OTP-adjacent flows, and in-call voice on realtime chat or streaming instead.

Follow the job lifecycle

Every job moves through three stages: submit a JSONL file of requests, poll the job status until it reads completed, then download from results_url. Partial failures still complete — check per-row statuses.

# 1. Submit
curl -X POST https://api.sagea.space/v1/batch \
  -H "Authorization: Bearer $SAGEA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"input_file": "file-ktm-reviews", "model": "sage-2-5-celer"}'
 
# 2. Poll
curl https://api.sagea.space/v1/batch/batch-abc123 \
  -H "Authorization: Bearer $SAGEA_API_KEY"
 
# 3. Fetch results_url from the completed job, then download
curl -O "<results_url_from_status_response>"

Each input line needs a custom_id you choose — for example review-0001 — so results join back to your rows even if ordering changes. Endpoint schemas live in the Batch API reference.

Poll without hammering the API

Poll with exponential backoff, persist the job ID, and make completion handling idempotent so a retried webhook does not double-apply results.

import time
 
job_id = job.id
for attempt in range(20):
    status = client.batch.retrieve(job_id)
    print(f"attempt {attempt}: {status.status}")
    if status.status == "completed":
        break
    if status.status == "failed":
        raise RuntimeError(f"batch {job_id} failed")
    time.sleep(min(30 * (attempt + 1), 300))
 
results = client.batch.results(job_id)
for row in results.iter_lines():
    apply_to_database(row)  # keyed by custom_id

Alert after 2 hours without completed, and keep the original JSONL for 30 days so you can resubmit only the failed custom_id values instead of the whole file.

Mix endpoints in one account

One API key and one billing account cover chat, audio, OCR, and Helios batch rows. Split by endpoint, not by account: chat summaries on sage-2-5-celer, narration drafts via audio, scanned receipts via OCR, and KYC re-checks via Helios.

jobs = {
    "reviews_np": client.batch.create(input_file="file-reviews", model="sage-2-5-celer"),
    "invoices": client.batch.create(input_file="file-receipts-ktm", model="sage-2-4-actus"),
}

Track cost per job label — Kathmandu teams often find OCR plus sage-2-5-celer summarization cheaper than one giant sage-2-4-actus pass over raw images. See per-model rates in Choosing a Model.

Best practices

• Validate 20 sample rows realtime before submitting 20,000 to batch.
• Keep prompts deterministic (temperature 0.2) so reruns are comparable.
• Shard files at 10k rows each for easier retries and parallel completion.
• Store results_url outputs in your own bucket — links expire.
• Reconcile every custom_id: success, model refusal, and row error.

What's next

• Batch API reference
• Chat Completions reference
• Invoice extraction cookbook