Skip to main content
Olostep’s API is designed around objects. Understanding this design helps you build more effective integrations.

​
Everything is an Object

Every resource in Olostep is an object with a unique identifier. Whether you create it via the API, SDK, or dashboard — you get back an object you can reference, update, and query.

​
Objects Can Have Lifecycles

Some Olostep objects track state through a status field. This state machine pattern lets you know exactly where each resource is in its lifecycle.

​
Batches

Batches have two levels of status: the batch itself and individual items. Batch Status:
Batch-level failures are extremely rare. Batches almost always complete — even if some URLs fail, the batch itself reaches completed status. In the rare case of a catastrophic infrastructure failure (e.g., LLM service outage during enrichment), the batch may fail. This affects less than 0.01% of batches.
Item Status: Each URL in a batch is tracked as an individual item with its own status: Items can fail due to:
  • URL is blocked or returns an error
  • Parser output missing
  • Network/fetch errors
Failed items include an error object with code and message explaining the failure. The batch still completes — check each item’s status when processing results.

​
Crawls

Crawls always complete. Even if a crawl finds 0 URLs (due to robots.txt blocking or invalid start URL), the crawl status will be completed. Check the pages_count field to verify results.

​
Monitors

Monitors are long-lived objects with a richer lifecycle than one-shot resources:
Creating a monitor returns HTTP 202 with status: provisioning. The monitor becomes active once planning resolves its tracked targets — poll GET /v1/monitors/:monitor_id or stream provisioning events with ?stream=1. Only active monitors can be paused, and only paused monitors can be resumed. Updates return 409 while the monitor is still provisioning.

​
Retrieve Pattern

Many objects produce content that can be retrieved later. The retrieve_id pattern lets you fetch content without re-processing.
This pattern is used by:
  • Batch items — Each processed URL gets a retrieve_id
  • Crawl pages — Each crawled page gets a retrieve_id
The /v1/retrieve endpoint accepts formats parameter to specify which content types to return (html, markdown, json, text).

​
Webhooks: Event-Driven Updates

Instead of polling for status changes, configure webhooks to receive events when objects change state.

​
Metadata: Your Data Alongside Ours

Attach custom key-value pairs to objects using metadata. This lets you link Olostep resources to your internal systems.

​
Summary