Fetches information about a specific crawl.

Crawl Info

Fetches information about a specific crawl using its `crawl_id`.

Authorization

Retrieve crawl information

Olostep Docs

Welcome

Welcome 👋

Authentication

Scrape

The average scrape latency is an industry-leading 2-6 seconds

Latency

JS Rendering

Olostep's pricing is up to 60% cheaper than competitors

Cost effectiveness

Scrape a url with provided configuration and get content.

Create Scrape

Can be used to retrieve response for a scrape.

Get Scrape

Starts a new batch. You receive an `id` that you can use to track the progress of the batch as shown [here](/api-reference/batches/info).

Start Batch

Retrieves the status and progress information about a batch. To retrieve the content for a batch, see [here](/api-reference/batches/items)

Batch Info

Retrieves the list of items processed for a batch. You can then use the `retrieve_id` to get the content with the Retrieve Endpoint

Batch Items

Starts a new crawl. You receive a `id` to track the progress. The operation may take 1-10 mins depending upon the site and depth and pages parameters.

Start Crawl

Fetches the list of pages for a specific crawl.

Crawl Pages

Retrieve content of processed batches and crawls urls.

Retrieve Content

This is the endpoint to start the scraping agent (and optionally retrieve the scraped content using the expand parameters)

Start agent

This is the endpoint to retrieve the content of a scraped page (HTML/Markdown/PDF) by using a datasetId. Think of the datasetId as cached pointer to the page

Scrapes

Batches

Crawls

Utility

Legacy APIs

Crawl Info

Authorizations

Path Parameters

Response