Welcome

Olostep Docs

Authentication

Scrape

Get all the URLs on a website in seconds. Useful for site structure analysis or decide which url to scrape.

Maps

Reuse custom cookies, authentication, and cached data for your requests

Context

Use Olostep in AI Agents and apps like Claude Desktop, Cursor, Windsurf etc

MCP Server

The average scrape latency is an industry-leading 2-6 seconds

Latency

JS Rendering

Olostep's pricing is up to 60% cheaper than competitors

Cost effectiveness

Learn how to extract content as LLM-friendly markdown from any web page.

Get Markdown of a Website

Get the Markdown of a Website

SERP API to scrape Google search results using Olostep and a Python code snippet.

Google Search Results

Get Google Search Results (python)

SERP API to scrape Google search results using Olostep and a Javascript code snippet.

Google Search Results (JS)

Get Google Search Results (JS)

SERP API to scrape Google Maps location data using Olostep and a Python code snippet.

Google Maps Results

Get Google Maps Results

Get a complete map of Stripe's website structure to understand its organization and discover available content.

Get all the URLs of a Website

Extract All URLs from Stripe's Website

Filter and extract only the blog URLs from Stripe's website for targeted content analysis.

Get specific URLs in Website

Extract Blog URLs from Stripe's Website

Learn how to crawl and extract content from Stripe's blog posts.

Crawl specific pages in Website

Crawl and Extract Content from Stripe's Blog Pages

Learn how to use Olostep to track product prices at scale on an e-commerce.

Price Monitoring

Price Tracking with Olostep

Start a batch scrape to extract content from up to 100k URLs in 5-7 mins

Batch Scraping

Get the content of multiple websites in one go

Scrape a url with provided configuration and get content.

Create Scrape

Can be used to retrieve response for a scrape.

Get Scrape

Starts a new batch. You receive an `id` that you can use to track the progress of the batch as shown [here](/api-reference/batches/info).

Start Batch

Retrieves the status and progress information about a batch. To retrieve the content for a batch, see [here](/api-reference/batches/items)

Batch Info

Retrieves the list of items processed for a batch. You can then use the `retrieve_id` to get the content with the Retrieve Endpoint

Batch Items

Starts a new crawl. You receive a `id` to track the progress. The operation may take 1-10 mins depending upon the site and depth and pages parameters.

Start Crawl

Fetches information about a specific crawl.

Crawl Info

Fetches the list of pages for a specific crawl.

Crawl Pages

This endpoint allows users to get all the urls on a certain website. Uses cursor to paginate larger websites by limiting each response size to 10MB.

Get Started

Features

Integrations

Why Olostep?

Welcome

Explore Features

API Reference