Skip to main content
Your OpenClaw agent can read documentation, but it can’t read the web. The Olostep Web Agent plugin fixes that — search, scrape, crawl, and extract structured data from any website, directly inside your agent workflow. One install gives you 13 skills for high-level tasks (debug an error from live StackOverflow threads, write integration code from current docs, research tools with structured comparisons) and a 9-tool MCP server for direct programmatic access. JS-heavy SPAs, CAPTCHAs, Cloudflare, residential proxies — handled automatically.

​
Installation

clawhub install olostep
That’s it. If you prefer to wire the MCP server manually, add this to your OpenClaw configuration: 
{
  "mcpServers": {
    "olostep": {
      "command": "npx",
      "args": ["-y", "olostep-mcp"],
      "env": {
        "OLOSTEP_API_KEY": "your-api-key-here"
      }
    }
  }
}
Get a free API key at olostep.com/auth — 500 requests/month, no credit card.

​
Skills

​
Core Data Skills

These six skills are the building blocks. Each one does one thing well, and your agent can compose them for complex workflows.

scrape

Any URL to clean markdown, HTML, JSON, or text. Full browser rendering, anti-bot bypass, geo-targeting, browser actions (click, scroll, type), and pre-built parsers for common site types.

search

Three modes: AI-synthesized answers with citations, raw Google SERP data (organic results, PAA, knowledge graph), and domain-scoped URL discovery.

crawl

Start from one URL, follow links, scrape every page discovered. Set max pages, include/exclude URL patterns, and control crawl depth.

batch

Scrape up to 10,000 URLs in parallel with full rendering. Tag each URL with custom_id to map results back to your sources.

map

Discover every URL on a website without scraping any of them. Filter by glob patterns, rank by search query relevance, cap with top_n.

answers

Ask a plain-language question, get an AI-synthesized answer grounded in live web sources. Pass a json parameter to get structured output matching any schema you define.

​
Workflow Skills

These seven skills chain multiple core operations together. They handle the thinking — your agent just picks the right skill for the job.

research

“Compare the top 3 ORMs for Node.js” — the agent searches multiple sources, scrapes pricing and feature pages, and returns a structured comparison with citations and a recommendation.

debug-error

Paste a stack trace or error message. The agent searches GitHub issues and StackOverflow for that exact error, scrapes the relevant threads, and returns a fix grounded in what developers who hit the same problem actually did.

docs-to-code

Point at a docs URL. The agent scrapes the current API reference and writes working integration code from what is actually published — not from stale training data that may reference deprecated methods.

migrate-code

Give it your current version and target version. The agent scrapes the migration guide, extracts every breaking change with before/after patterns, and rewrites your code to match.

extract-schema

Turn any unstructured webpage — product listings, job posts, articles — into typed JSON matching a TypeScript interface, JSON schema, or database model you provide.

integrate

Auto-detects your stack (language, framework, AI toolkit) and writes a complete Olostep SDK integration: install commands, client setup, tool wiring, and a verification step.

setup

Configure the Olostep API key and verify the connection. Includes troubleshooting for common setup issues.

​
MCP Tools

The bundled MCP server (olostep-mcp) gives your agent 9 tools it can call directly. Use these when you need fine-grained control beyond what the workflow skills provide.
ToolWhat it does
scrape_websiteScrape a single URL to markdown, HTML, JSON, or text
get_webpage_contentFetch a webpage as clean, LLM-ready markdown
search_webSearch the live web, get AI-synthesized answers
google_searchStructured Google SERP data — organic results, PAA, knowledge graph
answersAsk a question, get a cited answer with optional structured JSON output
batch_scrape_urlsScrape up to 10,000 URLs in parallel with full rendering
create_crawlCrawl a website by following links from a starting URL
create_mapDiscover all URLs on a website, filterable by pattern and query
get_website_urlsFind and retrieve relevant URLs from a specific domain

​
What This Looks Like in Practice

​
”Why is this failing?” — Debug from the live web

You paste ECONNRESET when calling Stripe webhook endpoint into your agent. It searches GitHub issues and StackOverflow for that exact error, scrapes the three most relevant threads, and returns a concrete fix — not “check your network settings,” but the actual timeout configuration that solved it for other developers hitting the same wall.

​
”Write the integration” — Code from current docs, not stale training data

You need to integrate a payment API. The docs-to-code skill scrapes the current API reference — the one published today, not the version your model was trained on six months ago — and writes working code using the parameters and endpoints that actually exist.

​
”Which one should I use?” — Structured tool comparisons

Evaluating ORMs? Comparing auth providers? The research skill searches multiple sources, scrapes real pricing pages and feature matrices, and returns a structured comparison table with citations. You get a recommendation backed by what’s actually on each product’s website, not by training-data popularity.

​
Build a RAG pipeline from any docs site

map   → discover every URL on the docs site
batch → scrape all pages in parallel as clean markdown
      → feed into your vector store
Two skills, one pipeline. Works on documentation sites with hundreds or thousands of pages.

​
Extract structured data at scale

map   → find all product, listing, or job URLs
batch → scrape each page with a pre-built parser → typed JSON
      → pipe into your database, API, or seed files

​
Migrate to a new framework version

You’re upgrading Next.js 13 to 15. The migrate-code skill scrapes the official migration guide, extracts every breaking change with before/after code patterns, and rewrites your files to match — based on the real documentation, not on the model’s best guess.

​
Pre-built Parsers

Pass the parser parameter to any scrape call and get typed JSON back instead of raw content. No schema definition needed — these handle the extraction for you.
ParserReturns
@olostep/google-searchOrganic results, knowledge graph, People Also Ask
@olostep/amazon-it-productPrice, rating, features, availability
@olostep/extract-emailsEvery email address found on the page
@olostep/extract-calendarsStructured calendar events
@olostep/extract-socialsSocial media profile links

Install on ClawHub

Plugin listing, version history, and one-command install

Get API Key

500 free requests/month, no credit card

MCP Server Docs

Full MCP server setup for Cursor, Claude Desktop, and other clients

API Reference

Complete endpoint documentation with examples