Olostep CLI

NPM package: olostep-cli Repository: github.com/olostep-api/olostep-cli CLI for the Olostep API — scrape, search, map, crawl, answer, and batch the web from your terminal. Every command returns JSON so it pipes cleanly into jq, agents, and CI. Pure JavaScript, Node 18+, no native binaries to download. Installs in under a second, starts in ~200 ms, ships as a single ~100 KB bundle.

Install

Requirements: Node.js 18+.

npm (recommended)
One-liner (macOS / Linux)
One-liner (Windows)
No-install (npx)

npm install -g olostep-cli
olostep init

curl -fsSL https://olostep.com/install.sh | sh
olostep init

The script checks Node 18+, runs npm install -g olostep-cli, and falls back to sudo if needed.

iwr -useb https://olostep.com/install.ps1 | iex
olostep init

Same idea as the macOS/Linux script, but for PowerShell.

npx -y olostep-cli@latest --help

Good for trying a single command without a global install.

olostep init is the recommended next step — it signs you in, installs the Olostep skills into your AI agents, and configures the MCP server, all in one command. The one-liner scripts wrap npm install -g olostep-cli with a Node 18+ check and a sudo fallback, so they work even if you’re unsure of your local setup. Platforms: macOS (Apple Silicon and Intel), Linux (x64 and arm64), Windows (x64 and arm64).

Set up

One command does everything — sign in, install skills, and install the MCP server:

olostep init

Flags: --skills-only, --mcp-only, --no-browser, --relogin. To just sign in (no skills/MCP):

olostep login
olostep login --no-browser     # print the URL (useful over SSH)

The browser opens to the Olostep auth page; you click Authorize, and the CLI saves your key locally. Alternative — set an env var. Good for CI:

export OLOSTEP_API_KEY=your_key_here

Get a key from the API Keys dashboard. Where the key is stored (after olostep login):

OS	Path
macOS	`~/Library/Application Support/olostep-cli/credentials.json`
Linux	`~/.config/olostep-cli/credentials.json`
Windows	`%USERPROFILE%\AppData\Roaming\olostep-cli\credentials.json`

Sign out

olostep logout            # prompts to confirm, then removes credentials.json
olostep logout --dry-run  # preview only — see what would happen
olostep logout --yes      # skip the confirmation (for scripts)
olostep logout --json     # machine-readable output

logout also warns you if OLOSTEP_API_KEY / OLOSTEP_API_TOKEN env vars or a .env file in your current directory still hold a key — those take priority over the credentials file, so deleting the file alone may not be enough. The output includes the exact unset commands for PowerShell and bash/zsh.

Quick start

olostep login

olostep search "best web scraping APIs 2025" --limit 5
olostep answer "What does Olostep do?"
olostep map "https://example.com" --top-n 20
olostep scrape "https://example.com" --formats markdown
olostep crawl "https://docs.example.com" --max-pages 50
olostep batch-scrape urls.csv --formats markdown,html

Every command prints its JSON result to stdout by default. Pass --out <path> to save to a file.

What can it do?

You want to…	Command	Olostep product
Search the web	`search`	Searches
Get a researched answer	`answer`	Answers
Discover URLs on a site	`map`	Maps
Pull one page	`scrape`	Scrapes
Pull every page on a site	`crawl`	Crawls
Pull many URLs from a CSV	`batch-scrape`	Batches
Extract structured fields	`--parser-id` on `batch-scrape`	Parsers
Refetch a result by ID	`scrape-get`	Scrapes
Tag/organize a batch	`batch-update`	Batches

Output

Every command prints its JSON result to stdout by default.

Flag	Behavior
(none)	Print JSON to stdout (UTF-8, indented)
`--out <path>`	Write JSON to that file instead
`--out -`	Explicitly stdout (same as default)

Progress and log lines go to stderr, so stdout stays clean for pipes.

olostep map "https://example.com" --top-n 20 | jq '.urls[:10]'
olostep scrape "https://example.com" | jq .result.markdown_content
olostep search "topic" --json | jq '.links[].url'

Choosing between them:

search — you want a list of relevant URLs and snippets for a query. The CLI searches the web for you.
answer — you want a synthesized answer, not raw page content. The CLI does the research for you.
scrape — you already have the URL and want clean content out.
crawl — you want every page on a site (or a filtered subset) without enumerating URLs by hand.
batch-scrape — you have a list of URLs and want them processed in parallel.

Commands

Use olostep <command> --help for every option.

`search`: live web search

Returns deduplicated organic links (URL, title, description).

Option	Description
`--limit`	Number of results, default 12, max 25
`--include-domains`	Comma-separated domains to restrict results to
`--exclude-domains`	Comma-separated domains to exclude
`--out`	File or `-`
`--json`	Machine-readable output

olostep search "TypeScript CLI tools" --limit 10
olostep search "open source projects" --include-domains "github.com" --limit 5
olostep search "AI agents" --json | jq '.links[].url'

`answer`: researched answer

Synchronous — returns when the answer is ready.

Option	Description
`--out`	File or `-`
`--json-format`	Optional JSON shape for structured output

olostep answer "What does this company build?" --out answer.json
olostep answer "Extract facts" --json-format '{"company":"","year":""}' --out -

`map`: discover URLs

Option	Description
`--out`	File path or `-`
`--top-n`	Max URLs to return
`--search-query`	Optional query to guide discovery
`--include-subdomain` / `--no-include-subdomain`	Subdomains
`--include-url` / `--exclude-url`	Repeatable URL patterns
`--cursor`	Pagination cursor

olostep map "https://example.com" --top-n 100 --search-query "blog"

`scrape`: one URL

Formats: html, markdown, text, json, raw_pdf, screenshot (comma-separated; default markdown).

Option	Description
`--formats`	Comma-separated
`--country`	Country code (e.g. `US`, `GB`)
`--wait-before-scraping`	Wait before scrape (ms)
`--payload-json` / `--payload-file`	Advanced options as JSON (e.g. `"max_age": 86400` to opt into caching — see Caching)

olostep scrape "https://example.com" --formats markdown,html
olostep scrape "https://example.com" --payload-file options.json --out -

`scrape-get`: fetch by ID

olostep scrape-get "scrape_abc123" --out -

`crawl`: whole site

Starts a crawl, polls until finished, then retrieves page contents. Retrieve formats: markdown, html, json. Notable flags: --max-pages, --max-depth, --include-subdomain, --include-external, --include-url, --exclude-url, --search-query, --top-n, --webhook, --crawl-timeout, --formats, --pages-limit, --pages-search-query, --poll-seconds, --poll-timeout, --dry-run.

olostep crawl "https://docs.example.com" --max-pages 50 --formats markdown,html
olostep crawl "https://example.com" --max-pages 10 --dry-run

`batch-scrape`: CSV

CSV must have a header row with custom_id (or id) and url columns.

custom_id,url
example,https://example.com
iana,https://iana.org
docs,https://docs.olostep.com

Option	Description
`--formats`	`markdown`, `html`, `json` (comma-separated)
`--country`	Optional country code
`--parser-id`	Parser ID for structured extraction
`--poll-seconds`, `--log-every`, `--items-limit`	Polling and paging
`--dry-run`	Print payload and exit

olostep batch-scrape urls.csv --formats markdown,html
olostep batch-scrape urls.csv --parser-id "<PARSER_ID>" --out results.json

Synchronous — polls until the batch completes, then retrieves every item.

`batch-update`: batch metadata

Requires one of --metadata-json or --metadata-file (JSON object).

olostep batch-update "batch_abc123" --metadata-json '{"team":"growth"}'
olostep batch-update "batch_abc123" --metadata-file meta.json

Auth commands

olostep login                      # browser PKCE sign-in
olostep logout                     # remove saved credentials
olostep status                     # show auth state, config paths, version

olostep auth login                 # same as olostep login
olostep auth logout                # same as olostep logout
olostep auth status                # same as olostep status
olostep auth set-key <key>         # save an API key directly (no browser)

auth set-key is useful for CI and scripts — write the key directly without going through the browser flow.

Install the MCP server

The CLI writes the Olostep MCP server into your agent’s config — no JSON editing.

olostep mcp install                          # detect agents, hosted endpoint
olostep mcp install --agent cursor           # only Cursor
olostep mcp install --transport stdio        # local npx instead of hosted
olostep mcp install --no-global              # write into current project
olostep mcp install --dry-run --json         # plan only
olostep mcp uninstall                        # remove the olostep entry
olostep list mcp                             # show which agents have it

Option	Description
`--agent`	Specific agent, repeatable. Supported: `cursor`, `claude`, `claude-desktop`, `windsurf`, `vscode`, `kilo`, `opencode`, `continue`, `codex`
`--all-agents` / `--no-all-agents`	Target every detected agent (default)
`--transport`	`http` (hosted, recommended) or `stdio` (local `npx olostep-mcp`)
`--global` / `--no-global`	Per-user config (default) vs project-local
`--api-key`	Key to embed; defaults to resolved credentials
`--dry-run`	Show the plan without writing
`--json`	Machine-readable output

The hosted endpoint at https://mcp.olostep.com/mcp uses Authorization: Bearer <key> — no local Node process required. The CLI merges only the olostep key into your existing config. Restart your agent after install.

Skills for AI agents

The CLI ships 13 Olostep skills — SKILL.md files installed into Claude Code, Cursor, and other agents so they know what Olostep can do and when to use it.

olostep add skills                     # install all into every detected agent
olostep skills install                 # same (alias)
olostep skills update                  # re-install / refresh all skills
olostep skills list                    # see what's installed and where
olostep skills uninstall               # remove all skills

Filter what gets installed:

olostep add skills --category usage    # core web-data skills only
olostep add skills --skill scrape --skill map
olostep add skills --agent cursor --agent claude

See Skills for the full list and options.

Health checks

olostep doctor                         # run all checks
olostep doctor --skip-network          # auth + config only, no HTTP calls
olostep doctor --json                  # NDJSON — one record per check (good for CI)
olostep doctor --fail-on-warn          # exit 1 on warnings too

Checks: API key present, API key reachable, MCP endpoint reachable, config file exists for each detected agent. CI usage:

olostep doctor --json --skip-network | jq 'select(.status == "fail")'

Version & updates

olostep version                        # CLI version, Node version, channel
olostep version --json                 # machine-readable: { cli, node, channel }
olostep update                         # update to latest (npm install -g olostep-cli@latest)
olostep update --check                 # check whether a newer version is available, don't install

Environment variables

Variable	Effect
`OLOSTEP_API_KEY`	API key
`OLOSTEP_API_TOKEN`	API key (legacy alias)
`OLOSTEP_JSON=1`	Force JSON output on every command (same as `--json` globally)
`OLOSTEP_NO_UPDATE_CHECK=1`	Silence the “update available” notice
`OLOSTEP_CLI_CONFIG_DIR`	Override the credentials directory

Windows / PowerShell notes

PowerShell tokenizes , and * differently from bash — quote arguments:

olostep scrape "https://example.com" --formats "markdown,html"
olostep map   "https://example.com" --include-url "/*"
olostep answer "Extract facts" --json-format '{"company":"","year":""}'

Single quotes are safest for JSON values (no $ interpolation).

See what’s installed

olostep list skills    # installed Olostep skills and which agents have them
olostep list mcp       # which agents have the Olostep MCP server, and the transport

Global flags

Flag	Description
`-V`, `--version`	Version
`-h`, `--help`	Help

--out, --timeout, and --api-key are available on every data command.

Security

Keep API keys out of source control; rotate if leaked. olostep logout removes the local credentials file and tells you if any env-var sources still hold a key.

Maps · Crawls · Batches · Answers · Searches · Skills · MCP Server

Get Started

Features

Balance & Billing

Install

Set up

Sign out

Quick start

What can it do?

Output

Commands

`search`: live web search

`answer`: researched answer

`map`: discover URLs

`scrape`: one URL

`scrape-get`: fetch by ID

`crawl`: whole site

`batch-scrape`: CSV

`batch-update`: batch metadata

Auth commands

Install the MCP server

Skills for AI agents

Health checks

Version & updates

Environment variables

Windows / PowerShell notes

See what’s installed

Global flags

Security

​Install

​Set up

​Sign out

​Quick start

​What can it do?

​Output

​Commands

​search: live web search

​answer: researched answer

​map: discover URLs

​scrape: one URL

​scrape-get: fetch by ID

​crawl: whole site

​batch-scrape: CSV

​batch-update: batch metadata

​Auth commands

​Install the MCP server

​Skills for AI agents

​Health checks

​Version & updates

​Environment variables

​Windows / PowerShell notes

​See what’s installed

​Global flags

​Security

​Related

Install

Set up

Sign out

Quick start

What can it do?

Output

Commands

`search`: live web search

`answer`: researched answer

`map`: discover URLs

`scrape`: one URL

`scrape-get`: fetch by ID

`crawl`: whole site

`batch-scrape`: CSV

`batch-update`: batch metadata

Auth commands

Install the MCP server

Skills for AI agents

Health checks

Version & updates

Environment variables

Windows / PowerShell notes

See what’s installed

Global flags

Security

Related