Skip to main content
The Olostep Native App on the Snowflake Marketplace brings live web data into Snowflake. Olostep is web data infrastructure for AI agents and enterprise AI workflows. Its APIs provide live web search, scraping and crawling to extract, clean and structure data from the web. Call Olostep from SQL to search the web, scrape pages into clean markdown, map and crawl websites, and get answers with sources. Batch scrapes and crawls are saved as tables in your account, so you can join web data with your own data, load it into Cortex Search or use it in Cortex Agents. View on the Snowflake Marketplace →

Before you start

  • An Olostep account with an API key: get one free, no credit card required. Your first 500 credits are included.
  • A Snowflake account with external access: the app calls the Olostep API through an external access integration, which trial accounts don’t support.
  • A role that can approve app connections: approving the connection to api.olostep.com needs the MANAGE APPLICATION SPECIFICATIONS privilege, which ACCOUNTADMIN and SECURITYADMIN have.

Setup

1

Get the app

Open the listing and click Get. The examples on this page assume you install it as olostep.
2

Approve the connection to the Olostep API

The app creates its own external access integration for api.olostep.com, and can’t call the API until you approve it. In Snowsight, go to Catalog > Apps and select the app. Select the Settings icon, then open the Configurations tab. Under External connections, expand Details next to External access integrations and approve the api.olostep.com endpoint.
3

Add your API key

On the same Configurations tab, under Credentials, add a GENERIC_STRING secret that holds your API key from the Olostep dashboard to Olostep API key.

Procedures

Every procedure returns a VARIANT. On failure it returns an object with status set to error. Rate limits, server errors and timeouts are retried automatically.

Usage

Save pages as tables

batch_scrape and crawl save one row per page to olostep.results.<table_name>, with the columns URL, MARKDOWN, ERROR and RETRIEVED_AT. Using an existing table name replaces that table.
Batches and crawls usually finish in a few minutes. If one is still running after 20 minutes, the procedure returns its id with status set to in_progress. Save its pages later with:
Tables in results are removed if you uninstall the app, so copy anything you want to keep into your own database.

Troubleshooting

  • “Could not reach api.olostep.com”: the connection isn’t approved yet. Approve it as in step 2.
  • “Unknown user-defined function INTERNAL.OLOSTEP_REQUEST”: the API key isn’t added yet. Add it as in step 3.
  • HTTP 401 in the error: the secret doesn’t hold a valid Olostep API key. Check it in the Olostep dashboard.

Resources