Before you start
- An Olostep account with an API key: get one free, no credit card required. Your first 500 credits are included.
- A Snowflake account with external access: the app calls the Olostep API through an external access integration, which trial accounts don’t support.
- A role that can approve app connections: approving the connection to
api.olostep.comneeds theMANAGE APPLICATION SPECIFICATIONSprivilege, whichACCOUNTADMINandSECURITYADMINhave.
Setup
1
Get the app
Open the listing and click Get. The examples on this page assume you install it as
olostep.2
Approve the connection to the Olostep API
The app creates its own external access integration for
api.olostep.com, and can’t call the API until you approve it. In Snowsight, go to Catalog > Apps and select the app. Select the Settings icon, then open the Configurations tab. Under External connections, expand Details next to External access integrations and approve the api.olostep.com endpoint.3
Add your API key
On the same Configurations tab, under Credentials, add a
GENERIC_STRING secret that holds your API key from the Olostep dashboard to Olostep API key.Prefer SQL? Do steps 2 and 3 in a worksheet
Prefer SQL? Do steps 2 and 3 in a worksheet
Procedures
Every procedure returns a
VARIANT. On failure it returns an object with status set to error. Rate limits, server errors and timeouts are retried automatically.
Usage
Save pages as tables
batch_scrape and crawl save one row per page to olostep.results.<table_name>, with the columns URL, MARKDOWN, ERROR and RETRIEVED_AT. Using an existing table name replaces that table.
id with status set to in_progress. Save its pages later with:
results are removed if you uninstall the app, so copy anything you want to keep into your own database.
Troubleshooting
- “Could not reach api.olostep.com”: the connection isn’t approved yet. Approve it as in step 2.
- “Unknown user-defined function INTERNAL.OLOSTEP_REQUEST”: the API key isn’t added yet. Add it as in step 3.
- HTTP 401 in the error: the secret doesn’t hold a valid Olostep API key. Check it in the Olostep dashboard.