Scrapes
Create Scrape
Scrape a url with provided configuration and get content.
POST
/
v1
/
scrapes
Copy
curl --request POST \
--url https://api.olostep.com/v1/scrapes \
--header 'Authorization: Bearer <token>' \
--header 'Content-Type: application/json' \
--data '{
"url_to_scrape": "<string>",
"wait_before_scraping": 123,
"formats": [
"html"
],
"remove_css_selectors": "default",
"actions": [
{
"type": "wait",
"milliseconds": 1
}
],
"country": "<string>",
"transformer": "postlight",
"remove_images": false,
"remove_class_names": [
"<string>"
],
"parser": {
"id": "<string>"
},
"llm_extract": {
"schema": {}
},
"links_on_page": {
"absolute_links": true,
"query_to_order_links_by": "<string>",
"include_links": [
"<string>"
],
"exclude_links": [
"<string>"
]
},
"screen_size": {
"screen_type": "default",
"screen_width": 123,
"screen_height": 123
},
"metadata": {}
}'
Copy
{
"id": "<string>",
"object": "<string>",
"created": 123,
"metadata": {},
"url_to_scrape": "<string>",
"result": {
"html_content": "<string>",
"markdown_content": "<string>",
"text_content": "<string>",
"json_content": "<string>",
"screenshot_hosted_url": "<string>",
"html_hosted_url": "<string>",
"markdown_hosted_url": "<string>",
"text_hosted_url": "<string>",
"links_on_page": [
"<string>"
],
"page_metadata": {
"status_code": 123,
"title": "<string>"
}
}
}
Authorizations
Bearer authentication header of the form Bearer <token>
, where <token>
is your auth token.
Body
application/json
Response
200
application/json
Successful response with the scrape initiation details.
The response is of type object
.
Copy
curl --request POST \
--url https://api.olostep.com/v1/scrapes \
--header 'Authorization: Bearer <token>' \
--header 'Content-Type: application/json' \
--data '{
"url_to_scrape": "<string>",
"wait_before_scraping": 123,
"formats": [
"html"
],
"remove_css_selectors": "default",
"actions": [
{
"type": "wait",
"milliseconds": 1
}
],
"country": "<string>",
"transformer": "postlight",
"remove_images": false,
"remove_class_names": [
"<string>"
],
"parser": {
"id": "<string>"
},
"llm_extract": {
"schema": {}
},
"links_on_page": {
"absolute_links": true,
"query_to_order_links_by": "<string>",
"include_links": [
"<string>"
],
"exclude_links": [
"<string>"
]
},
"screen_size": {
"screen_type": "default",
"screen_width": 123,
"screen_height": 123
},
"metadata": {}
}'
Copy
{
"id": "<string>",
"object": "<string>",
"created": 123,
"metadata": {},
"url_to_scrape": "<string>",
"result": {
"html_content": "<string>",
"markdown_content": "<string>",
"text_content": "<string>",
"json_content": "<string>",
"screenshot_hosted_url": "<string>",
"html_hosted_url": "<string>",
"markdown_hosted_url": "<string>",
"text_hosted_url": "<string>",
"links_on_page": [
"<string>"
],
"page_metadata": {
"status_code": 123,
"title": "<string>"
}
}
}
Assistant
Responses are generated using AI and may contain mistakes.