Passer au contenu principal

Documentation Index

Fetch the complete documentation index at: https://docs.olostep.com/llms.txt

Use this file to discover all available pages before exploring further.

​
Extracteur de Recherche Google avec Olostep

Ce guide montre comment utiliser l’API Olostep pour extraire les résultats de recherche Google et les convertir en données JSON structurées. Cela est particulièrement utile pour automatiser des tâches de recherche, recueillir des informations concurrentielles ou créer des applications nécessitant des données de recherche.

​
Comment ça fonctionne

L’exemple ci-dessous en Javascript montre comment rechercher l’URL du profil LinkedIn d’une personne spécifique (Patrick Collison) en utilisant la recherche Google et le parseur de recherche Google d’Olostep @olostep/google-search 
async function scrapeGoogleSearch(apiKey, query = "site%3Alinkedin.com+Patrick+Collison") {
  const endpoint = "https://api.olostep.com/v1/scrapes";

  const payload = {
    "formats": ["json"],
    "parser": {"id": "@olostep/google-search"},
    "url_to_scrape": `https://www.google.com/search?q=${encodeURIComponent(query)}&gl=us&hl=en`,
    "wait_before_scraping": 0,
  };

  const headers = {
    "Authorization": `Bearer ${apiKey}`,
    "Content-Type": "application/json"
  };

  try {
    const response = await fetch(endpoint, {
      method: "POST",
      headers: headers,
      body: JSON.stringify(payload)
    });

    const data = await response.json();
    console.log(JSON.stringify(data, null, 4));
    return data;
  } catch (error) {
    console.error("Error:", error);
    throw error;
  }
}

// Remplacez <API_KEY> par votre clé API Olostep réelle
scrapeGoogleSearch("<API_KEY>");

​
Format de Réponse

Lorsque tu fais une requête à l’API Olostep avec le parseur de recherche Google, tu recevras une réponse JSON comme l’exemple ci-dessous : 
{
    "id": "scrape_f2xghz17kt",
    "object": "scrape",
    "created": 1742679301,
    "metadata": {},
    "retrieve_id": "f2xghz17kt",
    "url_to_scrape": "https://www.google.com/search?q=site%253Alinkedin.com%2BPatrick%2BCollison&gl=us&hl=en",
    "result": {
        "html_content": null,
        "markdown_content": null,
        "text_content": null,
        "json_content": "{\"searchParameters\":{\"type\":\"search\",\"engine\":\"google\",\"q\":\"site:linkedin.com Patrick Collison\"},\"knowledgeGraph\":{\"description\":\"Experience. Stripe Graphic · Stripe. -. Education. Massachusetts Institute of Technology Graphic · Massachusetts Institute of Technology. 2006 - 2010 ...\"},\"organic\":[{\"title\":\"Patrick Collison - Stripe\",\"link\":\"https://www.linkedin.com/in/patrickcollison\",\"position\":1,\"snippet\":\"Experience. Stripe Graphic · Stripe. -. Education. Massachusetts Institute of Technology Graphic · Massachusetts Institute of Technology. 2006 - 2010 ...\",\"meta\":\"10.8K+ followers\"},{\"title\":\"The Stripe Story: How Patrick Collison Revolutionized ...\",\"link\":\"https://www.linkedin.com/pulse/stripe-story-how-patrick-collison-revolutionized-online-anshuman-jha-jzzic\",\"position\":2,\"snippet\":\"The Early Years: A Genius in the Making. Patrick Collison wasn't just bright—he was a supernova. By age 10, he'd devoured university-level math ...\"},{\"title\":\"In 2005, Patrick Collison was a 16-year-old winning ...\",\"link\":\"https://www.linkedin.com/posts/itselanagold_in-2005-patrick-collison-was-a-16-year-old-activity-7308533537576497154-w5vC\",\"position\":3,\"snippet\":\"In 2005, Patrick Collison was a 16-year-old winning Ireland's Young Scientist of the Year competition. By 2008, he and his younger brother ...\"},{\"title\":\"Patrick Collison on the importance of waiting a really long ...\",\"link\":\"https://www.linkedin.com/posts/the-startup-archive_patrick-collison-on-the-importance-of-waiting-activity-7286001819145707520-1mdI\",\"position\":4,\"snippet\":\"Patrick argues you should also view every person you hire as bringing along another 50 people just like them if your company is successful.\"},{\"title\":\"Tim Ferriss' Post - Patrick Collison — CEO of Stripe (#353)\",\"link\":\"https://www.linkedin.com/posts/timferriss_patrick-collison-ceo-of-stripe-353-activity-7271892372358148096--dsK\",\"position\":5,\"snippet\":\"Author of 5 #1 NYT/WSJ bestsellers, early-stage investor, host of The Tim Ferriss Show podcast (1B+ downloads), and collector of the strange.\"},{\"title\":\"Patrick Collison wanted a guide to Stripe's culture that ...\",\"link\":\"https://www.linkedin.com/posts/first-round-capital_patrick-collison-wanted-a-guide-to-stripes-activity-7304833456948097024-Tt6h\",\"position\":6,\"snippet\":\"Patrick Collison wanted a guide to Stripe's culture that convinced 50% of candidates not to join. And Eeke de Milliano was tasked with ...\"},{\"title\":\"The Collison brothers (John & Patrick) explain why Stripe is ...\",\"link\":\"https://www.linkedin.com/posts/marcelvanoost_the-collison-brothers-john-patrick-explain-activity-7301586346349850624-L-4U\",\"position\":7,\"snippet\":\"The Collison brothers (John & Patrick) explain why Stripe is better off staying Private: \\\" This is our life's work. We're not going anywhere ...\"},{\"title\":\"Stripe CEO Patrick Collison on Crafting a Culture ...\",\"link\":\"https://www.linkedin.com/posts/jennifer-chatman-8086a918_stripe-ceo-patrick-collison-on-crafting-a-activity-7231753022849085440-0RE5\",\"position\":8,\"snippet\":\"When Patrick Collison and his brother John Collison founded digital payment company Stripe in 2010, he didn't come in with “any kind of ...\"},{\"title\":\"Patrick Collison on the importance of beauty and ...\",\"link\":\"https://www.linkedin.com/posts/the-startup-archive_patrick-collison-on-the-importance-of-beauty-activity-7247935993817751552-Qt6h\",\"position\":9,\"snippet\":\"Patrick Collison on the importance of beauty and craftsmanship when building products “If Stripe is a monstrously successful business, ...\"}]}",
        "llm_extract": null,
        "screenshot_hosted_url": null,
        "html_hosted_url": null,
        "markdown_hosted_url": null,
        "json_hosted_url": "https://olostep-storage.s3.us-east-1.amazonaws.com/json_f2xghz17kt.json",
        "text_hosted_url": null,
        "links_on_page": [],
        "page_metadata": {
            "status_code": 200,
            "title": ""
        }
    }
}
La réponse contient :
  • Informations de base sur la requête : id, object, horodatage created, url_to_scrape
  • Objet résultat avec des URLs pour accéder à différents formats de données
  • json_content avec des résultats de recherche structurés incluant :
  • searchParameters : Informations sur la requête de recherche
  • knowledgeGraph : Informations détaillées sur le sujet de recherche (lorsque disponible)
  • organic : Liste des résultats de recherche avec titre, lien, position et extrait
  • peopleAlsoAsk : Questions connexes que les utilisateurs recherchent couramment
  • relatedSearches : Requêtes de recherche suggérées connexes
json_content est la partie principale de la réponse avec les résultats de recherche structurés. Tu peux accéder directement au contenu JSON depuis la réponse ou utiliser l’URL hébergée fournie dans la réponse.

​
Réponse Structurée : json_content

{
  "searchParameters": {
    "type": "search",
    "engine": "google",
    "q": "site:linkedin.com Patrick Collison"
  },
  "knowledgeGraph": {
    "description": "Experience. Stripe Graphic · Stripe. -. Education. Massachusetts Institute of Technology Graphic · Massachusetts Institute of Technology. 2006 - 2010 ..."
  },
  "organic": [
    {
      "title": "Patrick Collison - Stripe",
      "link": "https://www.linkedin.com/in/patrickcollison",
      "position": 1,
      "snippet": "Experience. Stripe Graphic · Stripe. -. Education. Massachusetts Institute of Technology Graphic · Massachusetts Institute of Technology. 2006 - 2010 ...",
      "meta": "10.8K+ followers"
    },
    {
      "title": "The Stripe Story: How Patrick Collison Revolutionized ...",
      "link": "https://www.linkedin.com/pulse/stripe-story-how-patrick-collison-revolutionized-online-anshuman-jha-jzzic",
      "position": 2,
      "snippet": "The Early Years: A Genius in the Making. Patrick Collison wasn't just bright—he was a supernova. By age 10, he'd devoured university-level math ..."
    },
    {
      "title": "In 2005, Patrick Collison was a 16-year-old winning ...",
      "link": "https://www.linkedin.com/posts/itselanagold_in-2005-patrick-collison-was-a-16-year-old-activity-7308533537576497154-w5vC",
      "position": 3,
      "snippet": "In 2005, Patrick Collison was a 16-year-old winning Ireland's Young Scientist of the Year competition. By 2008, he and his younger brother ..."
    },
    {
      "title": "The Collison brothers (John & Patrick) explain why Stripe is ...",
      "link": "https://www.linkedin.com/posts/marcelvanoost_the-collison-brothers-john-patrick-explain-activity-7301586346349850624-L-4U",
      "position": 4,
      "snippet": "The Collison brothers (John & Patrick) explain why Stripe is better off staying Private: \" This is our life's work. We're not going anywhere ..."
    },
    {
      "title": "Patrick Collison on the importance of waiting a really long ...",
      "link": "https://www.linkedin.com/posts/the-startup-archive_patrick-collison-on-the-importance-of-waiting-activity-7286001819145707520-1mdI",
      "position": 5,
      "snippet": "Patrick argues you should also view every person you hire as bringing along another 50 people just like them if your company is successful."
    },
    {
      "title": "Tim Ferriss' Post - Patrick Collison — CEO of Stripe (#353)",
      "link": "https://www.linkedin.com/posts/timferriss_patrick-collison-ceo-of-stripe-353-activity-7271892372358148096--dsK",
      "position": 6,
      "snippet": "Author of 5 #1 NYT/WSJ bestsellers, early-stage investor, host of The Tim Ferriss Show podcast (1B+ downloads), and collector of the strange."
    },
    {
      "title": "Patrick Collison on the importance of beauty and ...",
      "link": "https://www.linkedin.com/posts/the-startup-archive_patrick-collison-on-the-importance-of-beauty-activity-7247935993817751552-Qt6h",
      "position": 7,
      "snippet": "Patrick Collison on the importance of beauty and craftsmanship when building products "If Stripe is a monstrously successful business, ..."
    },
    {
      "title": "Stripe founder Patrick Collison tells the story of almost ...",
      "link": "https://www.linkedin.com/posts/the-startup-archive_stripe-founder-patrick-collison-tells-the-activity-7235977194211000321-V-Cd",
      "position": 8,
      "snippet": "Stripe founder Patrick Collison tells the story of almost naming the company PayDemon Patrick and John Collison founded Stripe in 2010 when ..."
    },
    {
      "title": "Patrick Collison created $50 billion of value as a co- ...",
      "link": "https://www.linkedin.com/posts/tom-alder_patrick-collison-created-50-billion-of-value-activity-7239241304780513281-isxK",
      "position": 9,
      "snippet": "Patrick Collison created $50 billion of value as a co-founder of Stripe. He has also built the largest carbon removal program in the world."
    }
  ]
}
Olostep fournit également un fichier JSON hébergé avec les résultats de recherche structurés. Tu peux accéder au fichier JSON en utilisant le champ json_hosted_url dans la réponse : Si tu souhaites également obtenir le contenu HTML et Markdown des résultats de recherche, tu peux inclure ces formats dans le paramètre formats et Olostep les renverra dans la réponse et fournira des URLs hébergées pour chaque format.

​
Scénarios d’Utilisation Exemple

​
1. Trouver des Profils LinkedIn

Dans l’exemple ci-dessus, nous recherchons le profil LinkedIn de Patrick Collison en utilisant la requête de recherche site:linkedin.com Patrick Collison. Cette technique peut être utilisée pour trouver des profils professionnels pour n’importe quelle personne.

​
2. Rechercher des Entreprises

Tu peux modifier la requête pour rechercher des informations sur une entreprise : 
// Rechercher des informations sur une entreprise
scrapeGoogleSearch(apiKey, "Stripe company information revenue");

​
3. Suivre des Articles de Presse

Trouver des nouvelles récentes sur un sujet spécifique : 
// Rechercher des nouvelles récentes sur l'IA
scrapeGoogleSearch(apiKey, "artificial intelligence news after:2023-01-01");

​
4. Analyse Concurrentielle

Surveiller la présence en ligne des concurrents : 
// Rechercher des mentions de concurrents
scrapeGoogleSearch(apiKey, "\"Company X\" vs \"Company Y\" comparison");

​
Notes Importantes

  1. Paramètres de Recherche : Les paramètres gl=us et hl=en définissent la géolocalisation aux États-Unis et la langue en anglais. Ajuste-les selon tes besoins.

​
Conclusion

Une fois que tu as les données des résultats de recherche, tu peux :
  1. Analyser des points de données spécifiques d’intérêt
  2. Stocker les résultats dans une base de données
  3. Construire une interface de recherche personnalisée
  4. Déclencher des actions basées sur les résultats de recherche
  5. Intégrer avec d’autres APIs ou services
Si tu as besoin d’extraire différents points de données ou de personnaliser le comportement de la recherche, n’hésite pas à nous contacter à ‘info@olostep.com`