ScrapegraphAI icon

ScrapegraphAI

Consume ScrapegraphAI API

Actions3

Overview

The Search Scraper operation of the ScrapegraphAI node allows users to perform a search query or provide instructions for what to search and extract via the ScrapegraphAI API. This node sends the user’s prompt to the API, which processes the request and returns relevant search results or extracted data.

This operation is beneficial when you want to automate web searches or extract specific information based on natural language queries without manually scraping websites. For example, you could use it to gather summarized data about a topic, find product details, or collect news snippets by simply providing a descriptive search prompt.

Properties

Name Meaning
User Prompt The search query or instructions describing what to search and extract from the web.

Output

The node outputs a JSON object containing the response from the ScrapegraphAI Search Scraper API. This typically includes the search results or extracted content based on the user prompt. The exact structure depends on the API response but generally contains fields with the retrieved data.

No binary data output is indicated in the code.

Dependencies

  • Requires an active API key credential for the ScrapegraphAI service.
  • Makes HTTP POST requests to https://api.scrapegraphai.com/v1/searchscraper.
  • The node expects the API key to be provided in the header as "SGAI-APIKEY".

Troubleshooting

  • Common issues:

    • Invalid or missing API key will cause authentication errors.
    • Malformed or unclear user prompts may result in incomplete or irrelevant search results.
    • Network connectivity problems can cause request failures.

  • Error handling:

    • If the node is set to continue on failure, errors are caught and returned as JSON with an error field containing the error message.
    • Otherwise, the node execution will stop and throw the error.

Links and References

Discussion