ScrapingDog icon

ScrapingDog

Get data from ScrapingDog API

Actions6

Overview

This node performs Google Search queries using an external scraping API service. It allows users to search for keywords on Google and retrieve search results programmatically within n8n workflows. The node supports advanced search features such as fetching featured snippets, ads, People Also Ask (PAA) sections, Knowledge Graphs, and more.

Common scenarios where this node is beneficial include:

  • Automating market research by extracting search result data.
  • Monitoring keyword rankings or competitor presence on Google.
  • Gathering data for SEO analysis or content planning.
  • Integrating Google search results into custom dashboards or reports.

For example, a user can input a keyword like "best laptops 2024" and retrieve the top 10 Google search results along with advanced snippets if enabled.

Properties

Name Meaning
Keyword The search term or phrase to query on Google.
Advance Search Boolean option to enable advanced featured snippets including Ads, People Also Ask, Knowledge Graphs, etc.
Page The page number of search results to retrieve (1 to 10).
Location The geographic location to tailor the search results (e.g., United States, Germany, India).
Results Number of search results to return (from 10 up to 100 in increments of 10).

Output

The node outputs JSON data representing the search results returned by the scraping API. The structure depends on the API response but generally includes:

  • Search result items such as titles, URLs, snippets.
  • If advanced search is enabled, additional fields like ads, knowledge graph data, and PAA questions may be included.
  • Metadata about the request such as status codes.

No binary data output is produced by this node.

Example simplified output snippet:

{
  "results": [
    {
      "title": "Example Result Title",
      "link": "https://example.com",
      "snippet": "Short description or excerpt from the page."
    }
  ],
  "ads": [...],
  "knowledgeGraph": {...},
  "peopleAlsoAsk": [...]
}

Dependencies

  • Requires an API key credential for the external scraping service.
  • The node makes HTTP GET requests to the scraping API endpoint.
  • No additional environment variables are needed beyond the API key configuration.

Troubleshooting

  • HTTP errors: If the API returns non-200 status codes, the node throws an error with the HTTP status and message. Check that the API key is valid and has sufficient quota.
  • Empty or incomplete results: Could be due to incorrect parameters such as unsupported location codes or invalid page numbers.
  • Network issues: Ensure the n8n instance has internet access to reach the scraping API.
  • API limits: Exceeding rate limits or quotas on the scraping service will cause errors; monitor usage accordingly.

Links and References

  • Google Search
  • Documentation for the external scraping API (not provided here, check your API provider's docs)

Discussion