ScrapingDog icon

ScrapingDog

Get data from ScrapingDog API

Actions6

Overview

This node performs web searches using the Bing Search engine via an external scraping API service. It allows users to query Bing with specific keywords and retrieve search results programmatically. This is useful for automating data collection, market research, SEO analysis, or monitoring online content related to particular topics.

For example, a user can input a keyword like "latest electric cars," specify the country as "United States," and request 20 results from page 2 of the search results. The node will then return structured JSON data containing the search results from Bing for that query.

Properties

Name Meaning
Keyword The search term or phrase to query on Bing.
Page The page number of the search results to retrieve (1 to 10).
Country The country code to localize the search results. Options include Australia, Brazil, Canada, China, France, Germany, India, Mexico, Italy, Japan, Russia, United States, United Kingdom.
Results Number of search results to return. Options range from 10 up to 100 in increments of 10.
Filters Advanced filtering options as a string, e.g., date range filters encoded in a specific format.

Output

The node outputs JSON data representing the Bing search results returned by the scraping API. The structure of the JSON depends on the API response but generally includes details such as titles, snippets, URLs, and other metadata for each search result.

If an error occurs during the API call, the output JSON will contain an error field set to true, along with a descriptive message and HTTP status information if available.

The node does not output binary data.

Dependencies

  • Requires an API key credential for the external scraping service.
  • The node makes HTTP GET requests to the scraping API endpoint dedicated to Bing search.
  • No additional environment variables are required beyond the API key credential configuration.

Troubleshooting

  • Common issues:

    • Invalid or missing API key will cause authentication failures.
    • Network connectivity problems may lead to HTTP errors.
    • Requesting too many results or unsupported page numbers might cause unexpected API responses.
    • Incorrect country codes or filter strings may result in no or incomplete results.

  • Error messages:

    • HTTP error! status: XXX <statusText> indicates an HTTP failure; check network and API key validity.
    • Errors with error: true and a message field indicate issues either from the API or internal processing.

  • Resolution tips:

    • Verify the API key is correctly configured and active.
    • Ensure the query and parameters conform to expected formats.
    • Limit the number of results and pages within supported ranges.
    • Check the scraping service status if persistent errors occur.

Links and References

Discussion