ScrapingDog icon

ScrapingDog

Get data from ScrapingDog API

Actions6

Overview

This node integrates with the ScrapingDog API to perform a Google News search based on user-defined criteria. It allows users to query Google News for recent articles matching specific keywords, filter results by country, language, time period, and other parameters, and retrieve a customizable number of news items.

Common scenarios where this node is beneficial include:

  • Monitoring news coverage on specific topics or brands.
  • Gathering recent news articles for content curation or research.
  • Automating news alerts and summaries in workflows.

For example, a user could search for "climate change" news articles limited to the United States, retrieving the top 10 most recent articles published in English within the past week.

Properties

Name Meaning
Query The search term or phrase to query Google News articles (e.g., "climate change").
Results Number of news articles to return, ranging from 1 to 100.
Additional Fields A collection of optional filters and settings:
- Country ISO country code to localize results. Options include US, UK, India, Canada, Australia, Germany, France, Japan, Brazil, or a custom 2-letter code.
- Custom Country Code If "Other" is selected for Country, specify a custom 2-letter ISO country code here (e.g., "mx" for Mexico).
- Page Page number for paginated results; 0 is the first page, 1 is the second, etc.
- Domain Google domain to use for localized results (e.g., "google.co.uk" for the UK).
- Language Language of the news results. Supported languages include English, Spanish, French, German, Italian, Portuguese, Russian, Japanese, Chinese (Simplified and Traditional), Korean, Arabic, Hindi, Dutch, Swedish, Norwegian, Danish, Finnish.
- Language Restriction (lr) Limit search results to specific language(s) using the format lang_{language_code} (e.g., "lang_en").
- Geographic Location (UULE) Parameter specifying geographic location for tailored results, encoded as a string (e.g., "w+CAIQIFJlbGF5IFN0YXRlcw==").
- Time-Based Search (tbs) Filter results by time period: Any time, Past hour, Past 24 hours, Past week, Past month, or Past year.
- Safe Search Filter adult content: Off or Active.
- No Auto-Correct (nfpr) Boolean flag to exclude results from auto-corrected queries (true to exclude, false to include).

Output

The node outputs JSON data containing the search results returned by the ScrapingDog API's Google News endpoint. Each item in the output typically represents a news article with details such as title, snippet, source, publication date, and URL.

If binary data were involved (not indicated here), it would represent downloadable content like images or documents, but this node focuses on JSON news data.

Dependencies

  • Requires an active ScrapingDog API key credential configured in n8n.
  • The node sends HTTP GET requests to the ScrapingDog API endpoint https://api.scrapingdog.com/google_news.
  • Proper internet connectivity and valid API credentials are necessary.

Troubleshooting

  • Invalid API Key or Authentication Errors: Ensure that the API key credential is correctly set up and has sufficient permissions.
  • Empty or No Results Returned: Check the query string for typos or overly restrictive filters (e.g., very narrow country/language/time settings).
  • Rate Limiting or Quota Exceeded: The ScrapingDog API may limit request rates; verify your plan limits and usage.
  • Incorrect Country or Language Codes: Use valid ISO codes and supported language options to avoid errors or unexpected results.
  • Pagination Issues: Remember that page numbering starts at 0; requesting pages beyond available results will return empty sets.

Links and References

Discussion