Overview
This node integrates with the ScraperCity API to run web scrapers, check the status of scraping jobs, and download the results. It is useful for automating data extraction from various online services such as Apollo, Yelp, Zillow Agents, and more. For example, users can start a scrape job on a selected service by providing a JSON payload, then monitor the job status, and finally download the scraped data in CSV or JSON format.
Use Case Examples
- Start a scrape job on the Apollo service by sending a JSON payload with search parameters.
- Check the status of a previously started scrape job using its run ID.
- Download the results of a completed scrape job in CSV or JSON format.
Properties
| Name | Meaning |
|---|---|
| Service | Selects the specific scraping service to use, such as Apollo, Yelp, or Zillow Agents. |
| Payload (JSON) | A free-form JSON object containing the request body to send to the scraper endpoint. This defines the parameters for the scrape job. |
Output
Binary
Outputs scraped data as a CSV file when the download format is CSV.
JSON
json- Contains the JSON response from the ScraperCity API for scrape start and status operations, or the JSON data when downloading results in JSON format.binary- Contains the CSV file data when downloading results in CSV format, including file name and MIME type.
Dependencies
- Requires an API key credential for ScraperCity API authentication.
Troubleshooting
- Common issues include missing or invalid API credentials, resulting in authentication errors.
- Errors in the JSON payload format can cause the scrape job to fail; ensure the payload matches the expected structure for the selected service.
- Network or API endpoint issues may cause HTTP request failures; verify the base URL and network connectivity.
Links
- ScraperCity API Documentation - Official API documentation for ScraperCity, detailing endpoints and usage.