Scrapfly

Scrapfly data collection APIs for web page scraping, screenshots, and AI data extraction

Actions4

Scrape Actions
- Scrape Web Page
- Scrape API Request
Extraction Actions
- Extract Data From an HTML, Text, or Markdown Document Using AI
Screenshot Actions
- Capture Web Page Screenshot

Overview

This node integrates with Scrapfly's data collection APIs to perform web page screenshots, scraping, data extraction, and account management. Specifically, the "Capture Web Page Screenshot" operation under the "Screenshot" resource allows users to capture images of web pages programmatically.

Common scenarios for this node include:

Automatically generating visual snapshots of websites for monitoring or archival purposes.
Capturing screenshots of product pages, news articles, or dashboards for reporting.
Creating thumbnails or previews of URLs in automated workflows.
Testing website appearance across different resolutions or geolocations.

For example, a user can input a URL and specify options like image format (PNG, JPG), resolution, or whether to capture the full page or just the viewport. The node then returns the screenshot image data accordingly.

Properties

Name	Meaning
URL	The web page URL to capture a screenshot from.
Additional Fields	A collection of optional parameters to customize the screenshot:
- File Name	Custom file name for the screenshot without extension; defaults to using the URL if not set.
- Format	Image format of the screenshot. Options: JPG, PNG, WEBP, GIF.
- Capture	Area of the page to capture. Defaults to "viewport". Can be "full page", a specific element via XPath or CSS selector, or a vertical part of the page.
- Resolution	Screen resolution for the screenshot in width x height format (e.g., 1950x1080).
- Country	Proxy geolocation country code to simulate browsing from a specific country (default is "us").
- Timeout	Maximum time allowed for the screenshot request in milliseconds (default 30000 ms). Controls how long Scrapfly tries to perform the scrape before timing out.
- Rendering Wait	Time in milliseconds for the headless browser to wait after loading the page before taking the screenshot (default 1000 ms).
- Wait For Selector	XPath or CSS selector that the browser waits for before capturing the screenshot, ensuring the element is loaded.
- Options	Additional screenshot behavior options. Multiple can be selected: Dark Mode, Block Banners, Print Media Format.
- Auto Scroll	Boolean flag to automatically scroll down the page before capturing, useful for lazy-loaded content (default false).
- JavaScript Injection	Custom JavaScript code to execute in the page context before taking the screenshot.
- Cache	Enable caching of the screenshot result to avoid repeated scrapes if the content hasn't changed (default false).
- Cache TTL	Time-to-live for the cache in seconds. After expiration, the screenshot will be refreshed (default 86400 seconds = 24 hours).
- Cache Clear	Force refresh the cache and perform a new scrape regardless of existing cached data (default false).

Output

The node outputs an array of JSON objects, each representing the result of a screenshot capture. The main output field json contains metadata about the screenshot, such as the URL, status, and possibly the image data or a link to the stored screenshot.

If binary data is included, it represents the actual screenshot image in the specified format (JPG, PNG, WEBP, or GIF).

Dependencies

Requires an active Scrapfly API key credential configured in n8n to authenticate requests.
Depends on Scrapfly’s external web scraping and screenshot service.
No additional environment variables are needed beyond the API key credential.

Troubleshooting

Timeouts: If the screenshot operation times out, consider increasing the Timeout property or adjusting Rendering Wait and Wait For Selector to ensure the page fully loads.
Invalid Selectors: Using incorrect XPath or CSS selectors in Capture or Wait For Selector may cause failures or empty screenshots.
Cache Issues: Enabling cache without proper TTL or clearing might return stale screenshots. Use Cache Clear to force refresh.
Unsupported URLs: Some websites with heavy anti-bot measures or requiring authentication may fail to render correctly.
Format Errors: Ensure the chosen image format is supported by your downstream workflow or application.