Actions8
Overview
The node "FireCrawl" provides functionality to map a website and retrieve URLs based on specified criteria. It is useful for scenarios where you want to crawl or scan a website to extract links that match certain search terms or patterns. For example, it can be used to gather all documentation pages from a developer site or collect URLs related to specific topics for further processing.
In the "Map A Website And Get Urls" operation, the node sends a request to an external API with a target URL and an optional search term to filter the URLs returned by the crawl.
Properties
| Name | Meaning |
|---|---|
| Url | The URL of the website to map. This is the starting point for the website crawling. |
| Search | A search term to filter specific URLs found during the crawl. Only URLs containing this term will be returned. |
| Use Custom Body | Boolean flag indicating whether to send a custom JSON body instead of the default parameters. |
| Custom Body | A JSON object representing a custom request body to send to the API. Used only if "Use Custom Body" is true. |
Output
The node outputs JSON data containing the results of the website mapping operation. This typically includes a list of URLs discovered on the target website that match the search criteria. The exact structure depends on the API response but generally consists of an array of URL strings or objects describing each URL.
If the node supports binary data output (not indicated in the provided code), it would represent downloadable content or files retrieved during the crawl, but no such indication is present here.
Dependencies
- Requires an external FireCrawl API service accessible via a base URL configured in the node credentials.
- An API key or authentication token must be provided through the node's credential configuration to authorize requests.
- The node sends HTTP requests with JSON payloads and expects JSON responses.
Troubleshooting
Common issues:
- Invalid or unreachable URL: Ensure the "Url" property is a valid and accessible website address.
- Authentication errors: Verify that the API key or authentication token is correctly set up in the node credentials.
- Network connectivity problems: Confirm that the n8n instance has internet access and can reach the FireCrawl API endpoint.
- Incorrect JSON in "Custom Body": If using a custom body, ensure the JSON syntax is valid to avoid parsing errors.
Error messages:
- HTTP 401 Unauthorized: Indicates missing or invalid API credentials. Check and update the API key.
- HTTP 400 Bad Request: Usually caused by malformed input parameters or invalid JSON in the custom body.
- Timeout or network errors: May occur if the API service is down or unreachable; retry later or check network settings.
Links and References
- FireCrawl official website: https://firecrawl.dev
- FireCrawl API documentation (if available) should be consulted for detailed request/response formats and additional features.