Actions8
Overview
The node "FireCrawl" integrates with the FireCrawl API to manage and check the status of crawl jobs. Specifically, the "Check Crawl Job Status" operation allows users to query the current state of a web crawling job by providing its unique crawl ID. This is useful in scenarios where you need to monitor or verify the progress and results of automated web data extraction tasks.
Practical examples include:
- Monitoring scheduled crawls to trigger downstream workflows once data is ready.
- Verifying if a crawl job has completed successfully before processing the extracted data.
- Debugging or auditing crawl jobs by checking their statuses programmatically.
Properties
| Name | Meaning |
|---|---|
| Crawl Id | The unique identifier of the crawl job whose status you want to check. |
| Use Custom Body | A boolean flag indicating whether to send a custom request body (true) or use the default request format (false). |
Output
The node outputs JSON data representing the status information of the specified crawl job. This typically includes fields such as the job's current state (e.g., running, completed, failed), timestamps, and possibly metadata about the crawl results.
If binary data output is supported (not indicated explicitly here), it would represent any files or raw data retrieved from the crawl job.
Dependencies
- Requires an API key credential for authenticating with the FireCrawl API.
- The base URL for the FireCrawl API must be configured in the node credentials.
- The node sends HTTP requests with JSON content type headers.
Troubleshooting
- Missing or invalid Crawl Id: Ensure that the "Crawl Id" property is provided and correctly corresponds to an existing crawl job.
- Authentication errors: Verify that the API key credential is valid and has the necessary permissions.
- Network or API endpoint issues: Confirm that the base URL is correct and the FireCrawl service is reachable.
- Using custom body incorrectly: If "Use Custom Body" is enabled, ensure the custom payload matches the API requirements; otherwise, the request may fail.
Common error messages might include unauthorized access, not found errors for invalid crawl IDs, or bad request errors due to malformed input.
Links and References
- FireCrawl official API documentation (please refer to your FireCrawl API provider’s docs for detailed endpoints and request/response formats).