Firecrawl icon

Firecrawl

Get data from Firecrawl API

Overview

The node "Firecrawl" is designed to interact with the Firecrawl API, specifically to retrieve the status of an extract job. This operation is useful when you have initiated a data extraction process via Firecrawl and want to check its current progress or completion status. Typical use cases include monitoring long-running data extraction tasks, automating workflows based on extract completion, or fetching metadata about the extract job.

For example, after starting an extract job in Firecrawl, you can use this node to periodically poll the extract status by providing the Extract ID, enabling your workflow to proceed only once the data is ready.

Properties

Name Meaning
Extract ID The unique identifier of the extract job whose status you want to retrieve.
Use Custom Body A boolean flag indicating whether to send a custom request body (not typically used here).
  • Extract ID is required and must be provided as a string.
  • Use Custom Body defaults to false and is generally not needed for this GET operation.

Output

The node outputs JSON data representing the status of the specified extract job. The exact structure depends on the Firecrawl API response but typically includes fields such as:

  • Extract job state (e.g., pending, running, completed, failed)
  • Progress details or percentage complete
  • Timestamps for start and completion
  • Any error messages if the extract failed

No binary data output is expected from this operation.

Dependencies

  • Requires an API key credential for authenticating with the Firecrawl API.
  • The base URL for the API can be configured via credentials; otherwise, it defaults to https://api.firecrawl.dev/v1.
  • The node sends HTTP requests with JSON content type headers.

Troubleshooting

  • Invalid Extract ID: If the provided Extract ID is incorrect or does not exist, the API may return a 404 or similar error. Verify the Extract ID is correct.
  • Authentication errors: Ensure the API key credential is valid and has necessary permissions.
  • Network issues: Check connectivity to the Firecrawl API endpoint.
  • Unexpected API responses: If the API changes or returns unexpected data, the node might fail to parse the response correctly.

Links and References

Discussion