BullMQ icon

BullMQ

Get, send and update data in Redis

Overview

This node integrates with BullMQ, a Redis-based queue system, to add jobs to queues. It is designed to enqueue tasks for asynchronous processing, which is useful in scenarios where you want to offload work from the main workflow or handle background jobs such as sending emails, processing images, or running long computations.

The node supports adding jobs either to queues associated with other workflows or to custom-named queues. It allows specifying job data, options like retries, delays, priority, and whether to wait for the job's completion before continuing.

Practical examples:

  • Adding a job to a workflow queue to trigger another workflow asynchronously.
  • Enqueuing a custom job with specific data and retry policies for background processing.
  • Waiting for a job to finish and retrieving its result before proceeding.

Properties

Name Meaning
Queue Source Selects the source of the queue: "Workflows" (queues linked to workflows) or "Custom" (user-defined name).
Queue Name The name of the custom queue to add the job to (required if Queue Source is "Custom").
Workflow Selects the workflow whose queue will receive the job (required if Queue Source is "Workflows").
Job Name The name of the job to publish to the queue (required).
Data Source Specifies where to get the job data from: "Previous Node" (data from previous node) or "Input" (manual input).
Job Data The data to send with the job when Data Source is "Input".
Wait Until Finished Boolean flag indicating whether to wait for the job to complete before continuing. Not recommended for long-running jobs.
Options Collection of optional job parameters:
- Attempts Number of times to retry the job on failure.
- Backoff Delay in milliseconds between retry attempts.
- Delay Delay in milliseconds before the job should be processed.
- Lifo Whether to process the job in Last-In-First-Out order instead of FIFO.
- Priority Job priority; higher numbers indicate higher priority.
- Remove On Complete Whether to remove the job from the queue after successful completion.
- Remove On Fail Whether to remove the job from the queue if it fails.
- Return Value Whether to return the job's result value after completion.
- timeToLive Time in milliseconds before the job is considered failed if not completed.

Output

The node outputs an array of items corresponding to each input item processed. Each output item contains:

  • json: An object representing the job details returned by BullMQ, including job ID, status, timestamps, and optionally the job's return value if requested.
  • pairedItem: Metadata linking the output item to the original input item index.

If the "Wait Until Finished" option is enabled, the output includes the final job state and optionally the job's return value.

No binary data output is produced by this node.

Dependencies

  • Requires a Redis server accessible via credentials configured in n8n.
  • Uses BullMQ library to interact with Redis queues.
  • Requires proper configuration of Redis credentials in n8n to connect successfully.

Troubleshooting

  • Common issues:

    • Failure to connect to Redis due to incorrect credentials or network issues.
    • Missing required parameters like queue name or workflow selection.
    • Using "Wait Until Finished" for long-running jobs may cause timeouts or blocking behavior.
    • Job addition failures if the workflow ID is invalid or the queue does not exist.
  • Error messages:

    • "The workflow did not return an id!" — Indicates that the selected workflow could not be resolved properly. Verify the workflow exists and is accessible.
    • "The job did not return an id!" — Job creation failed unexpectedly; check Redis connectivity and queue configuration.
    • "The operation \"add\" is not supported!" — Operation parameter is invalid or unsupported; ensure "Add" is selected.
    • Connection errors related to Redis credentials — Check that the API key or authentication token for Redis is correctly set up in n8n.

Links and References

Discussion