BullMQ icon

BullMQ

Get, send and update data in Redis

Overview

This node integrates with BullMQ, a Redis-based job queue system, allowing users to add jobs to queues managed in Redis. It is particularly useful for orchestrating background tasks, delayed processing, or workload distribution across multiple workers.

Common scenarios include:

  • Scheduling asynchronous workflows triggered by events.
  • Offloading heavy or time-consuming tasks from the main workflow.
  • Managing job priorities and retries in distributed systems.

For example, you can use this node to enqueue a data processing job that runs separately from the main workflow, optionally waiting for its completion before proceeding.

Properties

Name Meaning
Queue Source Selects the source of the queue: either "Workflows" (queues linked to workflows) or "Custom" (user-defined queue name).
Queue Name The name of the custom queue to which the job will be added (required if Queue Source is "Custom").
Workflow Selects a workflow whose associated queue will receive the job (required if Queue Source is "Workflows").
Job Name The name of the job to publish to the queue (required).
Data Source Specifies where to get the job data from: "Previous Node" uses data from the previous node; "Input" uses static input data.
Job Data Defines the data payload for the job when using "Input" as the data source.
Wait Until Finished Boolean flag indicating whether to wait for the job to finish before continuing. Not recommended for long-running jobs.
Options Collection of optional job parameters:
- timeToLive Time in milliseconds before the job should be considered failed.
- Return Value Whether to return the job's result value after completion.
- Delay Delay in milliseconds before the job is processed.
- Priority Job priority (higher number means higher priority).
- Attempts Number of retry attempts for the job.
- Backoff Backoff delay in milliseconds between retry attempts.
- Lifo Whether to process the job in Last-In-First-Out order instead of FIFO.
- Remove On Complete Whether to remove the job from the queue once completed.
- Remove On Fail Whether to remove the job from the queue if it fails.

Output

The node outputs an array of items corresponding to each input item, where each output item contains:

  • json: An object representing the job details returned by BullMQ, including job ID, status, timestamps, and other metadata.
  • If the "Wait Until Finished" option is enabled and "Return Value" is true, the output JSON will contain the job's return value instead of the full job object.

No binary data is produced by this node.

Dependencies

  • Requires a Redis server accessible via credentials configured in n8n.
  • Uses the BullMQ library to interact with Redis queues.
  • Requires proper configuration of Redis credentials within n8n to establish connection.

Troubleshooting

  • Missing or invalid Redis credentials: Ensure that the Redis credentials are correctly set up and tested in n8n.
  • Queue or workflow not found: When using "Workflows" as the queue source, the selected workflow must have a valid ID; otherwise, the node throws an error.
  • Job did not return an ID: This error indicates failure in adding the job to the queue; verify Redis connectivity and queue configuration.
  • Waiting for long-running jobs: Enabling "Wait Until Finished" for long-running jobs may cause timeouts or performance issues; avoid using this option for such cases.
  • Invalid job options: Parameters like priority, attempts, or backoff must be non-negative numbers; incorrect values may cause job addition failures.

Links and References

Discussion