AWS Kinesis icon

AWS Kinesis

Sends messages to a Kinesis stream
Join our community

Overview

This node sends messages to an AWS Kinesis stream. It is useful for streaming data pipelines, real-time analytics, or any scenario where you want to push JSON data or custom messages into a Kinesis stream for further processing or storage.

Typical use cases include:

  • Streaming event data from various sources into Kinesis for aggregation.
  • Sending logs or telemetry data in real time.
  • Integrating with AWS services that consume Kinesis streams.

The node can either send the incoming input data as JSON directly or send a custom message string. It also optionally supports creating the stream if it does not already exist.

Properties

Name Meaning
Stream Name of the Kinesis stream to which the messages will be published.
Send Input Data Whether to send the node's received input data as JSON to the Kinesis stream (true/false).
Message The custom message string to send when "Send Input Data" is false.
Allow Topic Creation Whether to create the Kinesis stream if it does not already exist (true/false).

Output

The output is a JSON array containing the results of the putRecords operation to Kinesis. Each element corresponds to a record sent and includes success or error information per record.

If no records are returned by the API, the node outputs a single object indicating success.

No binary data is output by this node.

Dependencies

  • Requires valid AWS credentials with permissions to access and write to Kinesis streams.
  • Uses the lifion-kinesis library internally to interact with AWS Kinesis.
  • The node expects AWS region, access key ID, and secret access key to be provided via configured credentials.

Troubleshooting

  • Common issues:

    • Incorrect or missing AWS credentials will cause authentication failures.
    • Specifying a non-existent stream without enabling stream creation will result in errors.
    • Network connectivity issues may prevent communication with AWS Kinesis.

  • Error messages:

    • Authentication errors typically indicate invalid or missing AWS credentials.
    • Stream not found errors occur if the stream does not exist and creation is disabled.
    • Permission denied errors suggest insufficient IAM permissions for the provided credentials.

  • Resolutions:

    • Verify AWS credentials and their permissions.
    • Enable "Allow Topic Creation" if you want the node to create the stream automatically.
    • Check network connectivity and AWS service availability.

Links and References

Discussion