X Scraper Stream Trigger icon

X Scraper Stream Trigger

Streams tweets in real-time using Rettiwt API stream function

Overview

This node streams tweets in real-time using the Rettiwt API's streaming capabilities. It allows users to monitor Twitter content based on various filtering criteria such as search terms, specific users, hashtags, mentions, or advanced custom filters. The node continuously emits new tweets matching the selected criteria and can also emit a start message when the stream begins.

Common scenarios where this node is beneficial include:

  • Real-time social media monitoring for brand mentions or competitor activity.
  • Tracking trending topics or hashtags relevant to marketing campaigns.
  • Gathering tweets from specific influential users or communities.
  • Filtering tweets by language, date range, or engagement metrics using advanced options.

Practical example: A marketing team could use this node to stream tweets containing certain hashtags related to their product launch, enabling them to respond quickly to customer feedback or questions.

Properties

Name Meaning
Stream Type The type of tweet stream to initiate. Options: Advanced Filter, From Users, Hashtags, Mentions, Search Terms.
Search Terms Terms to search for in tweets (used when Stream Type is "Search Terms").
Usernames Comma-separated list of usernames (without @) to stream tweets from (used when Stream Type is "From Users").
Mention Users Comma-separated list of usernames (without @) to monitor for mentions (used when Stream Type is "Mentions").
Hashtags Comma-separated list of hashtags (without #) to stream tweets containing these hashtags (used when Stream Type is "Hashtags").
Advanced Filter Options Collection of advanced filtering parameters including: Include Phrase, From Users, To Users, Language code, Start Date, End Date, Min Replies, Min Retweets, Min Likes, Top Tweets Only (used when Stream Type is "Advanced Filter").
Polling Interval How often to poll for new tweets in milliseconds. Default is 60000 (1 minute).
Options Additional options: Include Metadata (whether to include extra metadata in output), Stream Start Message (whether to emit a message when the stream starts).

Output

The node outputs JSON objects representing tweets that match the specified filter criteria. Depending on the "Include Metadata" option, each output item can be either:

  • The raw tweet object as received from the Rettiwt API.
  • An object containing:
    • tweet: The tweet data.
    • metadata: Additional information including:
      • streamType: The selected stream type.
      • filter: The applied filter parameters.
      • receivedAt: Timestamp when the tweet was received.
      • isManualTrigger (only for manual trigger outputs): Indicates if the tweet was fetched via manual polling.

Additionally, the node can emit special messages:

  • A "stream_start" message when the stream begins (if enabled).
  • Error messages with types like "error", "stream_error", or "manual_trigger_error" containing error details and timestamps.

The node does not output binary data.

Dependencies

  • Requires an API key credential for the Rettiwt API service.
  • Uses the Rettiwt API client library to connect and stream tweets.
  • No additional environment variables are required beyond the API key credential.

Troubleshooting

  • Missing Required Parameters: If required inputs like search terms, usernames, hashtags, or mention users are empty when their respective stream type is selected, the node will throw an error indicating the missing parameter. Ensure all required fields are filled.
  • API Connection Errors: Errors connecting to the Rettiwt API or during streaming will be logged and emitted as "stream_error" messages. Check your API key validity and network connectivity.
  • Stream Closure Issues: Errors closing the stream are logged but do not stop the node. These usually indicate network interruptions or internal cleanup issues.
  • No Tweets Received: If no tweets appear, verify that the filter criteria are correct and that there is active Twitter content matching those filters.
  • Rate Limits: Although not explicitly handled, excessive polling intervals or too broad filters may lead to rate limiting by the API.

Links and References

Discussion