X Scraper by Lab3 icon

X Scraper by Lab3

Scrape X Data with Lab3

Overview

The "X Scraper by Lab3" node enables scraping and interacting with data from the social media platform X (formerly Twitter). It supports operations on two main resources: Tweets and Users. For the User resource, it can retrieve user profile information and fetch tweets posted by a specific user.

This node is beneficial when you want to automate the extraction of public user data or tweets for analytics, monitoring, or integration into other workflows without using the official API. For example, you could use it to gather recent tweets from a competitor's account or collect user profile details for marketing research.

Specifically, the Get Tweets operation under the User resource fetches a list of tweets posted by a specified user, up to a defined limit.

Properties

Name Meaning
User The user whose tweets you want to retrieve. You specify this by their username (e.g., "n8n").
Limit Maximum number of tweets to return. Must be at least 1. Defaults to 20.

Output

The node outputs an array of JSON objects representing tweets retrieved from the specified user. Each object corresponds to a single tweet and contains all relevant tweet data as returned by the scraper library. This typically includes tweet text, creation date, tweet ID, and possibly metadata such as retweet and like counts.

No binary data output is produced by this operation.

Dependencies

  • Requires an API key credential with several authentication tokens and cookies (guest_id, kdt, twid, ct0, auth_token) to access X data.
  • Uses an internal scraper library ("Eliza") that handles session creation and requests to X.
  • No external API calls beyond the scraper library are made.
  • The node expects these credentials to be configured in n8n before execution.

Troubleshooting

  • Missing Credentials Error: If any required authentication tokens are missing from the credentials, the node will throw an error listing which ones are absent. Ensure all required tokens are correctly set in the credential configuration.
  • Failed to Initialize Scraper: Indicates issues creating the scraper session, possibly due to invalid or expired tokens. Refresh or reconfigure your credentials.
  • Empty or Partial Tweet Data: Could occur if the user does not exist, has no tweets, or if rate limits/blocking prevent data retrieval.
  • Limit Parameter Issues: Setting the limit below 1 will cause validation errors; ensure the limit is a positive integer.

Links and References

Discussion