n8n-nodes-remote-scraper

Custom n8n nodes for automating remote scrapers via the Remote Scraper Client API.

Nodes

Remote Scraper Session

Manages the lifecycle of remote scraper sessions.

Operations:

• Start: Create a new session with optional configuration
• Info: Get session details, status, and executed actions
• Close: Close session and cleanup resources (stops recordings, kills processes)

Remote Scraper Action (Sync)

Execute a single action immediately within a session.

• Waits for the action to complete
• Returns the action result
• Supports all available actions (app, screen, system, playwright)

Remote Scraper Action (Async)

Build and execute action workflows using a builder pattern.

Operations:

• Add Action: Queue an action for later execution (accumulated in workflow data)
• Execute: Run all queued actions, optionally waiting for completion

Installation

For Development

cd n8n_nodes
npm install
npm run build

Link to n8n

# In this directory
npm link

# In your n8n installation
npm link n8n-nodes-remote-scraper

For Production

# In your n8n custom nodes directory
npm install /path/to/n8n_nodes

Usage

Sync Workflow Example

[Session: Start] → [Action Sync: app.launch] → [Action Sync: screen.screenshot] → [Session: Close]

1. Session (Start): Creates a new session, outputs sessionId
2. Action Sync: Picks up sessionId automatically, executes action
3. Action Sync: Chains another action in the same session
4. Session (Close): Closes session and cleans up resources

Async Builder Workflow Example

[Session: Start] → [Action Async: Add] → [Action Async: Add] → [Action Async: Execute] → [Session: Close]

1. Session (Start): Creates session
2. Action Async (Add): Adds first action to queue
3. Action Async (Add): Adds second action to queue
4. Action Async (Execute): Sends all queued actions, waits for completion
5. Session (Close): Cleanup

Available Actions

Actions are fetched dynamically from the API. Categories include:

• app.*: Application management (launch, close, list)
• screen.*: Screen recording and screenshots
• system.*: System info, stats, wait
• playwright.*: Browser automation (navigate, click, fill, etc.)

Configuration

Credentials

Set up the Remote Scraper API credentials:

1. In n8n, go to Credentials → New
2. Search for "Remote Scraper API"
3. Enter your API URL (e.g., http://localhost:8000)
4. Test the connection (calls /health endpoint)

Session Options

When starting a session, you can configure:

• Max Action Retries: Number of retry attempts per action (default: 3)
• Retry Delay: Seconds between retries (default: 1)
• Action Timeout: Default timeout per action in seconds (default: 60)

Data Flow

Each node preserves data from previous nodes and adds its own output:

// After Session (Start)
{ sessionId: "abc123", status: "idle" }

// After Action Sync (app.launch)
{ sessionId: "abc123", pid: 1234, success: true, lastAction: {...} }

// After Action Sync (screen.screenshot)  
{ sessionId: "abc123", pid: 1234, filename: "shot.png", lastAction: {...} }

The sessionId flows through automatically, so you don't need to manually wire it.

Development

# Install dependencies
npm install

# Generate action properties (requires API to be running)
# Set REMOTE_SCRAPER_API_URL if API is not on localhost:8000
REMOTE_SCRAPER_API_URL=http://your-api:8000 npm run generate:properties

# Build (automatically runs generate:properties first)
npm run build

# Watch mode
npm run dev

# Lint
npm run lint
npm run lint:fix

Building Action Properties

The nodes use build-time property generation to create dynamic form fields for each action. This means:

1. At build time, the generate:properties script fetches all actions from your API
2. Generates TypeScript code with properties for each action parameter
3. Properties are shown/hidden based on selected action using displayOptions

To regenerate properties:

# With API running on localhost:8000
npm run generate:properties

# With custom API URL
REMOTE_SCRAPER_API_URL=http://your-api:8000 npm run generate:properties

The generated file nodes/RemoteScraper/generated-action-properties.ts is committed to git so builds work without requiring the API to be available.

License

MIT