Hyperbrowser icon

Hyperbrowser

Interact with websites using Hyperbrowser
Join our community

Actions6

Overview

The Hyperbrowser node enables interaction with websites through various automated operations such as scraping, crawling, data extraction, and browser automation using AI agents. It is useful for scenarios where you need to programmatically gather web content, extract structured data, or automate browsing tasks without manual intervention.

Common use cases include:

  • Scraping the main content or links from a webpage.
  • Crawling multiple pages of a website to collect data.
  • Extracting specific information from a page using AI-driven queries and schemas.
  • Automating complex browser interactions like form filling or navigation using AI agents.
  • Using proxies and CAPTCHA solving to handle protected or geo-restricted content.

For example, you could scrape product details from an e-commerce site, crawl blog posts across multiple pages, or instruct an AI agent to log into a website and download reports automatically.

Properties

Name Meaning
URL The web address (URL) to process.
Options Collection of additional settings:
- Max Steps Maximum number of steps the AI agent should take to complete the task (for browser automation).
- Maximum Pages Maximum number of pages to crawl when crawling a website.
- Only Main Content Whether to return only the main content of the page (true/false).
- Output Format Format of the output content: HTML, Links, or Markdown.
- Proxy Country Country code for the proxy server if proxy usage is enabled.
- Solve CAPTCHAs Whether to attempt solving CAPTCHAs during scraping (true/false).
- Timeout (Ms) Maximum time in milliseconds to wait for page navigation before timing out.
- Use Proxy Whether to route requests through a proxy server (true/false).
- Use Vision Whether to enable vision capabilities for browser automation using AI (true/false).

Output

The node outputs JSON data containing the results of the selected operation:

  • Scrape: Returns an object with the original URL, the scraped content in the requested format (HTML, links, or markdown), and the HTTP status.

    Example:

    {
  "url": "https://example.com",
  "content": "<scraped content>",
  "status": 200
}

  • Crawl: Returns the URL, an array of crawled page data in markdown format by default, and the status.

  • Extract: Returns the URL, extracted structured data based on the provided query and schema, and the status.

  • Browser Automation Operations (Browser Use, Claude Computer Use, OpenAI CUA): Return the final actions or results performed by the AI agent along with the status.

If an error occurs and "Continue On Fail" is enabled, the output will contain an error message and the operation attempted.

The node does not output binary data.

Dependencies

  • Requires an API key credential for the Hyperbrowser service.
  • Uses the @hyperbrowser/sdk library to interact with the Hyperbrowser API.
  • Optional proxy configuration requires network access to proxy servers.
  • CAPTCHA solving depends on the Hyperbrowser service capabilities.
  • No additional environment variables are required beyond the API key credential.

Troubleshooting

  • Common Issues:

    • Invalid or missing API key credential will cause authentication failures.
    • Network issues or incorrect proxy settings can prevent successful scraping or crawling.
    • Timeouts may occur if pages take too long to load; adjust the timeout property accordingly.
    • CAPTCHA challenges might block scraping unless "Solve CAPTCHAs" is enabled.
    • Unsupported operations will throw errors indicating the operation is not supported.

  • Error Messages:

    • "Operation \"<operation>\" is not supported": Indicates an invalid operation was selected.
    • Network or API errors will be returned from the Hyperbrowser SDK and surfaced in the node output if "Continue On Fail" is enabled.

To resolve errors:

  • Verify API credentials are correct and active.
  • Check network connectivity and proxy configurations.
  • Increase timeout values for slow-loading pages.
  • Enable CAPTCHA solving if encountering CAPTCHA blocks.
  • Ensure the selected operation matches the intended use case.

Links and References

Discussion