Overview

This node extracts content from a list of provided URLs based on user-defined options. It is useful for scenarios where you need to gather specific information or data chunks from multiple web pages, such as content summarization, data scraping, or research automation. For example, you can extract text or markdown content, include images, or retrieve metadata like favicons from the URLs.

Use Case Examples

1. Extracting key information from multiple news articles by providing their URLs and a natural language query.
2. Gathering product details and images from e-commerce pages for comparison or analysis.

Properties

Name	Meaning
URLs	A list of URLs to extract content from, allowing multiple URLs to be specified.
Options	Additional extraction settings to customize the output.

Output

JSON

• results - An array of extracted content objects from each URL, including text or markdown content, images (if requested), and metadata such as favicon URL.
• usage - Credit usage information included if requested in options.

Dependencies

• This node requires access to the Tavily API or service for content extraction, which may require an API key or authentication token.

Troubleshooting

• Ensure the URLs provided are valid and accessible; invalid or unreachable URLs will cause extraction to fail.
• If the query is provided but no content is returned, try adjusting the query or increasing the chunks per source.
• Advanced extraction may increase latency; if the node times out, consider using basic extraction or reducing the number of URLs.
• If images or favicons are requested but not returned, verify that the URLs actually contain such content and that the extraction depth is sufficient.

Links

• Tavily Official Website - Official site for the Tavily service used for content extraction.