Normalize URL

Normalizes a URL using the normalize-url package

Overview

This node normalizes URLs using a comprehensive set of options to clean and standardize them. It is useful in scenarios where consistent URL formatting is required, such as data cleaning, SEO optimization, or preparing URLs for comparison or storage. For example, it can remove unnecessary parts like authentication info, "www." prefix, trailing slashes, or sort query parameters to ensure URLs are uniform.

Properties

Name Meaning
URL The URL string to be normalized.
Options A collection of boolean options controlling how the URL is normalized:
- Normalize Protocol: Normalize the protocol part of the URL.
- Force HTTP: Force the protocol to HTTP.
- Force HTTPS: Force the protocol to HTTPS.
- Strip Authentication: Remove user authentication info from the URL.
- Strip Hash: Remove the hash fragment (after #) from the URL.
- Strip Protocol: Remove the protocol scheme (e.g., http://) from the URL.
- Strip Text Fragment: Remove text fragments from the URL.
- Strip WWW: Remove the "www." prefix from the URL.
- Remove Query Parameters: Remove all query parameters from the URL.
- Remove Trailing Slash: Remove trailing slash at the end of the URL path.
- Remove Single Slash: Remove single slash URLs (e.g., root slash).
- Remove Directory Index: Remove directory index files like "index.html" from the URL path.
- Remove Explicit Port: Remove explicit port numbers from the URL.
- Sort Query Parameters: Sort query parameters alphabetically.

Output

The node outputs an array of items with each item containing a JSON object with the following field:

  • normalizedUrl: The normalized URL string after applying the selected normalization options.

If an error occurs during normalization for an input item, the output JSON will contain:

  • error: The error message describing what went wrong.

No binary data output is produced by this node.

Dependencies

  • This node depends on the external npm package normalize-url to perform the URL normalization.
  • No additional API keys or external services are required.
  • No special environment variables or n8n credentials are needed.

Troubleshooting

  • Common issues:
    • Invalid URL format input may cause errors.
    • Conflicting options (e.g., forcing both HTTP and HTTPS) might lead to unexpected results.
  • Error messages:
    • If the URL is malformed, the node will throw an error indicating the invalid URL.
    • When continueOnFail is enabled, errors for individual items are returned in the output JSON under the error field instead of stopping execution.
  • Resolution:
    • Ensure input URLs are valid and properly formatted.
    • Review option selections to avoid contradictory settings.

Links and References

Discussion