LiteLLM

Interact with LiteLLM Proxy
Join our community

Actions2

Overview

This node integrates with the LiteLLM Proxy service to perform two main operations: generating chat completions and creating text embeddings. Specifically, for the "Embedding" operation, it sends input text to the LiteLLM API and retrieves a vector representation (embedding) of that text. This is useful in scenarios such as semantic search, recommendation systems, or any application requiring numerical representations of text data.

Practical examples include:

  • Converting user queries into embeddings for similarity matching.
  • Generating embeddings for documents to enable clustering or classification.
  • Feeding embeddings into machine learning models for downstream tasks.

Properties

Name Meaning
Model Select the LiteLLM model to use for generating embeddings. The list is dynamically loaded from the LiteLLM API.
Input Text The text string for which the embedding vector will be generated.

Output

The node outputs an array of JSON objects, each containing the full response from the LiteLLM /embeddings endpoint. The exact structure depends on the LiteLLM API but typically includes the embedding vector and possibly metadata about the request or model used.

No binary data output is produced by this operation.

Example output snippet (conceptual):

{
  "data": [
    {
      "embedding": [0.123, 0.456, ...],
      "index": 0
    }
  ],
  "model": "selected-model-id",
  "usage": {
    "prompt_tokens": 10,
    "total_tokens": 10
  }
}

Dependencies

  • Requires an API key credential for authenticating with the LiteLLM Proxy service.
  • The node makes HTTP requests to the LiteLLM API endpoints /models (to fetch available models) and /embeddings (to generate embeddings).
  • Proper configuration of the API base URL and authentication token is necessary in the n8n credentials setup.

Troubleshooting

  • Common issues:

    • Invalid or missing API key: The node will fail to authenticate with the LiteLLM API.
    • Selecting a model that does not support embeddings may cause errors.
    • Empty or invalid input text can result in API errors or empty embeddings.

  • Error messages:

    • Authentication errors typically indicate problems with the API key or credential setup.
    • HTTP 4xx or 5xx responses from the LiteLLM API suggest issues with the request payload or server availability.

  • Resolutions:

    • Verify the API key and ensure it has the required permissions.
    • Confirm the selected model supports the embedding operation.
    • Ensure the input text is non-empty and properly formatted.

Links and References

Discussion