LiteLLM

Interact with LiteLLM Proxy
Join our community

Actions2

Overview

This node integrates with the LiteLLM Proxy API to perform two main operations: generating chat completions and creating text embeddings. For the Chat Completion operation, it sends a user message to the LiteLLM service and receives a generated response based on the selected model. This is useful for building conversational AI applications, chatbots, or any scenario where automated text generation in response to user input is needed.

For example, you can use this node to:

  • Create an interactive chatbot that responds to user queries.
  • Generate dynamic replies or suggestions based on user messages.
  • Prototype conversational agents using different language models available via LiteLLM.

Properties

Name Meaning
Model Select the LiteLLM model to use for the operation. The list of models is dynamically loaded from the LiteLLM API.
User Message The content of the message sent by the user for which the chat completion (response) will be generated.

Output

The node outputs an array of JSON objects, each containing the full response from the LiteLLM API for the chat completion request. The structure of the json output field corresponds directly to the API's response format for chat completions, typically including generated text and metadata about the completion.

No binary data output is produced by this node.

Dependencies

  • Requires an API key credential for authenticating with the LiteLLM Proxy API.
  • The node makes HTTP requests to the LiteLLM API endpoints /chat/completions and /models.
  • The base URL and API key must be configured in the node’s credentials before execution.

Troubleshooting

  • Common issues:

    • Invalid or missing API key credential will cause authentication failures.
    • Selecting a model that does not exist or is unavailable may result in errors from the API.
    • Network connectivity problems can prevent successful API calls.

  • Error messages:

    • Authentication errors typically indicate invalid or expired API keys; verify and update credentials.
    • HTTP 4xx or 5xx errors from the API suggest issues with request parameters or server-side problems; check the model selection and message content.
    • Timeout or network errors require checking internet connection and API endpoint accessibility.

Links and References

Discussion