LiteLLM Chat Model icon

LiteLLM Chat Model

Modelo de chat compatível com agente usando LiteLLM
Join our community

Overview

This node integrates with the LiteLLM API to generate chat-based responses using a specified language model. It is designed for scenarios where users want to automate conversational AI tasks, such as customer support, virtual assistants, or interactive chatbots. The node accepts a series of chat messages and optional system prompts, then sends these to the LiteLLM API to receive generated text responses based on the input context.

Use Case Examples

  1. Generating a customer support reply based on user queries.
  2. Creating a virtual assistant response in a chat application.
  3. Automating conversational flows for interactive chatbots.

Properties

Name Meaning
Model The language model to be used for generating the chat response.
System Prompt An optional system prompt applied as the first message to set the context or behavior of the assistant.
Messages An array of chat messages forming the conversation history to provide context for the generation.
Temperature Controls the randomness of the generation, with values between 0 and 2.
Max Tokens Maximum number of tokens allowed in the generated response.
Safety Identifier An identifier to track users and apply safety policies.
Custom Headers Additional custom HTTP headers to include in the API request.
Base URL Override the default base URL of the LiteLLM API.
Frequency Penalty Penalty applied to reduce repetition in the generated text, ranging from -2 to 2.
Presence Penalty Penalty applied to encourage novelty in the generated text, ranging from -2 to 2.
Top P Controls the cumulative probability for token selection, between 0 and 1.
Response Format Expected format of the response, either plain text or JSON.
Timeout (ms) Maximum time in milliseconds to wait for the API response.
Max Retries Maximum number of retry attempts in case of request failures.
Service Tier Desired service level for the API request.
Metadata Key-value pairs for structured storage or tracking.
Prompt Cache Key Key used for caching prompts to optimize repeated requests.
Top Logprobs Number of top probable tokens to return, between 0 and 20.

Output

JSON

  • text - The generated chat response text.
  • model - The model used for generating the response.
  • usage - Token usage statistics for the request and response.
  • finishReason - Reason why the generation finished (e.g., stop sequence, max tokens).
  • raw - The raw response object returned from the LiteLLM API.

Dependencies

  • LiteLLM API key credential

Troubleshooting

  • Invalid JSON format in the 'Messages' input will cause an error; ensure the messages are a valid JSON array.
  • Custom headers must be valid JSON; invalid JSON will cause an error.
  • API request failures may occur; the node retries up to the configured 'Max Retries' before failing.
  • Timeout errors can occur if the API response takes longer than the configured timeout; increase the timeout if needed.

Discussion