Overview
This node acts as an interface to a language model API, designed for advanced usage within AI chains or agents. It allows users to generate text completions or structured JSON responses by specifying a model and various generation options. Typical use cases include generating conversational replies, completing text prompts, or producing structured data outputs in workflows that integrate AI capabilities.
Practical examples:
- Generating chatbot responses dynamically based on user input.
- Creating content summaries or expansions.
- Producing JSON-formatted data from natural language prompts for further automated processing.
Properties
| Name | Meaning |
|---|---|
| This node must be connected to an AI chain. Insert one | A notice indicating that this node requires connection to an AI chain or agent node to function properly. |
| If using JSON response format, you must include word "json" in the prompt in your chain or agent. Also, make sure to select latest models released post November 2023. | A notice shown when the response format is set to JSON, reminding users to include the word "json" in their prompt and to use recent models for proper JSON output. |
| Model | The language model used to generate completions. Options are dynamically loaded from the API endpoint /v1/models. Default is "gemini/2.0-flash". |
| Options | Additional parameters to customize the completion behavior: |
| Frequency Penalty | Penalizes new tokens based on their existing frequency in the generated text to reduce repetition. Range: -2 to 2. |
| Maximum Number of Tokens | Maximum number of tokens to generate in the completion. Supports up to 32,768 tokens depending on the model. Default is -1 (no limit). |
| Response Format | Output format of the response. Options: Text (regular text) or JSON (ensures the model generates valid JSON). Default is text. |
| Presence Penalty | Penalizes new tokens based on whether they appear in the text so far, encouraging the model to talk about new topics. Range: -2 to 2. |
| Sampling Temperature | Controls randomness of completions. Lower values produce more deterministic results. Range: 0 to 1. Default is 0.7. |
| Timeout | Maximum time allowed for the request in milliseconds. Default is 60000 (60 seconds). |
| Max Retries | Maximum number of retry attempts if the request fails. Default is 2. |
| Top P | Controls diversity via nucleus sampling. Values between 0 and 1, where lower values consider fewer likely options. Default is 1. |
Output
The node outputs data under the key ai_languageModel with the following structure:
json: Contains the text or JSON response generated by the language model, depending on the selected response format.- If the response format is JSON, the output guarantees valid JSON content suitable for downstream processing.
- No binary data output is produced by this node.
Dependencies
- Requires an API key credential for authentication with the LiteLLM service.
- Connects to the LiteLLM API base URL specified in credentials.
- Uses the OpenAI-compatible client library internally to communicate with the model endpoint.
- Requires connection to an AI chain or agent node in n8n to provide prompts and context.
Troubleshooting
Common issues:
- Failure to connect due to missing or invalid API credentials.
- Timeout errors if the model takes too long to respond; can be adjusted via the Timeout property.
- Rate limiting errors from the API; the node retries automatically but may fail after max retries.
- Invalid JSON output if the prompt does not include the word "json" when using JSON response format.
Error messages:
- Rate limit errors will throw a descriptive message advising to check API usage limits.
- Node API errors wrap underlying HTTP or API errors with additional context.
Resolutions:
- Ensure valid API key and base URL are configured.
- Include the word "json" in prompts when expecting JSON output.
- Adjust timeout and retry settings according to network conditions.
- Use latest models released after November 2023 for JSON mode.
Links and References
- OpenAI Models Overview
- Documentation on model parameters and usage can be found in the official LiteLLM or OpenAI API docs.