Overview
This node enables users to interact with a local large language model (LLM) for chat-based tasks. It sends a series of messages to the specified LLM model and receives generated completions in response. This is useful for scenarios such as building conversational agents, generating text based on prompts, or experimenting with different LLM models locally without relying on external cloud services.
Practical examples include:
- Creating a chatbot that answers user questions.
- Generating creative writing or code snippets based on input messages.
- Testing different LLM models by changing the model parameter.
Properties
| Name | Meaning |
|---|---|
| Model | The identifier or name of the local LLM model to use for generating chat completions. |
| Messages | A JSON array representing the conversation history or prompt messages sent to the model. Each message should have a "role" (e.g., "user") and "content" (the text). |
Output
The node outputs an array of items where each item contains a json field with the completion result returned from the local LLM. The structure of the output JSON corresponds to the response from the underlying LLM library's completion method.
json: Contains the generated chat completion data from the model.pairedItem: Indicates which input item this output corresponds to.
The node does not output binary data.
Dependencies
- Requires access to a local LLM API service configured via credentials that provide an API key and base URL.
- Uses an external library to communicate with the local LLM.
- The node expects the user to configure credentials containing the API key and URL for the local LLM service.
Troubleshooting
Common issues:
- Invalid JSON format in the "Messages" property can cause parsing errors.
- Incorrect or missing API key or URL in credentials will prevent successful communication with the local LLM.
- Specifying a non-existent or unsupported model name may lead to errors from the LLM service.
Error messages:
- JSON parsing errors when the "Messages" input is malformed. Ensure valid JSON syntax.
- Authentication errors if the API key or URL is incorrect or missing.
- Node operation errors indicating failure to get a completion from the model; check model name and service availability.
To resolve errors, verify the JSON format of messages, confirm credential correctness, and ensure the local LLM service is running and accessible.
Links and References
- LiteLLM GitHub Repository (example placeholder)
- Documentation for the local LLM API used (refer to your specific local LLM provider)
- n8n documentation on creating custom nodes and handling credentials