Overview
This node executes a job using the Lilypad Llama3 language model pipeline. It sends a prompt along with optional system instructions and various generation parameters to the model, then returns the generated text output. This node is useful for scenarios where you want to generate natural language responses, summaries, or completions based on custom prompts, such as chatbots, content creation, or automated report generation.
Practical examples:
- Generating creative writing or story continuations.
- Producing answers to user questions in a conversational AI.
- Summarizing long documents by providing appropriate prompts.
- Customizing model behavior via system messages and fine-tuning generation parameters.
Properties
| Name | Meaning |
|---|---|
| Module | The specific Lilypad Llama3 model pipeline to use (e.g., "ollama-pipeline:llama3-8b-lilypad1"). |
| System | System message to set up the model's behavior or context before generating a response. Typically used to guide the tone or style of the output. |
| Prompt | The main input text prompt for which the model will generate a response. |
| Output Format | Format of the response output. Options are: Plain Text (default) or JSON. |
| Options | Additional model parameters to control generation behavior: |
| - Mirostat | Sampling method for controlling perplexity. Options: Disabled, Mirostat, Mirostat 2.0. |
| - Mirostat ETA | Learning rate for how quickly the algorithm responds to feedback from generated text. Lower values slow adjustments; higher values make it more responsive. |
| - Mirostat TAU | Balances coherence vs diversity of output. Lower values produce more focused and coherent text. |
| - Context Size | Size of the context window used to generate the next token (number of tokens). |
| - Repeat Lookback Size | How far back the model looks to prevent repetition. Default 64; 0 disables; -1 equals full context size. |
| - Repeat Penalty | Strength of penalizing repeated tokens. Higher values penalize repetitions more strongly. |
| - Temperature | Controls creativity of the model output. Higher temperature increases randomness and creativity. |
| - Seed | Random seed for generation. Setting a fixed number makes output deterministic for the same prompt. |
| - Stop | Stop sequences that cause the model to stop generating further text when encountered. Multiple stop patterns can be specified. |
| - Tail Free Sampling (tfs) | Reduces impact of less probable tokens. Values >1 reduce impact more; 1 disables this feature. |
| - Predict Limit | Maximum number of tokens to predict. Default 128; -1 means infinite generation; -2 fills the context window. |
| - Top-K | Limits sampling to top K tokens to reduce nonsense. Higher values increase diversity; lower values make output more conservative. |
| - Top-P | Nucleus sampling parameter working with Top-K. Higher values increase diversity; lower values focus output. |
Output
The node outputs an array of items, each containing a json field with the model's response data. The exact structure depends on the chosen output format:
- If "Plain Text" is selected, the output contains the generated text as a string.
- If "JSON" format is selected, the output contains the response structured as JSON.
No binary data output is produced by this node.
Dependencies
- Requires an API key credential for authenticating with the Lilypad Llama3 service.
- Uses an internal helper function to queue jobs to the Lilypad service.
- No additional environment variables are explicitly required beyond the API credential.
Troubleshooting
Common issues:
- Invalid or missing API credentials will cause authentication failures.
- Incorrect module names may result in errors or no response.
- Improperly formatted prompts or system messages might lead to unexpected output.
- Setting incompatible or extreme options (e.g., very high temperature or predict limits) could cause timeouts or nonsensical results.
Error messages:
- Authentication errors indicate problems with the provided API key credential.
- Parameter validation errors suggest missing required fields like "Module" or "Prompt".
- Timeout or network errors imply connectivity issues with the Lilypad service.
Resolutions:
- Verify and configure the API key credential correctly in n8n.
- Double-check all required properties are set properly.
- Adjust generation parameters to reasonable values.
- Ensure network access to the Lilypad API endpoint.
Links and References
- Lilypad Llama3 Model Documentation (replace with actual URL if available)
- Language Model Generation Parameters Explained
- n8n Credential Setup Guide