Overview
This node integrates with DigitalOcean Gradient™ AI Serverless Inference to generate text completions using AI models. It supports two main input types for generating completions: a simple text prompt or a structured conversation with chat messages including roles (system, user, assistant). The node is useful for scenarios such as:
- Generating natural language responses or content based on a prompt.
- Building conversational AI agents by providing multi-turn chat history.
- Experimenting with different AI models and tuning generation parameters like temperature, max tokens, penalties, and stop sequences.
- Calling external tools or functions from within the generated completions.
Practical examples include drafting emails, summarizing text, creating chatbot replies, or generating code snippets dynamically.
Properties
| Name | Meaning |
|---|---|
| Model | The AI model used to generate the completion. Models can be selected from a list fetched dynamically from the API. |
| Input Type | How to provide input: either a "Simple Prompt" (single text string) or "Chat Messages" (a list of messages with roles). |
| Prompt | The text prompt to generate completions for (used if Input Type is "Simple Prompt"). |
| Messages | A collection of messages forming a conversation, each with a role (System, User, Assistant) and content (used if Input Type is "Chat Messages"). |
| Options | Additional options controlling generation behavior, including: |
| - Frequency Penalty | Penalizes new tokens based on their existing frequency in the text (-2.0 to 2.0). |
| - Logit Bias | JSON object mapping token IDs to bias values to modify likelihood of tokens appearing. |
| - Logprobs | Whether to return log probabilities of output tokens (boolean). |
| - Max Completion Tokens | Maximum tokens generated in the chat completion (controls cost). |
| - Maximum Number of Tokens | Maximum tokens to generate in the completion (default 2048, up to 131072). |
| - Metadata | Developer-defined metadata attached to the completion (JSON object). |
| - Number of Completions | How many completion choices to generate per input message (1 to 128). |
| - Presence Penalty | Penalizes new tokens based on whether they appear in the text so far (-2.0 to 2.0). |
| - Stop Sequences | Up to 4 sequences where generation will stop (comma-separated strings). |
| - Stream | Whether partial message deltas are sent (like ChatGPT streaming). |
| - Stream Options | Options for streaming response, e.g., include usage info before done message. |
| - Temperature | Controls randomness of completions (0 to 2). Lower values make output more deterministic. |
| - Tool Choice | Controls if and how the model calls external tools: Auto, None, Required, or specify a particular function. |
| - Tool Choice Function Name | Name of the function to call when tool choice is set to "function". |
| - Tools | List of tools the model may call, each defined by type (currently only "function"), name, description, and JSON schema parameters. |
| - Top Logprobs | Number of most likely tokens to return at each position (0 to 20), requires logprobs enabled. |
| - Top P | Nucleus sampling parameter (0 to 1) as an alternative to temperature. |
| - User Identifier | Unique identifier representing the end-user, useful for monitoring and abuse detection. |
Output
The node outputs JSON data containing the AI-generated completions. The structure typically includes:
- One or more completion choices, each with generated text or message content.
- Usage statistics such as token counts (if available).
- Optionally, log probabilities and other metadata depending on input options.
- If streaming is enabled, partial message deltas are emitted progressively.
If binary data were supported (not indicated here), it would represent media or files generated by the model, but this node focuses on textual completions.
Dependencies
- Requires an API key credential for DigitalOcean Gradient™ AI Serverless Inference.
- Network access to the DigitalOcean inference endpoint (
https://inference.do-ai.run/v1). - n8n environment configured with the appropriate API authentication credentials.
- Optional: JSON parsing support for properties like
logitBias,metadata, and tool parameters.
Troubleshooting
- Invalid Model Selection: Selecting a model not available or unsupported by the API will cause errors. Ensure the model list is refreshed and a valid model is chosen.
- Malformed JSON Inputs: Properties like
logitBias,metadata, and tool parameters require valid JSON strings. Invalid JSON will cause parsing errors. - Token Limits Exceeded: Setting
maxTokensormaxCompletionTokensbeyond model limits may result in errors or truncated output. - Streaming Issues: Enabling streaming without proper handling in downstream nodes or UI may cause incomplete data display.
- Tool Configuration Errors: Incorrectly specifying tool functions or parameters may lead to failures in tool invocation.
- API Authentication Failures: Missing or invalid API keys will prevent successful requests.
To resolve these issues:
- Validate JSON inputs carefully.
- Use recommended token limits per model documentation.
- Confirm API credentials are correctly configured.
- Review error messages returned by the API for specific guidance.