Overview
This node integrates with OpenAI GPT models to generate AI-powered text completions and streams responses in real-time. It is designed to send a user-defined message along with a system prompt to guide the model's behavior, then stream partial results as they are generated, followed by a final complete response.
Common scenarios include:
- Interactive chatbots that require streaming partial replies for better user experience.
- Automated content generation where immediate feedback on the generation progress is useful.
- Applications needing fine control over model parameters like temperature and token limits.
For example, you can use this node to build a customer support assistant that streams answers as they are generated, improving responsiveness and engagement.
Properties
| Name | Meaning |
|---|---|
| Model | The OpenAI GPT model to use. Options: GPT-4 Turbo, GPT-4, GPT-3.5 Turbo, GPT-4o, GPT-4o Mini |
| System Prompt | A prompt that sets the context or behavior of the AI assistant (e.g., "You are a helpful AI assistant.") |
| Message | The JSON-formatted message to send to the model, typically including role and content fields |
| Temperature | Controls randomness in the output; ranges from 0 (deterministic) to 2 (very random) |
| Max Tokens | Maximum number of tokens to generate in the completion |
| Base URL | Optional custom base URL for OpenAI-compatible APIs; leave empty to use standard OpenAI endpoints |
| Advanced Options | Collection of optional settings: |
| - Message State | Overrides the state label for streamed message chunks; options include Default (auto), Thinking, Responding, Active, Waiting, Complete |
| - Progress Message | Optional string to include as a progress message alongside the state during streaming |
Output
The node produces two outputs:
Stream: An array of streamed chunks representing partial responses from the OpenAI model. Each chunk includes formatted data about the current state of the message (e.g., responding, complete) and optionally a progress message. This allows downstream nodes or UI components to display incremental results as they arrive.
Result: A single final JSON object containing the complete response from the model, including metadata such as the full text, token usage, and original messages sent. This output is suitable for further processing or storage.
The json field in both outputs contains structured information about the AI-generated text and related metadata.
The node does not output binary data.
Dependencies
- Requires an API key credential for authenticating with OpenAI or compatible services.
- Supports specifying a custom base URL to connect to OpenAI-compatible APIs beyond the official OpenAI service.
- Uses internal helper functions for formatting streamed chunks and final responses.
Troubleshooting
- Invalid API Key or Authentication Errors: Ensure the provided API key credential is valid and has appropriate permissions.
- Malformed JSON in Message Property: The
Messageinput expects valid JSON. Invalid JSON will cause parsing errors. - Timeouts or Network Issues: Streaming relies on stable network connections; intermittent connectivity may disrupt streaming.
- Unsupported Model Selection: Selecting a model not supported by your API plan or endpoint may result in errors.
- Exceeding Token Limits: Setting
Max Tokenstoo high might cause request failures depending on model constraints.
If errors occur, the node returns formatted error objects on both outputs, which can be inspected for troubleshooting.