Heroku AI Chat Model icon

Heroku AI Chat Model

Language Model for Heroku AI (Managed Inference and Agents)
Join our community

Overview

This node integrates with Heroku AI's language models to generate text completions based on user input. It is designed for scenarios where users want to leverage advanced AI models hosted on Heroku for tasks such as content generation, summarization, question answering, or conversational agents. The node connects specifically to AI Agent or AI Chain nodes, enabling it to be part of larger AI workflows.

Practical examples include:

  • Generating creative writing or marketing copy using a selected AI model.
  • Automating customer support responses by integrating with chatbots.
  • Performing data analysis or summarization through extended reasoning capabilities.

Properties

Name Meaning
(notice) Informational notice indicating the node must connect to AI Agent or AI Chain nodes.
Model The specific Heroku AI model to use for generating completions. Options include various Claude and GPT models.
Temperature Controls randomness in the output; lower values produce more focused and deterministic results.
Maximum Tokens Maximum number of tokens the model will generate in the completion.
Frequency Penalty Penalizes repeated tokens to reduce repetition; higher values mean less repetition.
Presence Penalty Penalizes tokens that have already appeared to encourage new topics; higher values promote novelty.
Top P Nucleus sampling parameter; considers only tokens within the top P probability mass.
Options Additional settings:
- Streaming Whether to stream the response incrementally as it is generated (true/false).
- Extended Thinking Enables extended reasoning mode for more thorough analysis (true/false).
- Timeout Request timeout duration in milliseconds.

Output

The node outputs JSON data under the field json containing the AI-generated completion result from the selected Heroku AI model. This typically includes the generated text response. The node does not output binary data.

Dependencies

  • Requires an API key credential for Heroku AI service authentication.
  • Needs network access to Heroku AI inference API endpoints.
  • The node uses the heroku-langchain package internally to interact with the Heroku AI models.
  • Proper configuration of the API base URL and API key is necessary in n8n credentials.

Troubleshooting

  • Common Issues:

    • Failure to fetch available models due to incorrect or missing API credentials.
    • Request timeouts if the specified timeout is too low or network issues occur.
    • Errors when selecting unsupported or unavailable models.

  • Error Messages:

    • "Failed to fetch models: <error message>": Indicates problems retrieving model list; verify API key and network connectivity.
    • Timeout errors: Increase the timeout value in options or check network stability.

  • Resolutions:

    • Ensure valid API key credential is configured.
    • Confirm the API base URL is correct.
    • Adjust timeout settings if requests are timing out.
    • Select supported models from the provided list or enter a valid model ID.

Links and References

Discussion