Overview
This node acts as an AI Model Load Balancer that balances requests across two connected AI models. It is useful in scenarios where load distribution between two AI models is needed to optimize performance or availability. For example, it can be used to alternate requests between two different AI chat or generation models to ensure even usage and reduce the risk of overloading a single model.
Use Case Examples
- Alternating chat requests between two AI models to balance load.
- Distributing text generation requests evenly across two AI models.
Properties
| Name | Meaning |
|---|---|
| Request Options | Settings to control request batching, SSL certificate validation, proxy usage, and timeout duration. |
Output
JSON
response- The response from the selected AI model after processing the request.
Dependencies
- Requires two connected AI model nodes as inputs to balance requests between them.
Troubleshooting
- Error 'Could not find a connected model at input index X' indicates that one of the required AI model inputs is not connected. Ensure both models are connected to the node inputs.
- Error 'The selected model does not support chat/generation' means the chosen AI model does not have the required method. Verify the connected models support the intended operation.