ComfyUI Text To Video icon

ComfyUI Text To Video

Generate video from text prompt using ComfyUI
Join our community

Overview

This node generates a video from a text prompt using ComfyUI by submitting a user-defined workflow in JSON format. It sends the workflow to the ComfyUI API, queues the video generation, and polls for completion within a specified timeout period. Once the video is ready, it retrieves the video file and outputs it.

Common scenarios include:

  • Automating video creation workflows based on textual descriptions.
  • Integrating AI-driven video generation into larger automation pipelines.
  • Quickly prototyping video content from dynamic prompts.

Practical example:

  • A marketing team could input a JSON workflow describing a video concept and receive a generated video clip without manual editing.
  • Content creators can automate generating short videos from script snippets or storyboards encoded as JSON workflows.

Properties

Name Meaning
Workflow JSON The ComfyUI workflow described in JSON format that defines how the video should be generated. Must be valid JSON.
Timeout Maximum time in minutes to wait for the video generation process before timing out. Default is 30 minutes.

Output

The node outputs an array of items where each item contains:

  • json:

    • prompt_id: The unique identifier for the submitted video generation request.
    • filename: The name of the generated video file.
    • status: Status string, typically "success" upon successful generation.

  • binary:

    • video: Contains the generated video data encoded in base64.
      • data: Base64-encoded video content.
      • mimeType: Always "video/mp4".
      • fileName: The filename of the video.

This allows downstream nodes to access both metadata and the actual video binary data.

Dependencies

  • Requires access to a ComfyUI API endpoint with a valid URL and an API key credential for authentication.
  • The node expects the API to support:
    • POST /prompt to submit workflows.
    • GET /history/{prompt_id} to poll status.
    • GET /view?filename={filename} to download the video file.
  • The API key must be provided via node credentials.
  • Network connectivity to the ComfyUI service is essential.

Troubleshooting

  • Invalid workflow JSON: If the workflow JSON is malformed or not a valid object, the node throws an error prompting to check JSON syntax.
  • Failed to get prompt ID: Indicates the API did not return a prompt ID after submission; verify API availability and credentials.
  • Video generation timed out: Occurs if the video is not ready within the specified timeout. Increase timeout or check API responsiveness.
  • No video output found: The generation completed but no video was returned; possibly due to workflow issues or API errors.
  • Polling errors: Temporary network or API errors during polling are logged and retried, but persistent failures cause the node to error out.
  • API authorization errors: Ensure the API key is correct and has sufficient permissions.

Links and References

Discussion