Databricks icon

Databricks

Interact with Databricks API
Join our community

Actions42

Overview

This node integrates with the Databricks API, specifically supporting multiple resources including Unity Catalog. For the Unity Catalog - List Catalogs operation, it retrieves and lists catalogs available in your Databricks workspace. This is useful for workflows that need to dynamically access or manage data governance structures such as catalogs and schemas within Databricks.

Practical examples include:

  • Automating data catalog discovery before running queries.
  • Dynamically populating dropdowns in subsequent workflow steps based on available catalogs.
  • Integrating with data lineage or metadata management systems by fetching catalog information programmatically.

Properties

Name Meaning
Catalog Select a Unity Catalog to work with. The list is automatically populated from your Databricks workspace.
Schema Select a schema from the chosen catalog. The list is dynamically loaded based on the selected catalog.

The Catalog property offers options fetched via a GET request to /api/2.1/unity-catalog/catalogs, listing all catalogs with their names and optional comments.

The Schema property depends on the selected Catalog and fetches schemas via /api/2.1/unity-catalog/schemas?catalog_name={{catalog}}.

Output

The output JSON structure contains the retrieved catalog or schema information depending on the operation:

  • For List Catalogs, the output includes an array of catalog objects, each typically containing properties like name and optionally comment.
  • For List Schemas (implied by the schema property loading), the output includes an array of schema objects with similar descriptive fields.

No binary data output is involved in this operation.

Example output snippet for catalogs might look like:

{
  "catalogs": [
    {
      "name": "main_catalog",
      "comment": "Primary data catalog"
    },
    {
      "name": "analytics_catalog",
      "comment": ""
    }
  ]
}

Dependencies

  • Requires an active Databricks workspace with Unity Catalog enabled.
  • Needs an API authentication token credential configured in n8n to authorize requests.
  • The node uses the Databricks REST API endpoints under /api/2.1/unity-catalog/.
  • Network connectivity to the Databricks host specified in credentials.

Troubleshooting

  • Authentication errors: Ensure the API token credential is valid and has sufficient permissions to access Unity Catalog APIs.
  • Empty catalog or schema lists: Verify that Unity Catalog is enabled and catalogs/schemas exist in your Databricks workspace.
  • Network issues: Check firewall or proxy settings that may block access to the Databricks API endpoint.
  • API rate limits or permission denied errors: Confirm your user role and API quota in Databricks.

Common error messages:

  • "API Error: 401 Unauthorized" — Invalid or expired API token.
  • "API Error: 403 Forbidden" — Insufficient permissions for the requested resource.
  • "Network Error: No response received from server" — Connectivity issues or incorrect host URL.

Links and References

Discussion