Databricks icon

Databricks

Interact with Databricks API
Join our community

Actions42

Overview

This node interacts with the Databricks API to manage files within the Unity Catalog environment. Specifically, for the Files resource and the Get File Metadata operation, it allows users to retrieve metadata information about a file stored in a specified volume of a Unity Catalog schema.

Common scenarios where this node is beneficial include:

  • Automating data governance workflows by programmatically accessing file metadata.
  • Integrating Databricks file metadata retrieval into ETL or data pipeline processes.
  • Monitoring or auditing files stored in Unity Catalog volumes without manual intervention.

For example, a user can select a catalog, schema, and volume, specify a file path, and retrieve metadata such as file size, creation date, or other attributes exposed by the Databricks API.

Properties

Name Meaning
Catalog Select a Unity Catalog to access files from.
Schema Select a schema from the chosen catalog.
Volume Select a volume from the chosen catalog and schema.
File Name Name of the file (e.g., "myfile.txt" or "folder/myfile.txt") for which to get metadata.

The first three properties are dropdowns dynamically loaded from the Databricks API to help users pick valid catalogs, schemas, and volumes. The "File Name" is a string input specifying the exact file path within the selected volume.

Output

The output JSON contains the metadata information retrieved from the Databricks API about the specified file. This typically includes details such as file size, timestamps, permissions, and other relevant attributes depending on what the API returns for file metadata.

No binary data output is involved in this operation.

Example output structure (simplified):

{
  "fileName": "myfile.txt",
  "size": 12345,
  "createdAt": "2024-01-01T12:00:00Z",
  "modifiedAt": "2024-01-02T15:30:00Z",
  "owner": "user@example.com",
  "permissions": "read-write"
}

(Note: Actual fields depend on the Databricks API response.)

Dependencies

  • Requires an active Databricks account with appropriate permissions to access Unity Catalog resources.
  • Needs an API authentication token credential configured in n8n to authorize requests.
  • The node uses the Databricks REST API endpoints under /api/2.1/unity-catalog and /api/2.0/fs/files for loading options and file operations.
  • Network connectivity to the Databricks workspace host URL is necessary.

Troubleshooting

  • Common Issues:

    • Incorrect or missing API authentication token will cause authorization failures.
    • Selecting a catalog, schema, or volume that does not exist or to which the user lacks access will result in errors.
    • Specifying an invalid or non-existent file path will lead to "file not found" or similar errors from the API.

  • Error Messages:

    • API Error: <status> <statusText>: Indicates the API returned an error status code. Check credentials, permissions, and parameter correctness.
    • Network Error: No response received from server: Suggests network issues or incorrect host URL configuration.
    • Validation errors if required parameters like Catalog, Schema, Volume, or File Name are missing.

  • Resolution Tips:

    • Verify API token validity and permissions in Databricks.
    • Confirm the existence and spelling of Catalog, Schema, Volume, and File Name.
    • Ensure the n8n node's base URL matches your Databricks workspace URL.
    • Use the node's debug logs to inspect request URLs and payloads.

Links and References

Discussion