Databricks icon

Databricks

Interact with Databricks API

Overview

The "Upload File" operation of the Databricks Files resource allows users to upload a file directly into their Databricks workspace at a specified path. This node is useful for automating the process of adding or updating files in Databricks, such as scripts, configuration files, or data files needed for processing within the platform.

Common scenarios include:

  • Uploading ETL scripts or notebooks programmatically.
  • Adding JSON or XML configuration files required by Databricks jobs.
  • Storing images or binary data for use in machine learning workflows.
  • Automating file management tasks within Databricks workspaces.

Example: Upload a JSON configuration file to /Volumes/my-catalog/my-schema/my-volume/config/settings.json with content type application/json.

Properties

Name Meaning
Path The full path in the Databricks workspace where the file will be uploaded, e.g., /Volumes/my-catalog/.../file.txt.
File Contents The actual contents of the file to upload, provided as a string.
Content Type The MIME type of the file being uploaded. Options include:
- application/octet-stream (binary data)
- text/plain (plain text)
- application/json (JSON data)
- application/xml (XML data)
- image/jpeg (image data like PNG, JPEG)
Additional Fields - Overwrite: Boolean flag indicating whether to overwrite an existing file at the target path (default: false).

Output

The node outputs JSON data representing the result of the upload operation. Typically, this includes metadata about the uploaded file such as its path, size, and possibly confirmation of success.

If the node supports binary data output, it would represent the uploaded file's binary content or related binary response, but based on the static analysis, the primary output is JSON metadata confirming the upload.

Dependencies

  • Requires an API authentication token credential to access the Databricks API.
  • Needs the base URL of the Databricks instance configured in credentials.
  • No additional external dependencies beyond standard HTTP requests to the Databricks API.

Troubleshooting

  • File path errors: Ensure the specified path is valid and accessible within the Databricks workspace. Invalid paths may cause upload failures.
  • Overwrite conflicts: If Overwrite is set to false and a file already exists at the path, the upload will fail. Set Overwrite to true to replace existing files.
  • Authentication issues: Verify that the API token and host URL are correctly configured in the node credentials.
  • Content type mismatches: Make sure the Content Type matches the actual file content to avoid processing errors downstream.

Links and References

Discussion