Dataiku DSS icon

Dataiku DSS

Use the Dataiku DSS API

Actions364

Overview

This node integrates with the Dataiku DSS API, allowing users to perform a wide range of operations on various Dataiku DSS resources. Specifically for the Dataset resource and the List Partitions operation, it retrieves the list of partitions for a specified dataset within a project.

This is useful in scenarios where you need to programmatically explore or manage dataset partitions in Dataiku DSS, such as automating data workflows that depend on partition metadata or orchestrating downstream processing based on available partitions.

Example use case:
You have a large dataset partitioned by date, and you want to list all available partitions to trigger processing only on recent partitions or to audit partition availability.

Properties

Name Meaning
Project Key The unique key identifying the Dataiku DSS project containing the dataset.
Dataset Name The name of the dataset whose partitions you want to list.

These properties are required inputs to specify which dataset's partitions should be listed.

Output

The output is a JSON array where each item corresponds to a partition of the specified dataset. The exact structure depends on the Dataiku DSS API response for dataset partitions but typically includes partition keys and metadata describing each partition.

If the operation involves downloading files (not applicable here), binary data would be returned; however, for listing partitions, the output is purely JSON.

Dependencies

  • Requires an active connection to a Dataiku DSS instance.
  • Requires an API key credential for authenticating requests to the Dataiku DSS API.
  • The node expects the Dataiku DSS server URL and user API key to be configured in the credentials.

Troubleshooting

  • Missing Credentials Error: If the API key credential is not provided or invalid, the node will throw an error indicating missing credentials.
  • Required Parameter Missing: The node validates required parameters like Project Key and Dataset Name. Omitting these will cause errors specifying which parameter is missing.
  • API Request Failures: Network issues, incorrect server URL, or insufficient permissions can cause API call failures. Check connectivity and credential permissions.
  • Unexpected Response Format: If the API returns unexpected data, parsing errors may occur. Ensure the Dataiku DSS API version is compatible.

Links and References

These links provide further details on dataset partitions and API usage in Dataiku DSS.

Discussion