Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Hi again,
I've a managed folder under HDFS and want to create a subfolder under it but cannot find under the
API documentation the method or the function to do it. How can this be done ? It should be possible
because from the UI that is possible...
Thanks. Regards.
Giuseppe
since the notion of folder does not exist for all connection types that you can put managed folders on (as in: it doesn't exist for cloud storages), the is no API to directly create directories inside managed folders. The directories are automatically created as needed when you write a file. For example calling https://doc.dataiku.com/dss/latest/python-api/managed_folders.html#dataiku.Folder.upload_data with `upload_data("/subdir1/subdir2/stuff", [])` will create the sub-directories subdir1 and subdir2 if they don't exist. You could write empty files to force directories to be created.
since the notion of folder does not exist for all connection types that you can put managed folders on (as in: it doesn't exist for cloud storages), the is no API to directly create directories inside managed folders. The directories are automatically created as needed when you write a file. For example calling https://doc.dataiku.com/dss/latest/python-api/managed_folders.html#dataiku.Folder.upload_data with `upload_data("/subdir1/subdir2/stuff", [])` will create the sub-directories subdir1 and subdir2 if they don't exist. You could write empty files to force directories to be created.
Hi fchataigner2,
the upload_data() only exists in dataiku package. How can I do it outside of DSS? In the dataikuapi package there is no this method but put_file(). However put_file() doesn't create the subdirectory automatically like upload_data().
Here is a function that addresses your need
def make_dir(managed_folder_id, subfolder_rel_path):
"""
create a subfolder in a folder
subfolder_path is relative path such as "subfolder/"
or "my/subfolder/path/"
the function will create all necessary subfolders if several are nested and not created yet
if a folder already exists, the function will not create it, and not delete the content of it
"""
# Get the folder
client = dataiku.api_client()
project = client.get_default_project()
folder = project.get_managed_folder(managed_folder_id)
# Create an empty file called "empty_file.txt" in the current working directory
file_path = "empty_file.txt"
open(file_path, "w").close()
subfolder_rel_path_with_file = os.path.relpath(
os.path.join(subfolder_rel_path, "empty_file.txt")
)
# Put the file into the DSS managed folder
contents = folder.put_file(subfolder_rel_path_with_file, os.path.abspath(file_path))
# delete the file that has been copied to the subfolder
folder.delete_file(subfolder_rel_path_with_file)