0 votes
Hi,

 I would like to move from design node to automation node a project using a python script. Does Dataiku have a template or existing script for that? After I export a bundle how can I manage mappings and conflicts during the activate?

Thanks!
asked by

2 Answers

+1 vote

Hi, 

Please find attached an example script for exporting a bundle from a design node and deploying it to automation.

import dataikuapi

#Define the connections

#Design
host = "http://localhost:12345" # example to be changed
apiKey = "Foo123Foo123F123Foo123" # example to be changed

#Automation
host_auto = "http://localhost:23456" # example to be changed
apiKey_auto = "Bar234Bar234Bar234Bar234" # example to be changed

design_client= dataikuapi.DSSClient(host, apiKey)
auto_client = dataikuapi.DSSClient(host_auto, apiKey_auto)

version_bundle = "bundle_v1"
#Export bundle
project = design_client.get_project("MYSUPERPROJECT")
project.export_bundle(version_bundle)
project.download_exported_bundle_archive_to_file(version_bundle, "temp_bundle.zip")

#Import bundle
project_automation = auto_client.get_project("MYSUPERPROJECT")
project_automation.import_bundle_from_archive("temp_bundle.zip")

# Preload and activate bundle
project_automation.preload_bundle(version_bundle) # for code envs
project_automation.activate_bundle(version_bundle)

 

Note that it works also outside of DSS, which is why you need the API keys. To generate these, you can go to Administration > Security > Global/Personal API keys on the Design and Automation nodes.

At the moment, there is no way through the dataiku API to perform custom remapping of the connections. We assume that the connections are already setup and share the same logical name (although their inner values are different). Having said that, if you have already setup the custom remapping in the interface, promoting a new bundle through the API will keep the custom remapping.

Cheers,

Alex

answered by
I assume that "temp_bundle.zip" is downloaded locally in this example, but is there a way to push the bundle from the design node onto the automation node directly?
yes, you can use the template code locally on your design node:
import dataiku
design_client= dataiku.api_client()
For the download_exported_bundle_archive_to_file method, I advise using the Python library tempfile so that you download it to a temporary location. Otherwise, for archiving purposes, you could use a specific directory like /home/dataiku/project_bundles.
Alright, thanks Alex!
0 votes
Hi tomas,

You can package a project into a "bundle" and deploy it from the Design Node to the Automation Node.

Some links:

https://www.dataiku.com/learn/guide/tutorials/deploy-production.html
https://doc.dataiku.com/dss/latest/bundles/index.html
answered by
Yes, I am using bundle python script.
p = designClient.get_project("MYPROJECT")
p.export_bundle(bundleId)
Oh, I didn't understand you want to automate that with a Python script. Sorry.
947 questions
978 answers
989 comments
2,132 users

┬ęDataiku 2012-2018 - Privacy Policy