I need to create a lot of similar datasets

Solved!
UserBird
Dataiker
I need to create a lot of similar datasets
Is there a way to automate it?
1 Solution
jrouquie
Dataiker Alumni
If they come from SQL, go to
โ€œAdministration โ†’ Connectionsโ€, click the SQL connection and use โ€œMass datasets creationโ€.

If they are on Hadoop and described in a Hive/Impala metastore, use the script DATA_DIR/bin/dku hive-import.

If this doesn't suit your need, you might want to script the generation of the .json files describing the datasets (take existing datasets as example). Those files are in `DATA_DIR/config/projects/PROJECT_KEY/datasets/`. Please note that the exact json format might change between DSS versions. However, DSS gets the list of existing datasets simply by listing the contents of this directory, and this should be stable for some time.

View solution in original post

0 Kudos
1 Reply
jrouquie
Dataiker Alumni
If they come from SQL, go to
โ€œAdministration โ†’ Connectionsโ€, click the SQL connection and use โ€œMass datasets creationโ€.

If they are on Hadoop and described in a Hive/Impala metastore, use the script DATA_DIR/bin/dku hive-import.

If this doesn't suit your need, you might want to script the generation of the .json files describing the datasets (take existing datasets as example). Those files are in `DATA_DIR/config/projects/PROJECT_KEY/datasets/`. Please note that the exact json format might change between DSS versions. However, DSS gets the list of existing datasets simply by listing the contents of this directory, and this should be stable for some time.
0 Kudos

Labels

?
Labels (3)
A banner prompting to get Dataiku