Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
I'd like to access SQL output generated by a recipe programatically using Python API client.
Is that possible?
UPDATE:
I see this information is returned by /dip/api/flow/recipes/generic/get-status
Hi,
You find below a small code snippet from which you can inspire to apply to your use case:
# HELPER FUNCTION
CODE_RECIPES = ['sql_script', 'sql_query', 'impala', 'hive', 'spark_sql']
def get_recipe_sql(self, project_handle, recipe_name, recipe_type):
recipe_handle = project_handle.get_recipe(recipe_name)
if recipe_type in CODE_RECIPES:
dap = recipe_handle.get_definition_and_payload()
return dap.get_payload()
else:
status = recipe_handle.get_status()
return status.data.get('sql')
# GO
client = dataiku.api_client()
project_handle = client.get_project('MY_PROJECT')
recipes = project_handle.list_recipes()
#loop through every recipe of the project and print out the SQL if possible
for recipe in recipes:
recipe_name = recipe.get('name')
recipe_type = recipe.get('type')
try:
sql = get_recipe_sql(project_handle, recipe_name, recipe_type)
except Exception as e:
message = "Could not retrieve sql code from recipe {} in project {}: {}"
print(message.format(recipe_name, project, e))
continue
Cheers,