0 votes
I want to append data everyday after to manipulate it.
by

1 Answer

0 votes

Some datasets supports append, some not.  Note that this mode is only available on recipe with output dataset using an infrastructure allowing append (e.g. it is not possible with HDFS).

Or you can use partitioning and put new data every day into a new partition.

First you need to activate partitioning on the dataset. You need to choose between time based or discrete partitioning. If you choose time base partitioning, then you can partition by year, year+month, or year+month+day.

Or you can partition using any discrete value, for example you choose partitioning by City, or by Country.

Then when you have activated the partitioning on the dataset, the icon should change.

After that you need to enter into the parent recipe and define the partition range what you want to calculate:

 

by
edited by
What are the datasets that support append?
How can put new data into a new partition?
Sorry, I'm noob in Dataiku
see my updated answer.
Hi, and what are the datasets that support append?
I want to schedule my flow everyday, and the data result of my flow I want to append with the previous data result, because I must to create a report with this data.
Thanks
1,258 questions
1,292 answers
1,467 comments
11,809 users

┬ęDataiku 2012-2018 - Privacy Policy