0 votes

I run my filter recipe periodically. I would like to use append instead of overwrite to store only new tweets (not what is already stored with the recipe output). Is it possible?

by

1 Answer

0 votes

Hi,

this looks like a use case for partitioning. Note this is an advanced concept of DSS.

Imagine one uses "append", discovers a bug, fixes it, runs the flow again: the output dataset would then contain both the buggy and the correct data.

by
Do you have an example of how can do that?
1,194 questions
1,226 answers
1,377 comments
11,754 users

┬ęDataiku 2012-2018 - Privacy Policy