0 votes

I run my filter recipe periodically. I would like to use append instead of overwrite to store only new tweets (not what is already stored with the recipe output). Is it possible?

asked by

1 Answer

0 votes

Hi,

this looks like a use case for partitioning. Note this is an advanced concept of DSS.

Imagine one uses "append", discovers a bug, fixes it, runs the flow again: the output dataset would then contain both the buggy and the correct data.

answered by
945 questions
977 answers
986 comments
2,099 users

┬ęDataiku 2012-2018 - Privacy Policy