'Append instead of overwrite' does not append table after recipe

Question

Hi,

I am trying to use the functionality 'Append instead of overwrite' to store an output after writing a recipe. However, I don't see the results being appended to the table. Instead, Dataiku performs truncate table and then invokes insert table command.

When I use sync functionality, 'Append instead of overwrite' works. New rows are always added to the table.

Is this a bug within Dataiku?

P

Comments

I have an update regarding this question. dataiku support mentioned that currently it is not possible to append to datasets using a Spark recipes

Answer 1

Hi,

What is the kind of recipe and kind of output that you are trying to append to ?

Comments

Hi,

The flow is as follows:
Input is a dataset via a file upload. Then, it connects to a 'prepare' recipe, which performs column renaming and deleting. Then the result of this prepare recipe is supposed to be appended to an Oracle database.

When I only use 'sync', the result is successfully appended to the table.