0 votes
I have data set by name weather_data , i want to add data everyday to this dataset

How can i do this with python?
asked by dev
What do you mean with this? Do you want to create a new column in the data set that is named 'everyday'?
I think his intention was to append the data to existing data set (same schema).

2 Answers

0 votes
Of course, now that I read ot after a good sleep I see what you mean, thanks dev. It would depend on how you have your new data, I guess the easiest way will be with the panda's append function. I believe this page will help you: https://stackoverflow.com/questions/14988480/pandas-version-of-rbind
answered by
0 votes
You may want to use

- the append mode on the output dataset. This setting is available in the input/output tab of the recipe =>  The data with be append in the output dataset. Note that this mode is only available on recipe with output dataset using an infrastructure allowing append (e.g. it is not possible with HDFS)

- the partition on the output dataset. Each day you will write your weather data in the partition of that day on the dataset. This mode works whatever the connection. See https://doc.dataiku.com/dss/latest/partitions/index.html for more details.
answered by
620 questions
626 answers
458 comments
375 users