How do I fetch date from a filename and add it to a column in the file using DSS?

UserBird
Dataiker
How do I fetch date from a filename and add it to a column in the file using DSS?
For e.g. if the filename is Test_20170101 and the file has 3 columns test1, test2 and test3. I want my file now to have 4 columns - test1, test2 ,test3 and the new column date, with the value 20170101. How do I do this in DSS ?
5 Replies
AdrienL
Dataiker

Unfortunately there is no built-in feature to do this, you'd have to use a python recipe.



If you do this kind of things, you may want to look into partitioning.

0 Kudos
UserBird
Dataiker
Author
Is this still the case? I am trying to get source's latest partition date using following python code -

file_date = dataiku.dku_flow_variables["DKU_SRC_LAST_DATE"]

I was wondering if you have built-feature now after 1 and 1/2 year.
0 Kudos
AdrienL
Dataiker
There is still no built-in feature to do that.
0 Kudos
rmnvncnt
Level 3
Any update on this topic? Partitioning doesn't work in my case (since "Missing partitions as empty" is still not supported for discrete + time partitioning). I could create a new connector for each file in the folder, but this doesn't scale at all. Being able to add the source to the dataset could solve this problem.
0 Kudos
AdrienL
Dataiker
I'm not sure I understand your use case fully. I suggest you contact you Customer Success Manager and provide him/her with the details of what you need for a more tailored recommendation.
0 Kudos