Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Hi,
I've create a custom recipe with a Dataset and Managed Folder as input and a Dataset as output. The recipe runs well on DSS but it doesn't work when its configured to run on a Containerized Execution. These are the steps i took and also the error i received after running the receipt.
Setting up
Using it
Error received:
FileNotFoundError: [Errno 2] No such file or directory: '/mtn/...some path/57CFjKsj'
As far as i can tell, the Containerized Execution cannot see the files in Managed Folder. Could you advise what is going on?
Thank you.
Hi,
To run a recipe on Containerized Execution when using managed folders as input/output, you will need to use the Dataiku API for reading/writing files.
In other words, the "regular" local filesystem API which you were using with local execution cannot work anymore. Hence, you will need to use the following methods to interact with the managed folder:
The same APIs also exist in R, as documented here:
Hope it helps,
Alex Combessie
Hi @Alex_Combessie , thanks for that. That's really helpful. I have another question - how to remove the files in the managed folder on containerised execution?