0 votes
Hi, is there a way to limit the log size when running a job? Sometimes we run a job and that fails. When this happens all temp data and logging is stored which leads to disk space issues. In no time logging of over 100GB is created.
asked by

1 Answer

0 votes
Hi Martin,

You can perform such scheduled operations through the use of macros: https://doc.dataiku.com/dss/latest/operations/macros.html. As a matter of fact, we provide a built-in macro to clear old job logs. This macro can be run manually or scheduled through a scenario (if you have production capabilities activated in your license).

Note that this clears existing logs, so you may want to store them elsewhere for audit or compliance reasons.

Hope it helps,

Alex
answered by
992 questions
1,026 answers
1,079 comments
3,222 users

┬ęDataiku 2012-2018 - Privacy Policy