Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
My partition is defined like this:
participant='morella'/p_year='2022'/p_month='03'/p_day='01'/.*
with the tricky part that participant value is dynamic and is provided in the dataframe I want to write.
participant values can be: 'morella', 'irama', 'akhila', 'lucia', ...etc.
Thanks for you help
Operating system used: linux AWS
Hi,
You can mix discrete and time-based partitions this will likely create a high number of partitions. So probably sticking with just time time-based partitions should suffice and you can use wildcards for participants and then enrich with file info if needed to add participants back to the dataset: https://doc.dataiku.com/dss/latest/preparation/processors/enrich-with-record-context.html
Thanks