Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Hi @JBR ,
1) If you have partitions with "gaps" you can use the option : "Missing partitions as empty"
2) Indeed if there are major changes in the schema or recipes you would need to rebuild all partitions.
With the option from 1, it should ignore any gaps in your time based partitions. I don't understand why you want to append exactly. Each partition should be built independently and you would inherently append new partitions you run if your flow is partitioned.
To run for the last 7 days you would simply want to have all partition dependencies set to equal and in the scenario build PREVIOUS_DAY and CURRENT_DAY and run this daily for example. If you need to run last 7 days you would need to use variables that you set or use code.
Thanks,