Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
I need some help to figure out a few things with Dataiku to see if it will work for our needs. We have a different configuration than what Dataiku suggests as far as code environments and containerization, so if "this isn't possible" is the answer, so be it.
Long and short, based on our software development life cycle, we have a series of security scans that we require to be run as part of our pipelines when we're making docker images in general. As a result of this requirement of the SDLC, we have done two things in particular with our installation:
So the container creation process fits within our SDLC requirements, but we've hit an interesting situation in trying to get Dataiku to understand that there are newer containers available. The example requires some explanation:
Containerized Execution setup:
Config Name: gitlab-test
Image Registry URL: registry.gitlab.com/made-up-company/made-up-docker-folder
Base Name: int:latest
On first Test from the GUI with gitlab-test, Dataiku doesn't find the listed image locally, so it finds registry.gitlab.com/made-up-company/made-up-docker-folder/int:latest (number tag 200001) at the URL, downloads it correctly, and is immediately ready for use in Dataiku.
However, lets say that after that first pull Dataiku does, there's now a NEW version of registry.gitlab.com/made-up-company/made-up-docker-folder/int:latest (number tag 200002) that was made on Gitlab.
When I do the same process as I did for gitlab-test before, Dataiku first looks locally and finds the old image tag 200001 it pulled before using int:latest definition, but doesn't bother to look again at the gitlab registry for new image tag 200002, even though this new tag is now the true int:latest.
Is there a way to change the behavior of Dataiku to instead check the registry first for newer images, or does anyone have any ideas on how to automate docker image pulls via Dataiku?
Thanks!