Dataiku deployment on Kubernetes Cluster

[gizmo1177]

I have general question related to the Dataiku in a Kubernetes Cluster. It is more to verify if my understanding is correct. Please correct me in case I am wrong.

In general:

• To use Dataiku in a Kubernetes Cluster we first have to create a base image (https://doc.dataiku.com/dss/latest/apinode/kubernetes/index.html). This base image is used later to initialize new api nodes in the Cluster.
• A load balancer can dispatch the queries to the API nodes (https://doc.dataiku.com/dss/latest/apinode/operations/ha-deployment.html).
• Changes in the projects can be deployed by using the DSS UI. The API Deployer is responsible to distribute the changes to the nodes.

Is this correct?

[JordanB · Accepted solution]

Hi @gizmo1177,

This is correct! Note, when deploying your api service on a Kubernetes cluster, the load balancer is configured within your k8s Infrastructure in DSS UI under "service exposition". When you choose "load balancer" as your exposition mode, a load balancer annotation is applied to the manifest for the service and will be deployed with your api service to dispatch traffic.  

Thanks!

Jordan

[SaiNaveen1]

Hi Team,

I their any documentation on how to setup Load Balancer in Dataiku? 
If yes, please do share it here.

[JordanB]

Hi @SaiNaveen1,

If you are using a Managed Kubernetes Cluster on DSS, you will choose from one of the cloud platforms to deploy your cluster to (EKS, AKS, GKE). By default, Dataiku API services have their exposition mode set to Load Balancer. For example, if you are running an EKS cluster, this exposition method uses an AWS load balancer, which is automatically deployed for you (no further configuration is necessary). Note, there is a one to one relationship of Load Balancer to API deployed.

For further information, please see the following documentation: https://doc.dataiku.com/dss/latest/apinode/operations/ha-deployment.html#load-balancer

Thanks,

Jordan

[igour]

what about if we wanted to use the Arbitrary Yaml, we wanted to put all the service, gateway and virtualservice into single Yaml option given in the DSS, how it will work. 

Thank you in advance.

[Turribeach]

Please start a new thread. This thread is from 2023 and has been marked as solved already.