Dataiku Spark Remote Connection

[sangkim]

Hello, I am using Dataiku 12.5.2 and currently running Spark 2.4. Dataiku is installed on a server named A, while Spark is installed on a server named B, configured as a standalone Spark installation without Hadoop. Both server A and server B are capable of TCP communication and allow SSH access. How can I use the Spark on server B from Dataiku on server A?

※ I have confirmed that the Spark installed on server A, where Dataiku is also installed, is well integrated.
@spark

---

Operating system used: linux

[Turribeach]

Please review the different Spark integration options in the documentation:

https://doc.dataiku.com/dss/latest/spark/installation.html#setting-up-spark-integration

[sangkim]

I am using Dataiku and Spark in an on-premise environment. Spark is configured as a standalone installation without Hadoop. Due to the current project configuration, I cannot use Kubernetes (k8s) or Docker. I would like to integrate Dataiku on server A with Spark on server B.