0 votes
Hi,

I'm unable to get started with establishing connectivity between DSS and Hiveserver2.

HDFS integration works, and I have added the jars required for Hive client in a folder owned by dss user, and have added that folder path in Classpath

Specifying auth=maprsasl;saslQop=auth-conf in extra URL but the connection isn't working somehow. Tried to go through backend.log and here are the details:

 

[2019/02/12-00:05:53.706] [qtp400064818-452] [ERROR] [dku.dispatcher]  - Caught ERROR
org.springframework.web.util.NestedServletException: Handler processing failed; nested exception is java.lang.AssertionError
        at org.springframework.web.servlet.DispatcherServlet.triggerAfterCompletionWithError(DispatcherServlet.java:1275)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:951)
        at com.dataiku.dip.server.controllers.DKUDispatcherServlet.doDispatch(DKUDispatcherServlet.java:50)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:867)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:951)
        at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:842)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:827)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651)
        at com.dataiku.dip.shaker.server.ResourceFilter.doFilter(ResourceFilter.java:33)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1631)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:549)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:568)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1111)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:478)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1045)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
        at org.eclipse.jetty.server.Server.handle(Server.java:462)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:232)
        at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:534)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.AssertionError
 at com.dataiku.dip.hive.MetastoreInspectionService.newInspector(MetastoreInspectionService.java:82)
        at com.dataiku.dip.server.controllers.ConnectionsController.listHiveNames(ConnectionsController.java:117)
        at com.dataiku.dip.server.controllers.ConnectionsController$$FastClassBySpringCGLIB$$149f32e4.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:701)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:80)
        at com.dataiku.dip.server.controllers.AuditedCallAspect.doCall(AuditedCallAspect.java:40)
        at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:80)
        at com.dataiku.dip.server.controllers.CallTracingAspect.doCall(CallTracingAspect.java:78)
        at sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:633)
        at com.dataiku.dip.server.controllers.ConnectionsController$$EnhancerBySpringCGLIB$$b2dfab73.listHiveNames(<generated>)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:176)
        at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:440)
        at com.dataiku.dip.server.MissingParameterExceptionHandler.invokeHandlerMethod(MissingParameterExceptionHandler.java:18)
        at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:428)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:933)
        ... 28 more

 

Can you please help me understand the issue?
by

1 Answer

+1 vote
Hi,

It looks like DSS couldn't automatically detect Hive, and hasn't enabled Hive. Try editing bin/env-hadoop.sh and setting DKU_HIVE_ENABLED=true

You'll need to restart afterwards
by
Thanks, this worked. You are correct, DSS didn't enable Hive at first.
After enabling, DSS couldn't find Hive jdbc drivers-  the main issue was that the Hive client wasn't installed on the system. Installed Hive client on the system and re-ran hadoop integration and it worked.
1,156 questions
1,191 answers
1,343 comments
11,503 users

©Dataiku 2012-2018 - Privacy Policy