+1 vote

I'm using the dataiku VM and have installed Keras and Tensorflow in the dataiku python environment.

I can create a code notebook in the Lab and use the code and data from the following guide and it works http://machinelearningmastery.com/multi-class-classification-tutorial-keras-deep-learning-library/

However if I try to use the exact same code within a "Custom Python Model" under Analysis, I always get the error 

Training Failed… Sorry
global name 'Sequential' is not defined

 

The stacktrace in run/backend.log is:

[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  - 2017-07-17 20:07:19,939 INFO Fitting model:
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  - 2017-07-17 20:07:19,939 INFO   Model is: <keras.wrappers.scikit_learn.KerasClassifier object at 0x7f4c4cca53d0>
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  - 2017-07-17 20:07:19,939 INFO   train_X class: <type 'numpy.ndarray'>
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  - 2017-07-17 20:07:19,939 INFO   train_X shape: (123, 4)
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  - 2017-07-17 20:07:19,939 INFO   train_y shape: (123,)
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  - 2017-07-17 20:07:19,939 INFO END -  Fitting model
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  - 2017-07-17 20:07:19,939 ERROR API call failed : None
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  - Traceback (most recent call last):
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -   File "/home/dataiku/dataiku-dss-4.0.5/python/dataiku/doctor/server.py", line 87, in do_POST
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -     ret = api_command(arg)
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -   File "/home/dataiku/dataiku-dss-4.0.5/python/dataiku/doctor/dkuapi.py", line 45, in aux
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -     return api(**kwargs)
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -   File "/home/dataiku/dataiku-dss-4.0.5/python/dataiku/doctor/commands.py", line 253, in train_prediction_models_nosave
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -     pipeline)
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -   File "/home/dataiku/dataiku-dss-4.0.5/python/dataiku/doctor/prediction_entrypoints.py", line 43, in prediction_train_score_save
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -     (clf, out_params, prepared_X, iipd) = classification_fit(modeling_params, split_desc, transformed_train, prediction_type)
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -   File "/home/dataiku/dataiku-dss-4.0.5/python/dataiku/doctor/prediction/classification_fit.py", line 584, in classification_fit
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -     clf.fit(train_X, train_y, **kwargs)
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -   File "/home/dataiku/dss/condaenv/lib/python2.7/site-packages/keras/wrappers/scikit_learn.py", line 201, in fit
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -     return super(KerasClassifier, self).fit(x, y, **kwargs)
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -   File "/home/dataiku/dss/condaenv/lib/python2.7/site-packages/keras/wrappers/scikit_learn.py", line 138, in fit
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -     self.model = self.build_fn(**self.filter_sk_params(self.build_fn))
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -   File "<string>", line 16, in create_model
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  - NameError: global name 'Sequential' is not defined
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  - 2017-07-17 20:07:19,939 WARNING Handle failed
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  - 127.0.0.1 - - [17/Jul/2017 20:07:19] "POST /command HTTP/1.1" 500 -
[2017/07/17-20:07:19.940] [Exec-304] [INFO] [dku.utils]  - ('SCIKIT MODEL', <keras.wrappers.scikit_learn.KerasClassifier object at 0x7f4c4cca53d0>)
[2017/07/17-20:07:19.940] [MRT-339] [ERROR] [dku.analysis.prediction]  - Processing failed
com.dataiku.common.server.APIError$APIErrorException: global name 'Sequential' is not defined
        at com.dataiku.common.rpc.InternalAPIClient.handleStringResp(InternalAPIClient.java:629)
        at com.dataiku.common.rpc.InternalAPIClient.postFormToString(InternalAPIClient.java:435)
        at com.dataiku.dip.analysis.coreservices.AnalysisMLKernel.executeCommand(AnalysisMLKernel.java:103)
        at com.dataiku.dip.analysis.ml.prediction.PyRegularNoSavePredictionHandler$TrainAdditionalThread.process(PyRegularNoSavePredictionHandler.java:118)
        at com.dataiku.dip.analysis.ml.shared.PRNSTrainThread.run(PRNSTrainThread.java:66)

 

 

by

1 Answer

–1 vote
It looks like you're missing some imports - in this case Sequential
by
Unfortunately not. It has the exact same code between the notebook and the custom module. ie. they both include "from keras.models import Sequential"

It can run the code "model = Sequential()" directly in the custom python module  without issues,  so I think its something to do with the fact that the baseline_module() method which calls Sequential() is passed as a parameter in the line "estimator = KerasClassifier(build_fn=baseline_model, ..." and then attempted to be executed in a different space...
1,222 questions
1,250 answers
1,417 comments
11,783 users

©Dataiku 2012-2018 - Privacy Policy