Hi there!
To reduce project dependency on 3rd party paid services the StackStorm TSC has decided to move the Q/A from this forum to Github Discussions. This will make user experience better integrated with the native Github flow, as well as the questions closer to the community where they can provide answers.
Use 🔗 Github Discussions to ask your questions.
Hi,
I have installed a python3 package through the requirement.txt file.
It seems to be well installed and I checked:
activating the virtualenv then typing ‘python3’ then ’ import package’ works fine.
If I do the same import into an st2 action, the import fails with error.
I can post the errors but they may be too specific to the package.
I am wondering if they are other checks to do to debug this import error.
There were a couple of Python3-related fixes in 3.0.1 - e.g. Fix PYTHONPATH handling for Python runner actions using --python3 flag by Kami · Pull Request #4666 · StackStorm/st2 · GitHub
Which version are you using?
I can post the errors but they may be too specific to the package.
Probably worth posting the errors, because I think some of the package-specific things do matter.
I am now running : st2 3.0.1, on Python 2.7.12
The package in question is scikit-learn
the pack it is part of is created with the --python3 option
the error message is :
{
"stdout": "",
"result": "None",
"stderr": "Traceback (most recent call last):\n File \"/opt/stackstorm/st2/local/lib/python2.7/site-packages/python_runner/python_action_wrapper.py\", line 238, in _get_action_instance\n actions_cls = action_loader.register_plugin(Action, self._file_path)\n File \"/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/util/loader.py\", line 168, in register_plugin\n module = imp.load_source(module_name, plugin_abs_file_path)\n File \"/usr/lib/python3.5/imp.py\", line 172, in load_source\n module = _load(spec)\n File \"<frozen importlib._bootstrap>\", line 693, in _load\n File \"<frozen importlib._bootstrap>\", line 673, in _load_unlocked\n File \"<frozen importlib._bootstrap_external>\", line 665, in exec_module\n File \"<frozen importlib._bootstrap>\", line 222, in _call_with_frames_removed\n File \"/opt/stackstorm/packs/modal/actions/test.py\", line 4, in <module>\n from sklearn.gaussian_process import GaussianProcessRegressor\n File \"/opt/stackstorm/virtualenvs/modal/lib/python3.5/site-packages/sklearn/gaussian_process/__init__.py\", line 13, in <module>\n from .gpr import GaussianProcessRegressor\n File \"/opt/stackstorm/virtualenvs/modal/lib/python3.5/site-packages/sklearn/gaussian_process/gpr.py\", line 16, in <module>\n from .kernels import RBF, ConstantKernel as C\n File \"/opt/stackstorm/virtualenvs/modal/lib/python3.5/site-packages/sklearn/gaussian_process/kernels.py\", line 31, in <module>\n from ..metrics.pairwise import pairwise_kernels\n File \"/opt/stackstorm/virtualenvs/modal/lib/python3.5/site-packages/sklearn/metrics/__init__.py\", line 7, in <module>\n from .ranking import auc\n File \"/opt/stackstorm/virtualenvs/modal/lib/python3.5/site-packages/sklearn/metrics/ranking.py\", line 35, in <module>\n from ..preprocessing import label_binarize\n File \"/opt/stackstorm/virtualenvs/modal/lib/python3.5/site-packages/sklearn/preprocessing/__init__.py\", line 6, in <module>\n from ._function_transformer import FunctionTransformer\n File \"/opt/stackstorm/virtualenvs/modal/lib/python3.5/site-packages/sklearn/preprocessing/_function_transformer.py\", line 5, in <module>\n from ..utils.testing import assert_allclose_dense_sparse\n File \"/opt/stackstorm/virtualenvs/modal/lib/python3.5/site-packages/sklearn/utils/testing.py\", line 61, in <module>\n from nose.tools import raises as _nose_raises\n File \"/opt/stackstorm/st2/lib/python2.7/site-packages/nose/__init__.py\", line 1, in <module>\n from nose.core import collector, main, run, run_exit, runmodule\n File \"/opt/stackstorm/st2/lib/python2.7/site-packages/nose/core.py\", line 153\n print \"%s version %s\" % (os.path.basename(sys.argv[0]), __version__)\n ^\nSyntaxError: invalid syntax\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/opt/stackstorm/st2/local/lib/python2.7/site-packages/python_runner/python_action_wrapper.py\", line 334, in <module>\n obj.run()\n File \"/opt/stackstorm/st2/local/lib/python2.7/site-packages/python_runner/python_action_wrapper.py\", line 192, in run\n action = self._get_action_instance()\n File \"/opt/stackstorm/st2/local/lib/python2.7/site-packages/python_runner/python_action_wrapper.py\", line 245, in _get_action_instance\n raise exc_cls(msg)\nSyntaxError: Failed to load action class from file \"/opt/stackstorm/packs/modal/actions/test.py\" (action file most likely doesn't exist or contains invalid syntax): invalid syntax (core.py, line 153)\n\nTraceback (most recent call last):\n File \"/opt/stackstorm/st2/local/lib/python2.7/site-packages/python_runner/python_action_wrapper.py\", line 238, in _get_action_instance\n actions_cls = action_loader.register_plugin(Action, self._file_path)\n File \"/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/util/loader.py\", line 168, in register_plugin\n module = imp.load_source(module_name, plugin_abs_file_path)\n File \"/usr/lib/python3.5/imp.py\", line 172, in load_source\n module = _load(spec)\n File \"<frozen importlib._bootstrap>\", line 693, in _load\n File \"<frozen importlib._bootstrap>\", line 673, in _load_unlocked\n File \"<frozen importlib._bootstrap_external>\", line 665, in exec_module\n File \"<frozen importlib._bootstrap>\", line 222, in _call_with_frames_removed\n File \"/opt/stackstorm/packs/modal/actions/test.py\", line 4, in <module>\n from sklearn.gaussian_process import GaussianProcessRegressor\n File \"/opt/stackstorm/virtualenvs/modal/lib/python3.5/site-packages/sklearn/gaussian_process/__init__.py\", line 13, in <module>\n from .gpr import GaussianProcessRegressor\n File \"/opt/stackstorm/virtualenvs/modal/lib/python3.5/site-packages/sklearn/gaussian_process/gpr.py\", line 16, in <module>\n from .kernels import RBF, ConstantKernel as C\n File \"/opt/stackstorm/virtualenvs/modal/lib/python3.5/site-packages/sklearn/gaussian_process/kernels.py\", line 31, in <module>\n from ..metrics.pairwise import pairwise_kernels\n File \"/opt/stackstorm/virtualenvs/modal/lib/python3.5/site-packages/sklearn/metrics/__init__.py\", line 7, in <module>\n from .ranking import auc\n File \"/opt/stackstorm/virtualenvs/modal/lib/python3.5/site-packages/sklearn/metrics/ranking.py\", line 35, in <module>\n from ..preprocessing import label_binarize\n File \"/opt/stackstorm/virtualenvs/modal/lib/python3.5/site-packages/sklearn/preprocessing/__init__.py\", line 6, in <module>\n from ._function_transformer import FunctionTransformer\n File \"/opt/stackstorm/virtualenvs/modal/lib/python3.5/site-packages/sklearn/preprocessing/_function_transformer.py\", line 5, in <module>\n from ..utils.testing import assert_allclose_dense_sparse\n File \"/opt/stackstorm/virtualenvs/modal/lib/python3.5/site-packages/sklearn/utils/testing.py\", line 61, in <module>\n from nose.tools import raises as _nose_raises\n File \"/opt/stackstorm/st2/lib/python2.7/site-packages/nose/__init__.py\", line 1, in <module>\n from nose.core import collector, main, run, run_exit, runmodule\n File \"/opt/stackstorm/st2/lib/python2.7/site-packages/nose/core.py\", line 153\n print \"%s version %s\" % (os.path.basename(sys.argv[0]), __version__)\n ^\nSyntaxError: invalid syntax\n\n",
"exit_code": 1
}
the end of the error suggest that a python2 library is used insted of its python3 counterpart.
From the virtaulenv itself all works well:
(modal) root@ubuntu:/opt/stackstorm/virtualenvs/modal# python3
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sklearn
Looks like you’re using Python 3.5? I think our testing was with Python 3.6. Can you upgrade?
same problem with python 3.6
File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/python_runner/python_action_wrapper.py", line 238, in _get_action_instance
actions_cls = action_loader.register_plugin(Action, self._file_path)
File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/util/loader.py", line 168, in register_plugin
module = imp.load_source(module_name, plugin_abs_file_path)
File "/usr/lib/python3.6/imp.py", line 172, in load_source
module = _load(spec)
File "<frozen importlib._bootstrap>", line 684, in _load
File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/opt/stackstorm/packs/modal2/actions/test.py", line 4, in <module>
from sklearn.gaussian_process import GaussianProcessRegressor
File "/opt/stackstorm/virtualenvs/modal2/lib/python3.6/site-packages/sklearn/gaussian_process/__init__.py", line 13, in <module>
from .gpr import GaussianProcessRegressor
File "/opt/stackstorm/virtualenvs/modal2/lib/python3.6/site-packages/sklearn/gaussian_process/gpr.py", line 16, in <module>
from .kernels import RBF, ConstantKernel as C
File "/opt/stackstorm/virtualenvs/modal2/lib/python3.6/site-packages/sklearn/gaussian_process/kernels.py", line 31, in <module>
from ..metrics.pairwise import pairwise_kernels
File "/opt/stackstorm/virtualenvs/modal2/lib/python3.6/site-packages/sklearn/metrics/__init__.py", line 7, in <module>
from .ranking import auc
File "/opt/stackstorm/virtualenvs/modal2/lib/python3.6/site-packages/sklearn/metrics/ranking.py", line 35, in <module>
from ..preprocessing import label_binarize
File "/opt/stackstorm/virtualenvs/modal2/lib/python3.6/site-packages/sklearn/preprocessing/__init__.py", line 6, in <module>
from ._function_transformer import FunctionTransformer
File "/opt/stackstorm/virtualenvs/modal2/lib/python3.6/site-packages/sklearn/preprocessing/_function_transformer.py", line 5, in <module>
from ..utils.testing import assert_allclose_dense_sparse
File "/opt/stackstorm/virtualenvs/modal2/lib/python3.6/site-packages/sklearn/utils/testing.py", line 61, in <module>
from nose.tools import raises as _nose_raises
File "/opt/stackstorm/st2/lib/python2.7/site-packages/nose/__init__.py", line 1, in <module>
from nose.core import collector, main, run, run_exit, runmodule
File "/opt/stackstorm/st2/lib/python2.7/site-packages/nose/core.py", line 153
print "%s version %s" % (os.path.basename(sys.argv[0]), __version__)
Seems that nose is being pulled from 2.7. Could be some additional PYTHONPATH change needed.
It’s not your setup, it is a code issue. I think this needs an issue logged at github.com/StackStorm/st2/issues
Issue #4710 filled to track the issue.
Thanks! Hopefully @kami will see what’s wrong and fix it quickly (or tell us we need to switch to Bionic with all Python3)
I tried Ubuntu Bionoc --unstable --stagging
st2 3.1dev (7079635), on Python 3.6.8
and here the problem is solved.
Makes sense, since everything there is running on Python3, there’s no Python2 packages to pollute things.
Everything (?) will be better when we can run Py3, and only Py3 everywhere