Problem sensors kubernetes

api
(David) #1

Hi!

I have a problem with kubernetes pack.

When I try the sensors, they don’t work automatically, however, when I use them manually they respond correctly.

Here’s an example of the log when this happens:

    2019-04-29 13:00:18,410 139880156627184 WARNING sensor_wrapper [-] Sensor "watchBatchV1JobListForAllNamespaces" run method raised an exception: [('system library', 'fopen', 'Permission denied'), ('BIO routines', 'file_ctrl', 'system lib'), ('SSL routines', 'SSL_CTX_use_PrivateKey_file', 'system lib')].
Traceback (most recent call last):
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2reactor/container/sensor_wrapper.py", line 250, in run
    self._sensor_instance.run()
  File "/opt/stackstorm/packs/kubernetes/sensors/sensor_base.py", line 89, in run
    certfile=self.config['client_cert_path'])
  File "/opt/stackstorm/virtualenvs/kubernetes/lib/python2.7/site-packages/backports/ssl/core.py", line 689, in wrap_socket
    ctx.load_cert_chain(certfile, keyfile)
  File "/opt/stackstorm/virtualenvs/kubernetes/lib/python2.7/site-packages/backports/ssl/core.py", line 662, in load_cert_chain
    self._ctx.use_privatekey_file(keyfile or certfile)
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/OpenSSL/SSL.py", line 995, in use_privatekey_file
    self._raise_passphrase_exception()
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/OpenSSL/SSL.py", line 972, in _raise_passphrase_exception
    _raise_current_error()
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/OpenSSL/_util.py", line 54, in exception_from_error_queue
    raise exception_type(errors)
Error: [('system library', 'fopen', 'Permission denied'), ('BIO routines', 'file_ctrl', 'system lib'), ('SSL routines', 'SSL_CTX_use_PrivateKey_file', 'system lib')]
2019-04-29 13:00:18,412 139880156627184 ERROR (unknown file) [-] Traceback (most recent call last):

2019-04-29 13:00:18,412 139880156627184 ERROR (unknown file) [-]   File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2reactor/container/sensor_wrapper.py", line 382, in <module>

2019-04-29 13:00:18,412 139880156627184 ERROR (unknown file) [-]
2019-04-29 13:00:18,412 139880156627184 ERROR (unknown file) [-] obj.run()

2019-04-29 13:00:18,412 139880156627184 ERROR (unknown file) [-]   File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2reactor/container/sensor_wrapper.py", line 256, in run

2019-04-29 13:00:18,413 139880156627184 ERROR (unknown file) [-]
2019-04-29 13:00:18,413 139880156627184 ERROR (unknown file) [-] raise Exception(msg)

2019-04-29 13:00:18,413 139880156627184 ERROR (unknown file) [-] Exception
2019-04-29 13:00:18,413 139880156627184 ERROR (unknown file) [-] :
(Warren) #2

Hello. I’m curious if you have checked whether the file specified by client_cert_path has the correct permissions? The above trace seems to indicate a permission denied error. I suspect when you run manually the permissions are okay. Let us know whether this helps!

(Lindsay Hill) #3

Sensors run as the st2 user by default. But if you’re running it manually, you might be running it as root.

Like Warren says, probably something related to permissions on that client_cert_path file. Should be easy enough to change

Stackstorm does not start (mongod ends conection)
(David) #4

I LOVE YOU GUYS!!!

That was the problem, the certificates did not have the appropriate user permissions (root in this case).

Thank you.

(Tomaz Muraus) #5

On a related note - we should probably also update the sensor to catch such potential issues and throw a more user-friendly error message :slight_smile:

1 Like