Stackstorm does not start (mongod ends conection)

Hello,

I am a new StackStorm user. I have tried different OS (Ubuntu, CentOS…) and most of the time when we restart the machine, Stackstorm does not start. When I check the logs ( /var/log/mongodb/mongod.log) I see that it tries to open and close mutiple sessions, while CPU is near 100%. Sometime this problem does not happen and everything is fine, but when it does there is no way to fix it.

I’m using StackStorm 2.7.2, on Python 2.7

Thanks.

Hello. As a new StackStorm user, is there any reason you’ve chosen to use an older version of StackStorm? Does this same behavior occur with st2 2.10.x or 3.0.x? I’m curious what you are trying to do with stackstorm?

Hello!

I’ve updated to version 3.0 but the same thing keeps happening.

I have installed stackstorm many times and the same thing always happens. Sometimes it connects well and others are impossible.

I have installed stackstorm following the official guide.

LOG:

2019-05-03T20:19:47.425+0200 I -        [conn1792] end connection 127.0.0.1:53940 (65 connections now open)

2019-05-03T20:19:47.425+0200 I - [conn1793] end connection 127.0.0.1:53942 (65 connections now open)
2019-05-03T20:19:47.554+0200 I - [conn1786] end connection 127.0.0.1:53928 (63 connections now open)
2019-05-03T20:19:47.554+0200 I - [conn1787] end connection 127.0.0.1:53930 (63 connections now open)
2019-05-03T20:19:47.606+0200 I - [conn1783] end connection 127.0.0.1:53922 (61 connections now open)
2019-05-03T20:19:47.687+0200 I - [conn1794] end connection 127.0.0.1:53944 (60 connections now open)
2019-05-03T20:19:47.687+0200 I - [conn1795] end connection 127.0.0.1:53946 (60 connections now open)

Thanks!

I’m curious whether any st2 processes are not running? You can use st2ctl status. Do the st2 logs indicate any errors?

Hello!

This the exit of the command:

   [root@localhost ~]# st2ctl status
    ##### st2 components status #####
    st2actionrunner PID: 3687
    st2actionrunner PID: 3702
    st2actionrunner PID: 3709
    st2actionrunner PID: 3717
    st2actionrunner PID: 3727
    st2actionrunner PID: 3732
    st2actionrunner PID: 3737
    st2actionrunner PID: 3745
    st2actionrunner PID: 3753
    st2actionrunner PID: 3761
    st2api PID: 3608
    st2api PID: 10806
    st2stream PID: 3580
    st2stream PID: 10834
    st2auth PID: 3579
    st2auth PID: 10869
    st2garbagecollector PID: 3614
    st2notifier PID: 3575
    st2resultstracker PID: 3596
    st2rulesengine PID: 3583
    st2sensorcontainer PID: 3609
    st2chatops is not running.
    st2timersengine PID: 3601
    st2workflowengine PID: 3578
    st2scheduler PID: 3573
    mistral-server PID: 3584
    mistral.api PID: 3615
    mistral.api PID: 10803
    mistral.api PID: 10805

Only the Chatops not running.

The logs show no errors, except the log of mongod (/var/log/mongo/mongodb.conf).
Another example of the mongo log:

2019-05-04T02:21:56.524+0200 I NETWORK  [conn50] received client metadata from 127.0.0.1:45340 conn50: { driver: { name: "PyMongo", version: "3.7.2" }, os: { type: "Linux", name: "CentOS Linux 7.6.1810 Core", architecture: "x86_64", version: "3.10.0-957.12.1.el7.x86_64" }, platform: "CPython 2.7.5.final.0" }
2019-05-04T02:21:56.544+0200 I NETWORK  [thread1] connection accepted from 127.0.0.1:45342 #51 (44 connections now open)
2019-05-04T02:21:56.875+0200 I NETWORK  [conn51] received client metadata from 127.0.0.1:45342 conn51: { driver: { name: "PyMongo", version: "3.7.2" }, os: { type: "Linux", name: "CentOS Linux 7.6.1810 Core", architecture: "x86_64", version: "3.10.0-957.12.1.el7.x86_64" }, platform: "CPython 2.7.5.final.0" }
2019-05-04T02:21:56.875+0200 I -        [conn51] end connection 127.0.0.1:45342 (44 connections now open)
2019-05-04T02:21:56.966+0200 I -        [conn47] end connection 127.0.0.1:45334 (43 connections now open)
2019-05-04T02:21:57.746+0200 I -        [conn50] end connection 127.0.0.1:45340 (42 connections now open)[quote="warrenvw, post:4, topic:627, full:true"]

Thanks!

I would look at the st2 logs rather than the Mongo logs. Those Mongo logs just show connections.

My guess is you probably have RabbitMQ issues, possibly related to host name resolution

The problems I had with the sensors were (Problem sensors kubernetes) also causing the problem with mongod. At the moment it seems that everything works correctly.

Thank you very much for the help!.