St2chatops with Mattermost not replying

How about Mattermost HTTPS/TLS config?

MATTERMOST_WSS_PORT
MATTERMOST_HTTP_PORT
MATTERMOST_TLS_VERIFY
MATTERMOST_USE_TLS

Did you change that to force HTTP as well?

Yes, otherwise the bot wouldn鈥檛 be able to see my commands and execute them.

export MATTERMOST_WSS_PORT=80
export MATTERMOST_HTTP_PORT=80
export MATTERMOST_TLS_VERIFY=false
export MATTERMOST_USE_TLS=false

Just for the record, what are current st2/st2chatops package versions you鈥檙e running? Latest?

I鈥檓 out of ideas for now.
We鈥檒l likely need to try to reproduce it on latest st2 scripted install with default configs and Mattermost server.

If nothing works, might be indeed bug or incompatibility issues with recent Mattermost server which is probably different comparing to what we鈥檝e tested before.

How would I be able to check which version of the packages I am running?

Considering that even RocketChat has the same symptoms, shouldn鈥檛 the problem be with my installation of st2 rather than compatibility with Mattermost?

I have tried to curl http://127.0.0.1:9102/stream?st2-api-key=MY-API-KEY and the stream seems to work correctly with a lot of events showing up when I run a command from Mattermost. So I do not know why GET requests do not show up in nginx logs for the stream from the bot.

Google should help how to find package versions on CentOS7.

Alright, here is an example from my VM:

[vagrant@centos7 ~]$ yum list | grep @StackStorm
st2.x86_64                              3.1.0-1                        @StackStorm_stable
st2chatops.x86_64                       3.1.0-2                        @StackStorm_stable
st2mistral.x86_64                       3.1.0-1                        @StackStorm_stable
st2web.x86_64                           3.1.0-1                        @StackStorm_stable

Sorry, for some reason I thought there would be a st2ctl command to show the version.

yum list | grep @StackStorm does not work because I installed the packages locally due to the network not having access to the internet.

yum list | grep st2 says:
st2.x86_64 is 3.0.1-4
st2chatops.x86_64 is 3.0.1-1
st2mistral.x86_64 is 3.0.1-1
st2web.x86_64 is 3.0.1-1

Thanks, the versions look pretty much almost latest, at least no diff for Mattermost in that st2/st2chatops version range.

Tonight, I鈥檒l re-install again from home and try to use st2chatops with Slack instead to try to pinpoint my problem. I鈥檒l let you know how it went tomorrow.

1 Like

Good morning,

First off, I am sorry for the extremely late reply. This week has been very hectic for me.

Moving on, yesterday evening I installed StackStorm and Mattermost at home (which means that I have an Internet connection on my network).

Here鈥檚 the setup:

  • A CentOS 7 Minimal host for StackStorm. Stackstorm was installed using the one-line installation that is found in the Quick Installation section of the install guide. I only changed the adapter setting for the bot in st2chatops.env for Slack.
  • Created a Slack server online and created a bot account for it.

Result: Worked like a charm. No problems at all. The bot responds as expected.

Decided to install Mattermost to test with it.
Setup:

  • A CentOS 7 Minimal host for Mattermost. Followed the installation guide for RHEL 7 on the Mattermost website. Very barebones, HTTP only. No NGINX in front. Tried Mattermost 5.4.0, 5.9.0 and 5.12.3 (latest).

Result: All 3 versions had the same results that I鈥檝e had seen the beginning. Connects successfully, sees my commands and runs it on StackStorm. Receives the response from StackStorm but does nothing with it. If the logs are a DEBUG level, you can see a Chatops message received containing the response object. It has the message field that is correct for the comand that is ran. Mattermost 5.12.X implemented bot accounts to Mattermost (a bit like Slack), so I tried setting it up using that integration. Still same result.

At this point, I am unsure what the problem is. It seems to point to a Hubot or a Mattermost adapter problem.

Have you successfully tried to use Mattermost with st2chatops recently? I would love to see if you get the same results that I do. Obviously, I assume that you are more than likely very busy, and I don鈥檛 want to take a lot of your time. However, whenever you have a little of time, I would extremely appreciate it if you could try the install yourself.

Thank you very much for the help so far.

Good that you double-checked with Slack vs Mattermost :+1: This definitely would help in future during the debugging/root-cause-analysis.

This sounds like a bug.
Can you please report it under the Issues 路 StackStorm/hubot-stackstorm 路 GitHub repo ?


A bit of context here after looking at history: Mattermost support was initially contributed by one of our users and got merged into st2chatops around ~2 years ago:

For st2chatops we consider Slack as 100% supported and have end2end tests for it, while other adapters community-contributed, community-managed and not tested on a repeatable basis. Most probably Mattermost worked before, but stopped at some point due to Mattermost server upgrades/breaking changes which we didn鈥檛 follow.

@ChickenNotHD I have reproduced this issue and I am working on a fix. I might be able to issue a fix for a 3.1.x release, but this will definitely make its way into ST2 v3.2.

I will also get you a workaround until then.

@blag could you please post the workaround?

@lenba I will post the workaround once I have completed it. :slight_smile:

Fixed here:

The two fix commits:


Unfortunately, I have done a lot of refactoring of the chat provider adapters, so those commit diffs won鈥檛 apply cleanly to your existing installation.

I think the easiest way to publish a workaround would be to publish a new bugfix version of hubot-stackstorm for you to manually npm install on Monday.

The refactorings+fix will be in the next minor release of hubot-stackstorm (0.10.0), which I will also publish on Monday, once those three PRs are all reviewed, rebased, and merged in.

The workaround PR is here:

Once that is reviewed and merged I鈥檒l publish that branch to NPM as version 0.9.7. You should then be able to install the fixed version manually:

cd /opt/stackstorm/chatops
npm install hubot-stackstorm@0.9.7

I have not tested the manual workaround yet, but I will once I publish it to NPM.

Thank you very much for the help @blag
I will tell you if this fixes my issue once v0.9.7 is released.

I just released version 0.10.0 of hubot-stackstorm to NPM. Please try installing that version instead:

cd /opt/stackstorm/chatops
npm install hubot-stackstorm@0.10.0

That contains the fix and a large amount of refactoring.

If you do not want to install that version, an alternative is to manually apply these two commit diffs:


2 Likes

@blag I鈥檝e tested it and it works. Thank you for all your hard work!

1 Like

Sorry for the late reply.

Thank you very much for your help. Hubot now responds properly to my commands.

@lenba and @ChickenNotHD: Thanks for checking my work, glad it works for you now!