Hello All,
I’m attempting to setup GitHub - StackStorm-Exchange/stackstorm-icinga2: Icinga2 Integration pack and after setting up a rule in the Stackstorm UI, I noticed it would never get enforced.
Running the sensor with “/opt/stackstorm/st2/bin/st2sensorcontainer --config-file=/etc/st2/st2.conf --sensor-ref=icinga2.Icinga2StateChangeSensor” as root reveals that many of the detected state changes are causing exceptions in the sensor and thus are not dispatching the trigger. Example state changes causing exceptions below:
Exception Example 1:
2020-09-17 10:20:52,526 INFO [-] Connecting to event stream API.
2020-09-17 10:20:52,718 INFO [-] Processing event: {"check_result":{"active":true,"check_source":"###HOSTNAME###","command":["C:\Program Files\ICINGA2\/sbin/check_load.exe","-c","95","-w","90"],"execution_end":1600363364.508,"execution_start":1600363362.94,"exit_status":2.0,"output":"LOAD CRITICAL 97.28% ","performance_data":["load=97.28%;90;95;0;100"],"schedule_end":1600363364.512,"schedule_start":1600363364.512,"state":2.0,"ttl":0.0,"type":"CheckResult","vars_after":{"attempt":1.0,"reachable":true,"state":2.0,"state_type":0.0},"vars_before":{"attempt":1.0,"reachable":true,"state":0.0,"state_type":1.0}},"host":"###HOSTNAME###","service":"Windows Load","state":2.0,"state_type":0.0,"timestamp":1600363363.52031,"type":"StateChange"}
2020-09-17 10:20:52,719 ERROR [-] Icinga2StateChangeSensor Exception Invalid \escape: line 1 column 92 (char 91)
Traceback (most recent call last):
File "/opt/stackstorm/packs/icinga2/sensors/icinga2_state_change_events.py", line 66, in run
self.process_event(event)
File "/opt/stackstorm/packs/icinga2/sensors/icinga2_state_change_events.py", line 76, in process_event
event = json.loads(event)
File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python2.7/json/decoder.py", line 382, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Invalid \escape: line 1 column 92 (char 91)
2020-09-17 10:20:57,721 INFO [-] Connecting to event stream API.
Exception Example 2:
2020-09-17 10:29:03,692 INFO [-] Connecting to event stream API.
2020-09-17 10:29:29,318 INFO [-] Processing event: {"check_result":{"active":true,"check_source":"###HOSTNAME###","command":["C:\Windows\system32\WindowsPowerShell\v1.0\powershell.exe","-command","C:\Program` Files\ICINGA2\sbin\JobStatus.ps1","-Username","###USERNAME###","-Password","###PASSWORD###","; exit","$LASTEXITCODE"],"execution_end":1600363880.88,"execution_start":1600363878.467,"exit_status":0.0,"output":"OK: All Jobs have an EndTime Entry. ","performance_data":["EndTimeNullCount=0"],"schedule_end":1600363880.88,"schedule_start":1600363880.88,"state":0.0,"ttl":0.0,"type":"CheckResult","vars_after":{"attempt":1.0,"reachable":true,"state":0.0,"state_type":1.0},"vars_before":{"attempt":2.0,"reachable":true,"state":2.0,"state_type":0.0}},"host":"###HOSTNAME###","service":"Job Monitor","state":0.0,"state_type":1.0,"timestamp":1600363880.058318,"type":"StateChange"}
2020-09-17 10:29:29,320 ERROR [-] Icinga2StateChangeSensor Exception Invalid \escape: line 1 column 89 (char 88)
Traceback (most recent call last):
File "/opt/stackstorm/packs/icinga2/sensors/icinga2_state_change_events.py", line 66, in run
self.process_event(event)
File "/opt/stackstorm/packs/icinga2/sensors/icinga2_state_change_events.py", line 76, in process_event
event = json.loads(event)
File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python2.7/json/decoder.py", line 382, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Invalid \escape: line 1 column 89 (char 88)
2020-09-17 10:29:34,322 INFO [-] Connecting to event stream API.
Has anyone encountered this before and been able to resolve/work around this?