Since v3.1, we introduce a garbage collection routine for cleaning up orquesta workflow executions. In cases where there is unexpected failure in st2 services, such as prolong connection errors to rabbitMQ and etc., the workflow execution could be left in a running state. The purpose of this garbage collection routine is to cleanup and cancel these workflow executions.
Unfortunately, there is a bug in the clean up routine that misidentified and cancels long running workflow executions. We are diligently working on a bug fix and apologizes for any inconvenience.
In the meantime, if any user encountered this problem, the garbage collection can be disabled by setting the
gc_max_idle_sec option to a value of zero in the
workflow_engine of the
/etc/st2/st2.conf file and restarting st2.
[workflow_engine] gc_max_idle_sec = 0