This site is in read only mode. Please continue to browse, but replying, likes, and other actions are disabled for now.

⚠️ We've moved!

Hi there!

To reduce project dependency on 3rd party paid services the StackStorm TSC has decided to move the Q/A from this forum to Github Discussions. This will make user experience better integrated with the native Github flow, as well as the questions closer to the community where they can provide answers.

Use 🔗 Github Discussions to ask your questions.

Executions stuck in canceling state

Workflows Mistral
#1

Hi,

I have an issue with executions stuck in ‘canceling’ state. I have a concurrency policy that allow 20 executions. So all my new execution of this action are tagged automatically ‘cancel’. I decided to restart stackstorm but nothing changed. Someone has an idea ?

Thank

#2

I traveled in db st2 and I found:

select name,state from workflow_executions_v2 where name=“toto”

I found 6 occurrences of my execution in running state since several days and it’s seems to be the same date of an issue (I had to shutdown stackstorm). I think stackstorm is in global bad state, I tried to cancel this 6 executions but they stay in ‘canceling’ state (with st2 execution list)… But now, I can run new toto executions.

The only thing that makes me uncomfortable is st2 was canceling my execution whereas the policy allows 20 executions (not 6)

#3

Is that an issue with Mistral or Orquesta workflows?

Just a reminder here that full Mistral deprecation will happen really soon in st2 v3.3: Roadmap — StackStorm 3.2dev documentation and we would strongly suggest to follow the Orchestra as a workflow engine which is actively developed and supported (Orquesta — StackStorm 3.1.0 documentation)

#4

It’s a mistral worflows.

Are you saying that we can not use mistral for next version of st2

i’m using st2 2.8.1 and for some reason I can’t upgrade this version as I want.

#5

We’ll fully remove Mistral really soon in favor of Orquesta workflow. It’s there to cut some of the Mistral weakest points, including performance. No worries, you could upgrade up to st2 3.3 in any way in future which will be latest Mistral supported version and migrate your workflows to Orquesta, if you want.

StackStorm version you’re using v2.8.1 was released 1 year ago and since that we’ve had 12 releases which fixed a lot of bugs.

There is a high chance that your issue was fixed during that time.
I would suggest to try following the changelog Releases · StackStorm/st2 · GitHub (st2/CHANGELOG.rst at master · StackStorm/st2 · GitHub) and if you could spot your issue fixed from that log, - upgrading st2 gradually (with Upgrade Notes — StackStorm 3.1.0 documentation).

#6

Thx,

So we have to “translate” all mistral workflows into orquestra :frowning:

#7

Hi User42,
This is because Mistral gets out of sync with ST2. At a past job we ran to this frequently in our development environment. In our case we would pause the workflow and they would pause in mistral, and stay in pausing in ST2.

I no longer have access to the script that I wrote for this, but this is what I remember off the top of my head.

You will need the mistral execution ID, which is different from the ST2 execution ID. Look in MongoDB at the action execution collection. You’ll need to find the document for the ST2 execution ID. I think the field is parent_process_id. In that document will be mistral execution ID.

Once you have that you can use the mistral command to resume or cancel the process. You can execute the command “mistral execution-update -S NEWSTATE MISTRALEXECUTIONID”. This will tell mistral to transistion the workflow. When mistral transitions the workflow, it will call home to ST2 and it will reflect the updated state.

Sorry I don’t have more detailed steps.