Long running workflows stop without any errors

We have workflows that can run for 12-24 hours or more, depending on the data. This is one a k8s HA cluster with 3.1.0. It appears after 5-6 hours or so, the workflow just stops without any indication of why? Is there some sort of timeout I need to configure to prevent this?

Additional notes the execution still reads running, st2 execution list show running for 326784 seconds and climbing. When i do st2 execution get 5d4deccedf212700f841dfdf i see all actions completed?

Are you using the HA cluster documented at StackStorm HA Cluster in Kubernetes - BETA — StackStorm 3.1.0 documentation? The StackStorm HA cluster does not support mistral workflow but this issue is marked as mistral. Is it possible to provide a workflow definition? Also, does this happen all the time or occassionally?

Yes we are using the HA cluster in K8s as far as we can tell. Using helm charts. I can send what you need via email, that would be better for me. Ok so when you write the following:
The StackStorm HA cluster does not support mistral workflow but this issue is marked as mistral.
Did i miss that in the docs? Is it just not supported, or it actually will not work? Those are 2 different things…but i digress. I am open to moving to orquesta is this solves the issue. I have to setup another load test, i will do that tomorrow and let you know the results.

Mistral is going to be deprecated soon and it is not included in the Helm chart that we provided. Therefore we will not support it in k8s HA configuration. This is documented at st2workflowengine under the HA documentation. So if you are using k8s HA, please migrate your workflow to Orquesta.