Inconsistent Orquesta WF execution times for same workflow

I have a very simple workflow borrowed from examples in github as follows:
version: 1.0

description: A workflow demonstrating with items and concurrent processing.


  • members: [1,2,3,4,5,6,7,8,9,10]
  • concurrency: 5
  • trace_id

items: <% ctx(members) %>
concurrency: <% ctx(concurrency) %>
action: core.echo message="<% item() %>, resistance is futile!"


  • items: <% task(task1)$.result.stdout) %>

I am executing in docker on my mac. The purpose is to vary concurrency, number of items, action runners to understand how to appropriately size our k8s cluster (number of action runners, workflow engines, etc) and so forth. The issue is i am getting wildly varying times running the exact same wf above. Times that vary from 4 seconds to over 21 seconds. I can see (with the st2 tail execution) some actions just seem to stop and take 2-3 seconds, mind you same action in the loop above. I am using top and some other tools but i cannot determine why the WF just hangs periodically. This happens in production K8s cluster as well, so trying to determine how to debug this locally first. Any suggestions appreciated.