Define a workflow in Python

Hi guys,
I’m new to StackStorm and currently I’m exploring it’s abilities.

It seem very cool, but the Orquesta-based workflow it’s not so good for me.
In my use case I must manage very complex workflows, and I don’t want to go crazy by defining them with the YAML DSL provided by Orquesta.

At the same time, I would like to take advantage of the public packs available in the Exchange. So, essentially, I’d like to define my workflows using Python actions, invoking other actions (for instance, to query Active Directory, Oracle database, and so on) from the main one.

I have just seen the post Invoke an action from a Python action script, but the proposed solution requires a continuous polling of StackStorm, in order to check the completion of each action. So I think that this solution is very inefficient when trying to invoke many actions, and when you have many instances of your main action running.

Maybe, after two years, is there a better solution to do it?

Triggering one action from another and trying to workflow in python actions is an anti-pattern in the StackStorm world. These automations are best built and managed by re-usable blocks like Packs, Actions, Sensors and Rules separating the concerns.

If Orquesta looks too complex to you, try ActionChain — StackStorm 3.2.0 documentation which is linear and pretty straightforward. It also the fastest workflow engine because of its simplicity.

What is the example use case/logic you’d like to explore?

Let’s suppose, for demonstration purposes only, a workflow like to the following (in pseudocode):

WhenSomeEventOccours(params) {
  try {
    account = activeDirectoryLookupUser(params.uid)
    if account not found {
      publishEvent("RequirementNotSatisfied")
      abort()
    }
    assert(account is enabled)
    assert(activeDirectoryLookupGroup("root/{params.foo}/Groups/{params.bar}") is found)
    assert(httpGet("https://example.org/{params.zoo}") is 200)
    if params.alternativeUid {
      account = activeDirectoryLookupUser(params.alternativeUid)
      if account not found {
        account = activeDirectoryCreateAccount(params...)
        sendMail(account credentials)
      }
    }
    activeDirectoryAddUserToGroup(account, "OU/{params.foo}/Groups/{params.bar}")
    if (params.sync) {
      httpPost("https://example.org/{params.zoo}/member", account)
      rows = doSqlQuery("select * from ...")
      for each row in rows {
        httpPost("https://example.org/{params.zoo}/apply", row)
        httpPost("https://example.org/{params.zoo}/row/{row.id}/restrict", account)
      }
    }
    publishEvent("WorkflowCompleted", account, params)
  } catch (error) {
    publishEvent("WorkflowFailed", params, error)
    // then some event listener could send a notification to report the failure
  }
}

I think that every function such as activeDirectoryLookupUser, httpGet, doSqlQuery, etc. could come from an existent action pack, but the glue is not so trivial…

Nice!

Yes, exactly.
You want to split every logical step into a dedicated pack/action. And replace WhenSomeEventOccours with the StackStorm Rule (Rules — StackStorm 3.2.0 documentation).

What you have is pretty much a linear chain of actions. I think it’s OK to keep some of the custom logic in one place as you start. However as this code grows it becomes harder to manage, maintain, improve, re-use it.

In StackStorm world, Actions = Code and Workflows = yaml . They complement each other.

Yes, you can do some linear multi-step work and put everything in one Javascript/Python/any_language code Action and that’s no problem with that, if that works to start with something.

However to achieve something more useful/complex which includes nested/branched logic Workflows are inevitable.

Examples:



Can you handle that nicely with just Python/Javascript code placed in one Action?

Workflows are known to have 40 control patterns: http://www.workflowpatterns.com/patterns/control/ if not more. It becomes hard to write and maintain code which needs more advanced stuff. This is done with workflows:

  • run several tasks in parallel
  • merge their results
  • split
  • join
  • filter
  • conditions
  • loops
  • synchornize
  • retry
  • crash recovery, etc.
  • logging
  • even wait for confirmation from human when needed (Inquiries — StackStorm 3.2.0 documentation)

That’s where workflows help with its own DSL, structure, maintainability, readability, organization and clarity, while common actions are re-usable and shareable bricks. This gives a good separation of concerns and abstraction. StackStorm provides a framework to also observe the flow via UI like that, debug, manage individual steps and so on.

Think about it ^^ from the perspective of Monolyth vs Microservices approach. Pretty similar strategy here with the Workflows. Hope that helps.

1 Like

Thank you @armab for yours replies!

I’m sure it’s possible to do it using Orquesta, but it could be overkill for me (at least at the beginning), and sure, using Rules I can bind events to custom scripts in any language, but my initial question is a little bit different.

Let’s suppose that I write a Python script equivalent to the previous example.
The invocation of the activeDirectoryLookupUser pseudo-function could be translated in the execution of the action get_ad_user from the pack stackstorm-activedirectory.
So I would like to use it in my action, maybe through a high level API provided by StackStorm, instead of adopting some kind of library.

You said that it is an anti-pattern, but in my opinion it is a valid option when you want all the power of a programming language, along with the flexibility of StackStorm.

Sure, I think you’ve got the workaround described in Invoke an action from a Python action script for a quick start.

@m4dcoder What are your opinions about workflows as a python code in StackStorm actions?