Jsonpath-Query fails with Unexpected character:?

Hi,

I’m currently evaluation stackstorm and to be honest - I think I fell in love with it. Especially since I discovered jsonpath today.

But I have an issue with jsonpath queries.

This is the JSON object:

{
"target_health_state_out": [
{
  "TargetHealthDescriptions": [
    {
      "HealthCheckPort": "443",
      "Target": {
        "Id": "xxx1",
        "Port": 443
      },
      "TargetHealth": {
        "State": "unhealthy",
        "Reason": "Target.ResponseCodeMismatch",
        "Description": "Health checks failed with these codes: [503]"
      }
    },
    {
      "HealthCheckPort": "443",
      "Target": {
        "Id": "xxx3",
        "Port": 443
      },
      "TargetHealth": {
        "State": "healthy"
      }
    }
  ],
  "ResponseMetadata": {
    "RetryAttempts": 0,
    "HTTPStatusCode": 200,
    "RequestId": "38c05da5-6374-4f84-96f7-a58749af3ace",
    "HTTPHeaders": {
      "x-amzn-requestid": "38c05da5-6374-4f84-96f7-a58749af3ace",
      "date": "Mon, 23 Dec 2019 16:42:03 GMT",
      "content-length": "1019",
      "content-type": "text/xml"
    }
  }
},
{
  "TargetHealthDescriptions": [
    {
      "HealthCheckPort": "443",
      "Target": {
        "Id": "xxx1",
        "Port": 443
      },
      "TargetHealth": {
        "State": "unhealthy",
        "Reason": "Target.ResponseCodeMismatch",
        "Description": "Health checks failed with these codes: [502]"
      }
    }
  ],
  "ResponseMetadata": {
    "RetryAttempts": 0,
    "HTTPStatusCode": 200,
    "RequestId": "ddc3c7d5-6ff4-4216-b2a0-942b10e65b44",
    "HTTPHeaders": {
      "x-amzn-requestid": "ddc3c7d5-6ff4-4216-b2a0-942b10e65b44",
      "date": "Mon, 23 Dec 2019 16:42:04 GMT",
      "content-length": "760",
      "content-type": "text/xml"
    }
  }
},
{
  "TargetHealthDescriptions": [
    {
      "HealthCheckPort": "443",
      "Target": {
        "Id": "xxx3",
        "Port": 443
      },
      "TargetHealth": {
        "State": "healthy"
      }
    }
  ],
  "ResponseMetadata": {
    "RetryAttempts": 0,
    "HTTPStatusCode": 200,
    "RequestId": "cb0c4967-68ee-4a58-ae2c-b8309aa6d028",
    "HTTPHeaders": {
      "x-amzn-requestid": "cb0c4967-68ee-4a58-ae2c-b8309aa6d028",
      "date": "Mon, 23 Dec 2019 16:42:05 GMT",
      "content-length": "621",
      "content-type": "text/xml"
    }
  }
},
]
}

Now I want to select the Target.Id for all unhealthy targets. The jsonpath-query that works in Python and validators is:

<% jsonpath_query(ctx(all_targets_health_state), '$..TargetHealthDescriptions[?(@.TargetHealth.State=="unhealthy")].Target.Id') %>

These queries work fine as long as I don’t have any condition in there. But as soon as I add ?(@... I get the following error:

{
  "output": null,
  "errors": [
    {
      "message": "YaqlEvaluationException: Unable to evaluate expression '<% jsonpath_query(ctx(all_targets_health_state), '$..TargetHealthDescriptions[?(@.TargetHealth.State==\"unhealthy\")].Target.Id') %>'. JsonPathLexerError: Error on line 1, col 28: Unexpected character: ? ",
  "type": "error",
  "route": 0,
  "task_id": "seperateHealthyAndUnhealthyTargets"
   }
 ]
}

This is the whole task from the orquesta workflow:

  seperateHealthyAndUnhealthyTargets:
  action: core.echo
  input:
  # message: 'str(<% jsonpath_query(ctx(all_targets_health_state), "$..TargetHealth.State") %>)' <- this one works fine
  message: str(<% jsonpath_query(ctx(all_targets_health_state), '$..TargetHealthDescriptions[?(@.TargetHealth.State=="unhealthy")].Target.Id') %>)

I tried a whole bunch of different options regarding the quoting and tried to avoid any kind of escaping issue but I couldn’t figure it out.

I found posts in other communities that jsonpath-rw for example does not support filter extensions and jsonpath-rw-ext is required if you want to use filter extensions. Am I on the right track here? Any hint in any direction is highly appreciated!

Best regards,
Marcel

Ok, in the meantime I did some tests and can verify that this kind of extended query is not supported by jsonpath_rw. I will take a look at the more recent containers to see if this is an issue in the stackstorm setup with Python3, too. Will create an issue (and maybe a PR?!) if that’s the case.

Does anyone have an idea how to mitigate the issue in the demo environment? I could write the json to a file and parse it with JQ but there might be smarter solutions. I would also like to avoid to use my own python script as action for the current demo.

Best regards,
Marcel

Yep, you’re on the right track!

Using jq might be the most straightforward workaround, but you don’t necessarily need to write it to a file. You can use core.local (which is just a Bash shell) to echo out the all_targets_health_state context variable (in JSON) and immediately pipe it into jq:

seperateHealthyAndUnhealthyTargets:
  action: core.local
  input:
    command: "<% echo 'str(ctx(all_targets_health_state))' | jq '$..TargetHealthDescriptions[?(@.TargetHealth.State==\"unhealthy\")].Target.Id')) %>"

You will want to write/debug your Bash command manually before having StackStorm execute it, but that should get you started with jq.

If you’re interested in fixing this, you can either open a PR to ST2 that switches out the jsonpath-rw dependency for the jsonpath-rw-ext project (assuming that the latter is reverse compatible), or you can try to add support for filter extensions to jsonpath-rw itself, and then a small ST2 PR to bump the version of that dependency up to your version.

Hi @blag,

thank you! Using a local command is indeed a smarter solution than mine.

I think I’ll raise at least an issue but am not sure about the PR. I am currently working on different PRs and don’t want to start too much in parallel without getting anything done in the end. Otherwise this might be a good start to get in touch with the stackstorm code. We’ll see. :slight_smile: