Do not dump the config to disk


(Giorgos Karagkounis) #1

Is there a way I can skip the dump of the configs to the file system as pack_name.yaml on the /opt/stackstorm/config folder?
Right now having a jinja template argument as the value you have to update the corresponding key-value pair on the store. And you cannot do partial updates with the API on the configs endpoint. So updating a config that has some secret and some non-secret values from another system you have to do some work around that is not so easy.


(Lindsay Hill) #2

Are you referring to when using st2 pack config, or something else?


(Giorgos Karagkounis) #3

I refer to using the St2 API to update configs. I guess st2 pack config uses the same API.


(Lindsay Hill) #4

Correct, st2 pack config uses the /configs API.

I don’t think that writing configs to disk (or not) is actually your issue here - it sounds like there’s something a bit more going on, related to synchronizing related activities (datastore + configs). Can you explain a bit more about what you’re trying to do, and where it’s currently breaking down? That might help with trying to figure out a solution.


(Giorgos Karagkounis) #5

The thing I would like to achieve is that i don’t want to write the secrets of a pack’s configuration on a file on disk. So I tried to use the dynamic configuration explained on StackStorm docs using a Jinja template notations. If you want to update the configs though with the API these values are required so you have to update them too. You can overcome it by getting the secret client side and update it with the same value for example {{ st2kv.system.pack_name_api_secret_key }}. Something like that is done on St2 front-end.

But if the user want to update the secret of the config the the new value should be sent by the client.
This has as a result the config yaml file gets the new value and now you don’t have the dynamic assignment of the secret. The secret is on the file as plain text.


(Lindsay Hill) #6

If you want to update the secret, then why not just update the k/v you have in the datastore? Why do you need to update the configuration when you’re updating that dynamic value?


(Giorgos Karagkounis) #7

I would like to avoid this logic on client side.

Optimal: Client ask for config schema and config values. Then makes a put request for the pack with user’s input values.

Right Now:
If I have to update the k/v then : Client put request with the new values that are not secret and completing the secret values with the old {{ st2kv.system.pack_name_secret_key }} and then make a request for the key value store. This is too much logic to implement client side at the moment.


(Giorgos Karagkounis) #8

Searching on source code my suggestion is that there should be a configuration flag on st2 that desides if you want to dump configs on disk.


(Lindsay Hill) #9

I still don’t think that dumping configs to disk is your underlying problem here.

Optimal: Client ask for config schema and config values. Then makes a put request for the pack with user’s input values.

Right Now:
If I have to update the k/v then : Client put request with the new values that are not secret and completing the secret values with the old {{ st2kv.system.pack_name_secret_key }} and then make a request for the key value store. This is too much logic to implement client side at the moment.

In your Optimal scenario above, are you not using the datastore at all? Are you just setting those values in the configuration? Where do the datastore updates happen in that scenario?

How does avoiding writing the config to disk help in that scenario?

But if the user want to update the secret of the config the the new value should be sent by the client.

So just update the value in the datastore using the API. What does the pack config have to do with it, assuming your pack config has already been set up?


(Giorgos Karagkounis) #10

In my optimal scenario I just update tha values on Datastore. No need to have a yaml file or anything. And values that are secret get encrypted on the vault…

The just update the values on the datastore is not so easy if you have mixed values that are not dynamic.

Let me give you an explain here:

mypack.yaml :


api_secret:  "{{ st2kv.system.pack_name_api_secret }} "
base_url: www.example.com

Client -> GET https://{st2server}/api/v1/config_schema
Client -> GET https://{st2server}/api/v1/config/pack_name?show_secrets=true

User change api secret= New_value and base_url = www.service.com.

Now Client needs to keep the old secret ({{st2kv.system.pack_name_api_secret}}) value and :

Client -> PUT https://{st2server}/api/v1/config/pack_name
Payload:

 { 
   "api_secret": "{{st2kv.system.pack_name_api_secret}}",
   "base_url": "www.service.com"
}

And also update the kv store:

Client -> PUT https://{st2server}/api/v1/key/pack_name_api_secret
Payload:

{ 
  "name": "api_secret",
  "value": "New Value",
  "secret": true
}

I do not have an option to implement this logic on the client side.


(Lindsay Hill) #11

In my optimal scenario I just update tha values on Datastore

So why not just do that? Why are you updating the configuration every time?

Why not have your initial pack configuration reference datastore values, and then for changes you only need to update the datastore? In your above example base_url could also be a datastore value. It doesn’t have to be a secret.

What is your client in this case - presumably it is something custom - why is it unable to make two calls instead of one?