How to execute an external program within a python Action in asynchronize way


(Jimmy0628) #1

I am trying to start a logstash worker process from an action, but the action cannot exit normally unless I manually kill the logstash work.

def start_logstash(self, path_to_config, path_to_logstash):
    self.__update_logstash_config(path_to_config)
    executable = path_to_logstash + '/logstash'
    p = subprocess.Popen(executable + ' -f ' + path_to_config, stdout=subprocess.PIPE, shell=True, stderr=subprocess.PIPE, close_fds=True)

It seems that in the python action, the popen did not return immediatly as it does in normal python scripts. However, actually my purpose is just start the program and exit the action. Is there any restrction? Thank you


#2

What happens when you run the same command directly in a Python shell and Popen?

path_to_logstash =  # set this manually
path_to_config  =  # set this manually
executable = path_to_logstash + '/logstash'
p = subprocess.Popen(executable + ' -f ' + path_to_config, stdout=subprocess.PIPE, shell=True, stderr=subprocess.PIPE, close_fds=True)

(Jimmy0628) #3

it executes successfuly and exits, the subprocess will keep running after the main process exits if I run the same command directly in a Python shell.


#4

Is there anything from the subprocess stdout or stderr?


(Dmitri Zimine) #5

Did you consider running it as a shell action (local-shell-script runner)? Even if it’s a python script, you still can do it, and it would work just like it works in normal python. Not sure about the context which you’re trying to start logstash but just consider.


(Jimmy0628) #6

Actually yesterday I think I found the reason why it is being blocked. the stdout keeps printing messages to the stdout and stderror. I tried to use the Popen and set the stdout and stderr to NULL, but the problem persisted. However, if I use os.system(logstash_cmd + ‘>/dev/null 2>&1 &’) it worked.

As I know, the os.system is not recommended in the python scripts because of security reason, but Popen just simply did not work.


(Jimmy0628) #7

Thanks, I will have a try, I suppose it is a better solution.


(Jimmy0628) #8

I was trying to start the logstash with an updated configuration file and leave the logstash keeps running after I confirmed the status of the newly spawned logstash is in fully function. Yes, shell action is another option. Just curious if the Popen issue was observed by somebody else? :slight_smile:


#9

Glad you got it working! I didn’t find any similar issues when I did a quick search through our public GitHub, but I’ll keep an eye out for any similar issues.

Our Slack community might have other people who have also run into this issue. If you wish to find other people who have run into this issue so you can solve it (and use Popen to properly run your logstash command), I would recommend asking around in the #community channel.

However, this forum is still the best place for any technical questions or issues you may have.