Questions regarding st2 pack on stackstorm HA in kubernetes

Hi Team,

I have few queries regarding stackstorm packs for stackstorm ha on kubernetes.

1)I figured out to how to create a docker image for service now and create a docker image and use it helm configuration

2)I am trying to copy the st2 packs from standalone server to the stackstorm ha in kubernetes.
The names are like test_ansible,hello_st2 etc

when I try to create a docker image gettign the error:

docker build -t st2packs_test:0.9 --build-arg PACKS=hello_st2 st2packs-image

2020-11-24 14:27:06,523 INFO [-] Installing pack “hello_st2”
2020-11-24 14:27:06,839 ERROR [-] Failed to install pack “hello_st2”: No record of the “hello_st2” pack in the index.
The command ‘/bin/sh -c /opt/stackstorm/st2/bin/st2-pack-install ${PACKS}’ returned a non-zero code: 2

Is there a way I can add the name to the index ,if so how to do

[root@dc1udataocs301 st2packs-dockerfiles]# docker build -t st2packs_test:0.9 --build-arg PACKS=hello_st2 st2packs-image
Sending build context to Docker daemon 5.662MB
Step 1/7 : ARG PACKS=“file:///tmp/stackstorm-st2”
Step 2/7 : FROM stackstorm/st2packs:builder AS builder

Executing 2 build triggers

—> Using cache
—> Running in da4ddcebc38b
Removing intermediate container da4ddcebc38b
—> 3b1cdbeeb9ab
Step 3/7 : COPY packs/hello_st2 /tmp/stackstorm-st2/
—> 20edfa1b5f12
Step 4/7 : RUN ls -la /tmp/stackstorm-st2
—> Running in 04766987ebc3
total 12
drwxr-xr-x 7 root root 137 Nov 24 14:26 .
drwxrwxrwt 1 root root 28 Nov 24 14:26 …
drwxr-xr-x 2 root root 40 Aug 12 18:04 actions
drwxr-xr-x 2 root root 25 Aug 12 18:04 aliases
-rw-r–r-- 1 root root 3936 Aug 12 17:58 icon.png
-rw-r–r-- 1 root root 1285 Aug 12 17:58 pack.yaml
drwxr-xr-x 2 root root 65 Aug 12 18:04 policies
-rw-r–r-- 1 root root 9 Aug 12 17:58 requirements.txt
drwxr-xr-x 2 root root 24 Aug 12 18:04 rules
drwxr-xr-x 2 root root 44 Aug 12 18:04 sensors
Removing intermediate container 04766987ebc3
—> b685fd70a9da
Step 5/7 : RUN git config --global http.sslVerify false
—> Running in 8d38579121e3
Removing intermediate container 8d38579121e3
—> 4d6386076124
Step 6/7 : RUN /opt/stackstorm/st2/bin/st2-pack-install {PACKS} ---> Running in 24a099672558 2020-11-24 14:27:06,523 INFO [-] Installing pack "hello_st2" 2020-11-24 14:27:06,839 ERROR [-] Failed to install pack "hello_st2": No record of the "hello_st2" pack in the index. The command '/bin/sh -c /opt/stackstorm/st2/bin/st2-pack-install {PACKS}’ returned a non-zero code: 2

3)I tried to create multiple st2packs eg:

docker build --build-arg PACKS=servicenow ansible jira -t st2:0.1 st2packs-image

but no luck then I tried

docker build -t st2packs_test:0.2 --build-arg PACKS=servicenow --build-arg PACKS=ansible --build-arg PACKS=jira st2packs-image

the command executes only the last one eg servicenow ,ansible and jira ,it takes only jira.

Also facing the issue for ansible pack when installed alone ,for standalone also was facing the issue ,then install gcc with
yum install gcc and it worked ,not sure how do it for docker

Step 1/7 : ARG PACKS=“file:///tmp/stackstorm-st2”
Step 2/7 : FROM stackstorm/st2packs:builder AS builder

Executing 2 build triggers

—> Using cache
—> Using cache
—> 16cc22f2cba9
Step 3/7 : COPY packs/hello_st2 /tmp/stackstorm-st2/
—> Using cache
—> 4b2b0294c1c7
Step 4/7 : RUN ls -la /tmp/stackstorm-st2
—> Using cache
—> d6125dc06c83
Step 5/7 : RUN git config --global http.sslVerify false
—> Using cache
—> 5b68c4ea031a
Step 6/7 : RUN /opt/stackstorm/st2/bin/st2-pack-install ${PACKS}
—> Running in 9e02c88c3d1d
2020-11-24 15:17:51,629 INFO [-] Installing pack “ansible”
2020-11-24 15:17:52,430 INFO [-] Successfully installed pack “ansible”
2020-11-24 15:17:52,431 INFO [-] Setting up virtualenv for pack “ansible”
2020-11-24 15:17:52,432 INFO [-] Virtualenv path “/opt/stackstorm/virtualenvs/ansible” doesn’t exist

error:
----------------------------------------
ERROR: Command “/opt/stackstorm/virtualenvs/ansible/bin/python -u -c ‘import setuptools, tokenize;file=’”’"’/tmp/pip-install-stuuz20m/pykerberos/setup.py’"’"’;f=getattr(tokenize, ‘"’"‘open’"’"’, open)(file);code=f.read().replace(’"’"’\r\n’"’"’, ‘"’"’\n’"’"’);f.close();exec(compile(code, file, ‘"’"‘exec’"’"’))’ install --record /tmp/pip-record-em0jpki_/install-record.txt --single-version-externally-managed --compile --install-headers /opt/stackstorm/virtualenvs/ansible/include/site/python3.6/pykerberos" failed with error code 1 in /tmp/pip-install-stuuz20m/pykerberos/
WARNING: You are using pip version 19.1.1, however version 20.2.4 is available.
You should consider upgrading via the ‘pip install --upgrade pip’ command.
)

pip install --upgrade pip
Requirement already up-to-date: pip in /usr/local/lib/python3.6/site-packages (20.2.4)

docker build -t st2packs_test:0.2 --build-arg PACKS=servicenow --build-arg PACKS=jira --build-arg PACKS=ansible st2packs-image

docker build -t st2packs_test:0.2 --build-arg PACKS=ansible st2packs-image

Instead of:
--build-arg PACKS=ansible jira
you need:
--build-arg PACKS="ansible jira"

Thanks ,looks like
docker build --build-arg PACKS=“servicenow ansible jira” -t st2:0.9 st2packs-image
is working fine ,I was check on it and update you.

I wanted to copy the standalone packs from my standalone to the kubernetes how to do it ,I checked docker file location if possible ,I am getting not part of index issue

—> Running in eae1b84f4245
2020-11-24 16:04:09,862 INFO [-] Installing pack “servicenow”
2020-11-24 16:04:10,554 INFO [-] Successfully installed pack “servicenow”
2020-11-24 16:04:10,554 INFO [-] Setting up virtualenv for pack “servicenow”
2020-11-24 16:04:10,555 INFO [-] Virtualenv path “/opt/stackstorm/virtualenvs/servicenow” doesn’t exist
2020-11-24 16:04:18,488 INFO [-] Successfully set up virtualenv for pack “servicenow”
2020-11-24 16:04:18,488 INFO [-] Installing pack “hello_st2”
2020-11-24 16:04:18,584 ERROR [-] Failed to install pack “hello_st2”: No record of the “hello_st2” pack in the index.
The command ‘/bin/sh -c /opt/stackstorm/st2/bin/st2-pack-install ${PACKS}’ returned a non-zero code: 2

Documentation highlights how to install a pack from the local file system:

If you’re just starting with StackStorm I’d also recommend to try the Vagrant docker image Vagrant & Virtual Appliance — StackStorm 3.3.0 documentation which will get you up & running system quickly before diving deep into complexity with Docker and K8s.

Thanks I followed the document you provided
GitHub - StackStorm/st2packs-dockerfiles: Source for st2packs-builder and st2packs-runner images

we are using st2 on standalone servers and are doing POC for stackstom Ha on kubernetes

I am not sure what I am missing I am still getting the error

I copied my local pack test_ansible under the directory of st2packs-dockerfiles/st2packs-image

The dockerfile under st2packs-dockerfiles/st2packs-image contains the below detail

ARG PACKS=“file:///tmp/stackstorm-st2”

FROM stackstorm/st2packs:builder AS builder

considering you have your “local” pack under the stackstorm-st2 dir relative to Dockerfile

Here we copy local “stackstorm-st2” dir into Docker’s “/tmp/stackstorm-st2”

COPY test_ansible /tmp/stackstorm-st2/

Check it

RUN ls -la /tmp/stackstorm-st2

RUN /opt/stackstorm/st2/bin/st2-pack-install ${PACKS}
FROM stackstorm/st2packs:runtime

docker build --build-arg PACKS=“test_ansible” -t st2:0.9 st2packs-image
Sending build context to Docker daemon 5.674MB
Step 1/6 : ARG PACKS=“file:///tmp/stackstorm-st2”
Step 2/6 : FROM stackstorm/st2packs:builder AS builder

Executing 2 build triggers

—> Using cache
—> Using cache
—> cf4cf71c0f5a
Step 3/6 : COPY test_ansible /tmp/stackstorm-st2/
—> 41d826b03385
Step 4/6 : RUN ls -la /tmp/stackstorm-st2
—> Running in 4c4116737451
total 16
drwxr-xr-x 4 root root 116 Nov 24 16:44 .
drwxrwxrwt 1 root root 28 Nov 24 16:44 …
drwxrwxrwx 3 root root 69 Nov 24 13:51 actions
-rwxrwxrwx 1 root root 67 Sep 30 10:31 inventory
-rwxrwxrwx 1 root root 744 Nov 24 13:44 nicompress.yaml
-rwxrwxrwx 1 root root 440 Sep 30 11:57 nideletefile.yaml
-rwxrwxrwx 1 root root 227 Sep 30 10:33 pack.yaml
drwxrwxrwx 2 root root 41 Sep 30 12:54 rules
Removing intermediate container 4c4116737451
—> 01887da8098c
Step 5/6 : RUN /opt/stackstorm/st2/bin/st2-pack-install {PACKS} ---> Running in 055ba79781e5 2020-11-24 16:44:08,675 INFO [-] Installing pack "test_ansible" 2020-11-24 16:44:08,792 ERROR [-] Failed to install pack "test_ansible": No record of the "test_ansible" pack in the index. The command '/bin/sh -c /opt/stackstorm/st2/bin/st2-pack-install {PACKS}’ returned a non-zero code: 2

Looks like you’re mixing: PACKS=“file:///tmp/stackstorm-st2” vs PACKS=“test_ansible”

Thanks Eugen ,for the help ,yes I was mixing up Packs got is working now

Dockerfile

#######################################
ARG PACKS=“file:///tmp/stackstorm-st2”

FROM stackstorm/st2packs:builder AS builder

considering you have your “local” pack under the stackstorm-st2 dir relative to Dockerfile

Here we copy local “stackstorm-st2” dir into Docker’s “/tmp/stackstorm-st2”

COPY packs/hello_st2 /tmp/stackstorm-st2/

Check it

RUN ls -la /tmp/stackstorm-st2

RUN /opt/stackstorm/st2/bin/st2-pack-install ${PACKS}
FROM stackstorm/st2packs:runtime
##########################################

pwd:st2packs-dockerfiles/st2packs-image

created packs and inside packs created hello_st2

pwd:/st2packs-dockerfiles/st2packs-image/packs

total 28
-rw-r–r-- 1 test snm 9 Aug 12 11:58 requirements.txt
-rw-r–r-- 1 test snm 1285 Aug 12 11:58 pack.yaml
-rw-r–r-- 1 test snm 3936 Aug 12 11:58 icon.png
drwxr-xr-x 2 test snm 4096 Aug 12 12:04 actions
drwxr-xr-x 2 test snm 4096 Aug 12 12:04 aliases
drwxr-xr-x 2 test snm 4096 Aug 12 12:04 policies
drwxr-xr-x 2 test snm 4096 Aug 12 12:04 rules
drwxr-xr-x 2 test snm 4096 Aug 12 12:04 sensors

##############

docker build -t hello_st2:v0.0.2 st2packs-image
Sending build context to Docker daemon 5.677MB
Step 1/6 : ARG PACKS=“file:///tmp/stackstorm-st2”
Step 2/6 : FROM stackstorm/st2packs:builder AS builder

Executing 2 build triggers

—> Using cache
—> Using cache
—> bf9c631b7800
Step 3/6 : COPY packs/hello_st2 /tmp/stackstorm-st2/
—> 84da7e531ad5
Step 4/6 : RUN ls -la /tmp/stackstorm-st2
—> Running in 17617093689d
total 12
drwxr-xr-x 7 root root 137 Nov 25 08:30 .
drwxrwxrwt 1 root root 28 Nov 25 08:30 …
drwxr-xr-x 2 root root 40 Aug 12 18:04 actions
drwxr-xr-x 2 root root 25 Aug 12 18:04 aliases
-rw-r–r-- 1 root root 3936 Aug 12 17:58 icon.png
-rw-r–r-- 1 root root 1285 Aug 12 17:58 pack.yaml
drwxr-xr-x 2 root root 65 Aug 12 18:04 policies
-rw-r–r-- 1 root root 9 Aug 12 17:58 requirements.txt
drwxr-xr-x 2 root root 24 Aug 12 18:04 rules
drwxr-xr-x 2 root root 44 Aug 12 18:04 sensors
Removing intermediate container 17617093689d
—> 0e1451a395cd
Step 5/6 : RUN /opt/stackstorm/st2/bin/st2-pack-install ${PACKS}
—> Running in eccaaea63314
2020-11-25 08:30:33,714 INFO [-] Installing pack “file:///tmp/stackstorm-st2”
2020-11-25 08:30:33,734 INFO [-] Successfully installed pack “hello_st2”
2020-11-25 08:30:33,734 INFO [-] Setting up virtualenv for pack “hello_st2”
2020-11-25 08:30:33,735 INFO [-] Virtualenv path “/opt/stackstorm/virtualenvs/hello_st2” doesn’t exist
2020-11-25 08:30:39,394 INFO [-] Successfully set up virtualenv for pack “hello_st2”
Removing intermediate container eccaaea63314
—> cfbf9fc228fb
Step 6/6 : FROM stackstorm/st2packs:runtime

Executing 3 build triggers

—> Using cache
—> Using cache
—> 62902078a65b
Successfully built 62902078a65b
Successfully tagged hello_st2:v0.0.2