Custom Stackstorm Image

packs
docker

(Vivek Veerabahu Subramanian) #1

Hi all, We are planning on making a custom image of Stackstorm(for deploying with docker) inclusive of certain packs such as Kafka and Ansible, etc, then we wish to bake the image- We want to modify the image specified in this link: GitHub - StackStorm/st2-docker: Official docker container for StackStorm.. Is there any example code - where I can include a pack on entrypoint.sh or are there examples of modified docker files to enable an addition of a pack, request your help on this, thanks! Apart from the Github stackstorm docker pages, I have gone through the following links:
st2-docker/runtime/entrypoint.d at master · StackStorm/st2-docker · GitHub


(Lindsay Hill) #2

From Slack:

probably not an example of exactly what you want to do with that, but there’s a lot of ways to peel that onion from a Docker perspective. One way is to bake it in, which would require you to place the pack files into the container that you wish to have access to.

I suggest you follow st2-dockerfiles and stackstorm-ha for inspiration.

Since a lot of that work has been done for you


(Vivek Veerabahu Subramanian) #3

Thanks Lindsay for taking interest in my post, I have found a Github page-written by Shu Sugimoto, link: st2-docker-pack-prebuild-example/Dockerfile at master · shusugmt/st2-docker-pack-prebuild-example · GitHub
This is very similar to what I was looking for- I could build the image of the Dockerfile in the link as well, however am not able to build the dockerfile when I try to modify it- my goal is to include the Kafka Pack into the - to be created StackStorm image, Request help with this, here is the Dockerfile that I tried to build :

FROM stackstorm/stackstorm:latest AS base

COPY setup-pack-virtualenv.py /setup-pack-virtualenv.py


FROM base AS st2-examples

RUN cd /opt/stackstorm/packs \
 && git clone https://github.com/shusugmt/st2-pack-examples examples \
 && /setup-pack-virtualenv.py --pack examples


FROM base AS st2-napalm

RUN cd /opt/stackstorm/packs \
 && git clone https://github.com/stackstorm-exchange/stackstorm-napalm napalm \
 && /setup-pack-virtualenv.py --pack napalm

FROM base

RUN /setup-pack-virtualenv.py --pack core \
 && /setup-pack-virtualenv.py --pack packs \
 && /setup-pack-virtualenv.py --pack linux \
 && /setup-pack-virtualenv.py --pack chatops \
 && /setup-pack-virtualenv.py --pack kafka
 

COPY --from=st2-examples /opt/stackstorm/packs/examples /opt/stackstorm/packs/examples
COPY --from=st2-examples /opt/stackstorm/virtualenvs/examples /opt/stackstorm/virtualenvs/examples

COPY --from=st2-napalm /opt/stackstorm/packs/napalm /opt/stackstorm/packs/napalm
COPY --from=st2-napalm /opt/stackstorm/virtualenvs/napalm /opt/stackstorm/virtualenvs/napalm

The output that I got was :

Sending build context to Docker daemon  66.05kB
    Step 1/12 : FROM stackstorm/stackstorm:latest AS base
     ---> 28432a1994dc
    Step 2/12 : COPY setup-pack-virtualenv.py /setup-pack-virtualenv.py
     ---> Using cache
     ---> dba4337a6d11
    Step 3/12 : FROM base AS st2-examples
     ---> dba4337a6d11
    Step 4/12 : RUN cd /opt/stackstorm/packs  && git clone https://github.com/shusugmt/st2-pack-examples examples  && /setup-pack-virtualenv.py --pack examples
     ---> Using cache
     ---> cabb31dd60ee
    Step 5/12 : FROM base AS st2-napalm
     ---> dba4337a6d11
    Step 6/12 : RUN cd /opt/stackstorm/packs  && git clone https://github.com/stackstorm-exchange/stackstorm-napalm napalm  && /setup-pack-virtualenv.py --pack napalm
     ---> Using cache
     ---> 393ae69e7913
    Step 7/12 : FROM base
     ---> dba4337a6d11
    Step 8/12 : RUN /setup-pack-virtualenv.py --pack core  && /setup-pack-virtualenv.py --pack packs  && /setup-pack-virtualenv.py --pack linux  && /setup-pack-virtualenv.py --pack chatops  && /setup-pack-virtualenv.py --pack kafka
     ---> Running in f6aa9ccb09fd
    /opt/stackstorm/st2/local/lib/python2.7/site-packages/cryptography/hazmat/primitives/constant_time.py:26: CryptographyDeprecationWarning: Support for your Python version is deprecated. The next version of cryptography will remove support. Please upgrade to a 2.7.x release that supports hmac.compare_digest as soon as possible.
      utils.DeprecatedIn23,
    2019-03-19 16:00:19,860 INFO [-] Virtualenv path "/opt/stackstorm/virtualenvs/core" doesn't exist
    /opt/stackstorm/st2/local/lib/python2.7/site-packages/cryptography/hazmat/primitives/constant_time.py:26: CryptographyDeprecationWarning: Support for your Python version is deprecated. The next version of cryptography will remove support. Please upgrade to a 2.7.x release that supports hmac.compare_digest as soon as possible.
      utils.DeprecatedIn23,
    2019-03-19 16:00:23,284 INFO [-] Virtualenv path "/opt/stackstorm/virtualenvs/packs" doesn't exist
    /opt/stackstorm/st2/local/lib/python2.7/site-packages/cryptography/hazmat/primitives/constant_time.py:26: CryptographyDeprecationWarning: Support for your Python version is deprecated. The next version of cryptography will remove support. Please upgrade to a 2.7.x release that supports hmac.compare_digest as soon as possible.
      utils.DeprecatedIn23,
    2019-03-19 16:00:26,165 INFO [-] Virtualenv path "/opt/stackstorm/virtualenvs/linux" doesn't exist
    /opt/stackstorm/st2/local/lib/python2.7/site-packages/cryptography/hazmat/primitives/constant_time.py:26: CryptographyDeprecationWarning: Support for your Python version is deprecated. The next version of cryptography will remove support. Please upgrade to a 2.7.x release that supports hmac.compare_digest as soon as possible.
      utils.DeprecatedIn23,
    2019-03-19 16:00:39,616 INFO [-] Virtualenv path "/opt/stackstorm/virtualenvs/chatops" doesn't exist
    /opt/stackstorm/st2/local/lib/python2.7/site-packages/cryptography/hazmat/primitives/constant_time.py:26: CryptographyDeprecationWarning: Support for your Python version is deprecated. The next version of cryptography will remove support. Please upgrade to a 2.7.x release that supports hmac.compare_digest as soon as possible.
      utils.DeprecatedIn23,
    Traceback (most recent call last):
      File "/setup-pack-virtualenv.py", line 41, in <module>
        sys.exit(main())
      File "/setup-pack-virtualenv.py", line 37, in main
        setup_pack_virtualenv(cfg.CONF.pack, logger=LOG)
      File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/util/virtualenvs.py", line 85, in setup_pack_virtualenv
        raise Exception(msg)
    Exception: Pack "kafka" is not installed. Looked in: /opt/stackstorm/packs
    The command '/bin/sh -c /setup-pack-virtualenv.py --pack core  && /setup-pack-virtualenv.py --pack packs  && /setup-pack-virtualenv.py --pack linux  && /setup-pack-virtualenv.py --pack chatops  && /setup-pack-virtualenv.py --pack kafka' returned a non-zero code: 1

(Eugen C.) #4

@vivekrmk See the following descriptions to build a Docker image with custom packs: GitHub - StackStorm/stackstorm-ha: K8s Helm Chart (βeta!) that codifies StackStorm High Availability fleet as a simple to use reproducible infrastructure-as-code app. and StackStorm HA Cluster in Kubernetes - BETA — StackStorm 2.10.4 documentation

And Helper Docker images to bootstrap custom packs https://hub.docker.com/r/stackstorm/st2packs/

This is however all supported only by our K8s deployment (StackStorm HA Cluster in Kubernetes - BETA — StackStorm 2.10.4 documentation)