St2packs bloated

Why are the st2packs 115M + per pack? I can’t seem to figure out how to get this thinner.

That’s certainly not normal. The AWS pack has tons of actions and the size on disk only amounts to 21MB, so something is amiss.

There could be a few reasons I can think of:

  • You are counting the files in the pack, and all of the Python packages that it installs, in your number
  • You have an abnormally large file in your pack (the pack icon maybe?)

I have a few questions:

  1. How are you measuring the size of a pack?
  2. What is the output of du -h | tail -n1 when run from the root of the pack?

st2pack is the sidecar for k8. Simple du -f in the virtualenv root.

in lib/python3.6 of ANY of the virtuals, including the /opt/stackstorm/st2 virt env. They are all bloated with about 80Mb of extra stuff. Not sure how or why.

@blag /opt/stackstorm/virtualenvs/helloworld # du -h | tail -n1
113.4M
Sorry, missed #2 earlier today. Busy day.

Here is the bloat.
/opt/stackstorm/virtualenvs/helloworld/lib/python3.6/config-3.6m-x86_64-linux-gnu # du -h | tail -n1
80.9M .

Note, helloworld has 0 in requirements.txt

If you are talking about virtual environments itself (which I believe you are), that’s kind of a known issue.

For compatibility reasons, we don’t symlink various system level Python dependencies into pack specific virtual environment, but copy them over which results in rather larger pack virtual environment directories.

I forgot the reason why we decided to copy libraries over instead of symlinks, but there was some compatibility issue with some distro.

From changelog:

  • Use --always-copy option when creating virtualenv for packs from packs.setup_virtualenv action. This is required when st2actionrunner is kicked off from python within a virtualenv.

The biggest offender seems to be python3.6/config-3.6m-x86_64-linux-gnu directory which contains shared libraries (~80 MB).

So we should probably research again if can symlink that stuff from parent venv (/opt/stackstorm/st2) into pack one without breaking things.

So probably best next step is to open a feature request / research ticket against StackStorm/st2 github repo.

You could also try setting this config option to empty string (st2/st2.package.conf at master · StackStorm/st2 · GitHub) and test it works, but there are no guarantees.