Package your python code (with requirements) into a standalone zip file.
zipapps
is a pure-python library
, without any 3rd-party dependencies.
Inspired by shiv but unlike shiv
- zipapps may not unzip cache when without any C language-based library or dynamic modules(
.so/.pyd
), such asrequests
,bottle
, or your own pure python code. - cache path to unzip is
./zipapps_cache
by default, not theHOME
path. - the cache folders will be reused for the same app name, not like shiv to create many new versions and use much disk space.
- you can install requirements with
pip
while first running(not packaging) by lazy install mode, for cross-platform publishing and reducing your.pyz
file size. - combine multiple
venv.pyz
files for your code flexibly.
.pyz
to Python is like .jar
to Java. They are both zip archive files which aggregate many packages and associated metadata and resources (text, images, etc.) into one file for distribution. Then what you only need is a Python Interpreter as the runtime environment.
PS: The extension name .pyz could be any other suffixes even without ext names, so you can rename app.pyz
to app.zip
or app.py
or others as you wish.
Depends on PEP441, zipapp & zipimport.
pip install zipapps -U
- zipapps in
lazy install
mode-
python3 -m zipapps -c -d -a entry.py -m entry:main -o app.pyz aiohttp psutils
- the file size of
app.pyz
will be very small for args-c
(compressed) and-d
(lazy install mode) - the requirements will be installed while first running
- the file size of
-
- run app.pyz
-
python3 app.pyz
- waiting for
pip install
- sometimes you need to add
--user
to args or runchmod
for permission error in a shared environment
- waiting for
-
- zipapps with requirements
-
python3 -m zipapps -c -u AUTO -a entry.py -m entry:main -o app.pyz aiohttp psutils
- so you need not to install requirements at running
- but ensure the compatibility of the system environment and python version
-
- run app.pyz
-
python3 app.pyz
- libs with
.pyd/.so
caches will be unzipped to the./zipapps_cache/app
by-u AUTO
-
- zipapps with requirements
-
python3 -m zipapps -c -u AUTO -o venv.pyz -r requirements.txt
-
- run entry.py with venv.pyz
-
python3 venv.pyz entry.py
- cache will be unzipped to
./zipapps_cache/venv
for-u
is not null
-
Advance Usage
1.1 Code of script.py
print('ok')
1.2 build the app.pyz
python3 -m zipapps -c -a script.py -m script -p /usr/bin/python3
1.3 run this app
python3 app.pyz
or
./app.pyz
ouput
ok
Details:
-c:
compress files, 7.6KB => 3.3KB
-a:
add the script file to app.pyz, so you can use it while running
-m:
set the entry_point, also can be set as `-m script:main` as long as you has the main function
-p:
set the shebang line, so you can use `./app.pyz` instead of `python3 app.pyz`
2.1 The package example
and the code of __main__.py
└── example
├── __init__.py
└── __main__.py
def main():
print('ok')
if __name__ == "__main__":
main()
2.2 build the example.pyz
python3 -m zipapps -c -a example -o example.pyz -m example.main:main -p /usr/bin/python3
2.3 Run with python3 example.pyz
or ./example.pyz
output
ok
Details:
-m:
set the entry_point with format like `package.model:function`
-o:
set the output file path, you can set it some other paths like `/home/centos/example.abc` and run with `python3 /home/centos/example.abc`
no more new args.
3.1 The package example
and the code of __main__.py
└── example
├── __init__.py
└── __main__.py
def main():
import bottle
print(bottle.__version__)
if __name__ == "__main__":
main()
3.2 build the example.pyz
with requirements installed
python3 -m zipapps -c -a example -o example.pyz -m example.main:main bottle
3.3 Run with python3 example.pyz
or ./example.pyz
Output
0.12.19
Details:
bottle:
all the unhandled args like `bottle` will be used to `pip install`, so you can write `bottle` in requirements.txt and use like `-r requirements.txt`
WARNING: if the requirements have .pyd/.so
files, you should unzip them while running, and the pure python libs like bottle
or requests
no need to unzip anything. Read the 4th paragraph for more info.
4.1 The package example
and the code of __main__.py
└── example
├── __init__.py
└── __main__.py
def main():
import psutil
print(psutil.__version__)
if __name__ == "__main__":
main()
4.2 build the example.pyz
with requirements installed
python3 -m zipapps -c -a example -o example.pyz -m example.main:main -u AUTO -up TEMP/cache psutil
4.3 Run with python3 example.pyz
or ./example.pyz
Output
5.8.0
Details:
-u:
means the file or folder names you want to unzip while running. Here is the `AUTO`, will unzip the psutil package because of its .pyd or .so files included.
-up:
the unzip path of cache folder. TEMP / HOME / SELF are the built-in runtime args, but for the stable usage you can ignore this arg then use `./zipapps_cache/example`. The cache will be refreshed while you rebuild this pyz.
WARNING: unzip path can be overwrited by export ZIPAPPS_CACHE=./xxx
or export UNZIP_PATH=./xxx
while running.
5.1 Code of script.py
import bottle, requests
print(bottle.__version__, requests.__version__)
5.2 build the venv.pyz
python3 -m zipapps -c -o venv.pyz -p /usr/bin/python3 bottle requests
5.3.1 use the venv.pyz
like a middleware
python3 venv.pyz script.py
5.3.2 use the venv.pyz
like the interpreter
./venv.pyz script.py
even like is also valid
python3 venv.pyz -c "import bottle,requests;print(bottle.__version__, requests.__version__)"
Output
0.12.19 2.25.1
Details:
No `-m` arg here, then the pyz file will do like an interpreter which contains the installed requirements.
So you can use it like this:
> python3 venv.pyz
>>> import bottle
>>> bottle.__file__
6.1 Here is script.py
again
import bottle, requests
print(bottle.__version__, requests.__version__)
6.2 Build the script.pyz
python3 -m zipapps -c -o script.pyz -a script.py -m script requests
6.3 Build the requests.pyz
and bottle.pyz
respectively
python3 -m zipapps -c -o requests.pyz requests
python3 -m zipapps -c -o bottle.pyz bottle
6.4 And now run the script.pyz
with two requirements
python3 script.pyz --zipapps=bottle.pyz,requests.pyz
Output
0.12.19 2.25.1
Details:
--zipapps:
This arg will help you run some zipapp with given venv.pyz files, the paths is separated by commas.
7.1 Here is script.py
again again
import six
print(six.__file__)
6.2 Build the script.pyz
, this is very fast without downloading and installing 3rd packages.
python3 -m zipapps -c -o script.pyz -a script.py -m script -d six
6.3 Run this .pyz
file, and the requirements will be installed while first running.
python3 script.pyz
Output
Looking in indexes: https://pypi.xxx.com/simple/
Collecting six
Using cached https://xxx/packages/ee/ff/xxxx/six-1.15.0-py2.py3-none-any.whl (10 kB)
Installing collected packages: six
Successfully installed six-1.15.0
/tmp/zipapps_cache/script/_zipapps_lazy_pip/3.8/Linux/six.py
Details:
-d:
Lazy install mode is useful for distributing your cross-platform apps.
most common args:
-c
- to compress files, only for python3.7+.
-a xxx.py
- to add some files/folders into the zipped file.
-u=AUTO
- auto unzip the .pyd / .so files
-r requirements.txt
- install requirements with
pip install
- install requirements with
-o my_app.pyz
- output the zipped file as given path
-m app.__main__:main
- set the entry point
-p /usr/bin/python3
- set the
shebang
line
- set the
-d
- lazy install mode, requirements will be installed with
pip
while first running - Very useful
- zip file size will be very small, and the default unzip path is
SELF/zipapps_cache/
- lazy install mode, requirements will be installed with
Details:
python3 -m zipapps -h
-h, --help
- show the simple doc
--includes, --add, -a
- The given paths will be copied to
cache_path
while packaging, which can be used while running. The path strings will be splited by ",".- such as
my_package_dir,my_module.py,my_config.json
- often used for libs not from
pypi
or some special config files
- such as
- the
output
arg ofzipapps.create_app
- The given paths will be copied to
--output, -o
- The path of the output file, defaults to
app.pyz
. - the
output
arg ofzipapps.create_app
- The path of the output file, defaults to
--python, -p
- The path of the Python interpreter which will be set as the
shebang line
, defaults toNone
.- with shebang
/usr/bin/python3
you can run app with./app.pyz
directly, no need forpython3 app.pyz
- with shebang
- the
interpreter
arg ofzipapps.create_app
- The path of the Python interpreter which will be set as the
--main, -m
- The entry point function of the application, the
valid format
is:package.module:function
package.module
module:function
package
- the
main
arg ofzipapps.create_app
- WARNING: If the
--main
arg is set,python3 app.pyz
will not be able to used as venv likepython3 app.pyz xxx.py
- The entry point function of the application, the
--compress, -c
Boolean
value, compress files with the deflate method or not.- the
compressed
arg ofzipapps.create_app
--unzip, -u
- The names which need to be unzipped while running, splited by ","
without ext
, such asbottle,aiohttp
, or the complete path likebin/bottle.py,temp.py
. For.so/.pyd
files(which can not be loaded by zipimport), or packages with operations of static files.- if unzip is set to "*", then will unzip all files and folders.
- if unzip is set to AUTO, then will add the
.pyd
and.so
files automatically.
- the
unzip
arg ofzipapps.create_app
- The names which need to be unzipped while running, splited by ","
--unzip-path, -up
- If
unzip
arg is not null, cache files will be unzipped to the given path while running. Defaults tozipapps_cache
, support some internal variables as runtime args:TEMP/HOME/SELF
as prefix, for exampleHOME/zipapps_cache
TEMP
meanstempfile.gettempdir()
HOME
meansPath.home()
SELF
means.pyz
file path.
- And you can also overwrite it with environment variables:
ZIPAPPS_CACHE
orUNZIP_PATH
- the
unzip_path
arg ofzipapps.create_app
- If
-cc, --pyc, --compile, --compiled
- Compile .py to .pyc for fast import, but zipapp does not work unless you unzip it(so NOT very useful).
- the
compiled
arg ofzipapps.create_app
--cache-path, --source-dir, -cp
- The cache path of zipapps to store site-packages and
includes
files. If not set, will create and clean-up in TEMP dir automately. - the
cache_path
arg ofzipapps.create_app
--shell, -s
- Only while
main
is not set, used for shell=True insubprocess.run
.- very rarely used, because extra sub-process is not welcome
- the
shell
arg ofzipapps.create_app
- Only while
--main-shell, -ss
- Only for
main
is not null, callmain
withsubprocess.Popen
:python -c "import a.b;a.b.c()"
. This is used forpsutil
ImportError of DLL load.- very rarely used too
- the
main_shell
arg ofzipapps.create_app
- Only for
--strict-python-path, -spp
Boolean
value. Ignore global PYTHONPATH, only usezipapps_cache
andapp.pyz
.- the
ignore_system_python_path
arg ofzipapps.create_app
-b, --build-id
- The string to skip duplicate builds, it can be the paths of files/folders which splited by ",", then the modify time will be used as build_id. If build_id contains
*
, will useglob
function to get paths. For example, you can set requirements.txt as your build_id bypython3 -m zipapps -b requirements.txt -r requirements.txt
when you use pyz as venv.- very rarely used too too
- the
build_id
arg ofzipapps.create_app
- The string to skip duplicate builds, it can be the paths of files/folders which splited by ",", then the modify time will be used as build_id. If build_id contains
--zipapps, --env-paths
- Default
--zipapps
arg if it is not given while running. Support TEMP/HOME/SELF prefix.
- Default
--delay, -d, --lazy-pip, --lazy-install, --lazy-pip-install
- Install packages with pip while first running, which means requirements will not be install into pyz file.
- all the other (or
unknown
) args will be used by "pip install"- such as
-r requirements.txt
- such as
bottle aiohttp
- the
pip_args
arg ofzipapps.create_app
- such as
available args while the
.pyz
is running
--zipapps
- including some other pyz into PYTHONPATH
- often be used as
multiple venv combination
- for example
- building
python3 -m zipapps -o six.pyz six
python3 -m zipapps -o psutil.pyz -u AUTO psutil
python3 -m zipapps -o bottle.pyz bottle
- run
python3 six.pyz --zipapps=psutil.pyz,bottle.pyz -c "import psutil, bottle"
- building
- Package your code(package or model) into one zipped file.
- Pure python code without any 3rd-party dependencies.
- Python code with 3rd-party dependencies installed together.
- Some dependencies need to unzip them into the cache folder for dynamic modules(
.so/.pyd
) files exist, such aspsutil
.- This type of
pyz
isNOT cross-platform
.
- This type of
- Some dependencies need not to unzip them, such as requests / bottle.
- Some dependencies need to unzip them into the cache folder for dynamic modules(
- Python code with requirements but not be installed while building. (Recommended)
- The
lazy install
mode by the arg-d
. - But will need to be install at the first running(only once).
- This is
cross-platform
andcross-python-version
because of their installation paths is standalone.
- The
- Run with python interpreter from the venv path.
- which means the requirements(need to be unzipped) will be installed to the
venv
folder, not inpyz
file. - build your package into one
pyz
file with-m package.module:function -p /venv/bin/python
. - run the
pyz
file with/venv/bin/python app.pyz
or./app.pyz
.
- which means the requirements(need to be unzipped) will be installed to the
- Hadoop-Streaming's mapper & reducer scripts.
- Simple deployment towards different servers with
jenkins
, or other CI/CD tools.- Easy to uploads a clean
standalone
zip file.
- Easy to uploads a clean
- Distribute
zipapp
with embedded python, or share python programs to your friends (someone with python installed). - Use as a requirements zip path, or some
venv
usages.import sys;sys.path.insert(0, 'app.pyz')
(without .so/.pyd)python3 app.pyz script.py
- Other usages need to be found, and enjoy yourself.