How to Package Your Python Code Without Setting Your Hair on Fire

CC BY-SA https://500px.com/photo/67820671/fire-by-axel-schliemann

Dorian Puła

Software Development Engineer @ Points

@dorianpula

The Problem

You want to:

• Distribute your cool Python script or app.
• Share a modular library with coworkers.
• Make installation or deployment of code simpler.

The Promise

pip + virtualenv = ☺ + success!

mkvirtualenv venv
pip install my_cool_library

The Reality

Python packaging = ☹ + heartache

• Packages on PyPI break.
• Or PyPI breaks.
• Relocatable virtualenvs are flakey.
• It never actually works on the first time.
• The API and documentation sucks.
• Packaging in general sucks.

Here Be Dragons

A Twisted History of Python Packaging - PyCon CA 2012 http://pyvideo.org/video/1601/twisted-history-of-python-packaging

Packaging Made Simple

CC BY-SA https://www.flickr.com/photos/gcwest/281385799

A Simple Setup.py

from setuptools import setup, find_packages
setup(
    name='justcheckers',
    version='0.5.0',
    url='http://justcheckers.org/',
    license='GPL v3',
    author='Dorian Pula',
    author_email='dorian.pula@gmail.com',
    description='An advanced cross-platform checkers game.',

    install_requires=gather_requirements(),

    packages=find_packages(exclude=['tests']),
    include_package_data=True,

    classifiers=[ 'Development Status :: 3 - Alpha', ..., ],
)

Requirements

Use a requirements.txt

# User interface
PySide==1.2.2
Markdown==2.4.1
PyOpenGL>=3.0.0

# Core logic
enum34==0.9.19

Parsing Requirements

Homegrown

def gather_requirements():
    with open('requirements.txt') as req_file:
        raw_requirements = req_file.readlines()

    return [req.strip()
            for req in raw_requirements
            if req.strip() and not re.match('#|-(?!e)', req)]

Using PIP's API

import pip
def gather_requirements():
    return pip.req.parse_requirements('requirements.txt')

Manifests

Remember to include and check your MANIFEST.in

include requirements.txt
include assets
include setup.py
include docs/*.rst
exclude docs/*.html

Advanced Package & Tricks

CC BY-SA https://www.flickr.com/photos/stuckincustoms/5914448597

Testing Packages

The Quick and Dirty Way

pip install -e .

The Proper Way

python setup.py sdist

mkdir dist/my_library && cp dist/my_library*.tar.gz dist/my_library

python -m SimpleHTTPServer 9000

mkvirtualenv new_venv

pip install --index-url=http://127.0.0.1:9000/dist my_library==0.1.2.3

Console Scripts

Adding a console command

entry_points={
    'console_scripts': [
        'justcheckers = justcheckers.app:main',
    ]
},

Packaging in Scripts

bin=['scripts/deploy.sh']

Advice On Using PIP and Setuptools APIs

• The PIP maintainers would rather you didn't.

• PIP API is not publicized. setuptool's docs are not fun.

• Capturing output from pip or python setup.py often easier than API grokking.

  • Use python setup --name / --version to get a current package's info.
  • Use pip freeze to get installed requirements.

Advice On Extending setup.py

• Please don't. Use a task runner instead:

  • invoke - http://docs.pyinvoke.org/
  • fabric - http://fabfile.org/
  • paver - http://pythonhosted.org/Paver/

• If you insist, look at extend distutils.core.Commands

More About Python Packaging

• PIP Read the Docs: http://pip.readthedocs.org/
• Python Packaging User Guide: https://packaging.python.org/
• Distributing Python Modules: https://docs.python.org/2.7/distutils/index.html
• justCheckers Sample Project: http://github.com/dorianpula/justcheckers/tree/pyside-migration

Thank You!

CC BY-SA https://www.flickr.com/photos/bigpresh/7389234452

• Dorian Puła
• Twitter: @dorianpula
• Email: dorian.pula@points.com / dorian.pula@gmail.com
• WWW: http://dorianpula.ca/