terra incognita

Python package development environment setup

2015-03-06 13:34

Introduction

Since I got lots of questions on how to arrange and develop a Python package, I decided to write down everything about the what I learnt about it, finally. Just try to summarize the things.

Workflow

The goal is to create a Python project (or a package), including several models, in a well-defined approach.

Let's assume the project is called "forecaster", which tells you if it will snow tomorrow.

Set up the repository

The first thing is to create a directory. But we are not going to use mkdir to do so. In stead, because we want to have a well-organized structure for this project, it is a good practice to have a real Python package.

Here we could employ Paste Script to create your package. It will create the basic skeleton for your new project.

Install Paste Script by using pip:

[sudo] pip install PasteScript

Then you need find a place for your package now.

paster create -t basic_package forecaster

This command will create a project named forecaster. After answering questions (or simply enter nothing), you can find the forecaster project folder.

cd forecaster

We can see a sub-directory with the same name and something else:

.
├── forecaster
├── forecaster.egg-info
├── setup.cfg
└── setup.py

Virtualenv

It is recommended to have an isolated environment for your development.

[sudo] pip install virtualenv

After installing virtualenv, you can now create a virtualenv here.

virtualenv --no-site-packages env

If you prefer Python 3, use the following instead:

virtualenv -p python3 --no-site-packages env

Check the files again:

.
├── env
├── forecaster
├── forecaster.egg-info
├── setup.cfg
└── setup.py

Having the env directory now, we can activate the virual environment by:

. env/bin/activate

Note the leading dot (.) matters. Now welcome to the virtualenv world.

Development mode

When developing your project, you can "install" your pacakge in the development mode.

But why? Of couse you can write your code in the way you like. The key point is that when you import your package modules, you may encounter relavtive path issues, which may be something like ImportError.

Let's go into the development mode (make sure enable virtualenv first)

cd PARENT_PATH/forecaster
python setup.py develop

You can see your package is already in your pip list by:

pip freeze

The output should include something like:

forecaster===0.1dev-r0

Write the code

First go to the source folder:

cd PARENT_PATH/forecaster/forecaster

Create two files:

The model file: models.py

class Syracuse(object):

    def __init__(self):
        self.city_name = 'Syracuse'

    def forecast(self):
        return 'YES'

Another file: main.py

from forecaster.models import Syracuse


def will_it_snow_tomorrow(forecast_model):
    city = forecast_model()
    print('Will it snow tomorrow in {}? {}.'.format(
        city.city_name,
        city.forecast())
    )


def main():
    will_it_snow_tomorrow(Syracuse)

if __name__ == '__main__':
    main()

Then we can test our code:

python main.py

The output is:

Will it snow tomorrow in Syracuse? YES.

The answer is extremely accurate.

Back to the path issue

So what happens if we are not with the development mode in Virtualenv? To see the problem, just exit Virtualenv by:

deactivate

Okay, now run the main file again:

Traceback (most recent call last):
  File "main.py", line 1, in <module>
    from forecaster.models import Syracuse
ImportError: No module named forecaster.models

Got the point? The relative path is not available now.

Git setup

Well, you should know how to do that:

git init

There is a typical .gitignore file here.

Analyze your code

You may use Pylint to check your code.

Reference