Arch linux and python

Python/Virtual environment

virtualenv is a tool used to create an isolated workspace for a Python application. It has various advantages such as the ability to install modules locally, export a working environment, and execute a Python program in that environment.

Contents

Overview

A virtual environment is a directory into which some binaries and shell scripts are installed. The binaries include python for executing scripts and pip for installing other modules within the environment. There are also shell scripts (one for bash, csh, and fish) to activate the environment. Essentially, a virtual environment mimics a full system install of Python and all of the desired modules without interfering with any system on which the application might run.

In 2017, Pipenv was published which manages all the above tools — managing virtual environments of python interpreters, package dependencies, their activation and reproducible locking of versions in Pipfiles.

Installation

Python 3.3+ comes with a module called venv. For applications that require an older version of Python, virtualenv must be used.

Packages

Install one of these packages from the official repositories to use a Python virtual environment.

Usage

All three tools use a similar workflow.

Creation

Use venv or virtualenv to create the virtual environment within your project directory. Be sure to exclude the venv directory from version control—a copy of pip freeze will be enough to rebuild it.

This tool is provided by python (3.3+):

virtualenv

Use virtualenv for Python 3, available in python-virtualenv .

And virtualenv2 for Python 2, available in python2-virtualenv .

Activation

Use one of the provided shell scripts to activate and deactivate the environment. This example assumes bash is used.

Once inside the virtual environment, modules can be installed with pip and scripts can be run as normal.

To exit the virtual environment, run the function provided by bin/activate :

Python versions

The binary versions depend on which virtual environment tool was used. For instance, the python command used in the Python 2 example points to bin/python2.7 , while the one in the venv example points to bin/python3.7 .

One major difference between venv and virtualenv is that the former uses the system’s Python binary by default:

The virtualenv tool uses a separate Python binary in the environment directory:

virtualenvwrapper

virtualenvwrapper allows more natural command line interaction with your virtual environments by exposing several useful commands to create, activate and remove virtual environments. This package is a wrapper for both python-virtualenv and python2-virtualenv .

Installation

Now add the following lines to your

The line source /usr/bin/virtualenvwrapper.sh can cause some slowdown when starting a new shell. To fix this try using source /usr/bin/virtualenvwrapper_lazy.sh , which will load virtualenvwrapper the first time a virtualenvwrapper function is called.

Re-open your console and create the WORKON_HOME folder:

Basic usage

The main information source on virtualenvwrapper usage (and extension capability) is Doug Hellmann’s page.

Create the virtual environment:

Activate the virtual environment:

Install some package inside the virtual environment (say, Django):

After you have done your things, leave the virtual environment:

Pipenv

pipenv allows better managed CLI interactions by providing a single program that does all the functions of the above tools.

Installation

Basic usage

All commands can be executed in the project folder, and pipenv will recognize the specific situation — whether a virtualenv exists in the directory, locating it, and running on the specific virtual interpreter when pipenv is executed.

Источник

Python

Python is an interpreted, interactive, object-oriented programming language. It incorporates modules, exceptions, dynamic typing, very high level dynamic data types, and classes. It supports multiple programming paradigms beyond object-oriented programming, such as procedural and functional programming. Python combines remarkable power with very clear syntax. It has interfaces to many system calls and libraries, as well as to various window systems, and is extensible in C or C++. It is also usable as an extension language for applications that need a programmable interface. Finally, Python is portable: it runs on many Unix variants including Linux and macOS, and on Windows.

Contents

Installation

Python 3

Python 3 is the latest and actively developed version of the language. See What’s New in Python to see the latest changes in Python 3.

To install the current release of Python 3, install the python package.

If you would like to build the latest RC/betas from source, visit Python Downloads. The Arch User Repository also contains good PKGBUILDs. If you do decide to build the RC, note that the binary (by default) installs to /usr/local/bin/python3.x . As an alternative which does not require superuser capabilities and installs to the home directory, consider using pyenv .

Python 2

Python 2 is an older version of the language. Python 3 and Python 2 are incompatible. For an overview of the differences, see the historical version of the Python2orPython3 document.

Although Python 2 is no longer actively maintained, there are some packages that still depend on it. Python 2 may also be useful for developers maintaining, using or porting legacy Python 2 software.

To get the latest version of Python 2, install the python2 package.

Python 2 will happily run alongside Python 3. You need to specify python2 in order to run this version.

Any program requiring Python 2 needs to use /usr/bin/python2 , instead of /usr/bin/python , which points to Python 3. However, many legacy Python 2 scripts incorrectly specify python in their shebang line. To change this, open the program or script in a text editor and change the first line. The line may show one of the following:

In either case, change python to python2 and the program will then use Python 2 instead of Python 3.

Another way to force the use of python2 without altering the scripts is to call it explicitly with python2 :

Finally, you may not be able to control the script calls, but there is a way to trick the environment. It only works if the scripts use #!/usr/bin/env python . It will not work with #!/usr/bin/python . This trick relies on env searching for the first corresponding entry in the PATH variable.

First create a dummy folder:

Then add a symlink python to python2 and the config scripts in it:

Finally put the new folder at the beginning of your PATH variable:

To check which python interpreter is being used by env , use the following command:

A similar approach in tricking the environment, which also relies on #!/usr/bin/env python to be called by the script in question, is to use a virtual environment.

Alternative implementations

The python package installs CPython, the reference implementation of Python. However, there are also other implementations available. These implementations are usually based on older versions of Python and are not fully compatible with CPython.

Implementations available on Arch Linux include:

• PyPy — A Python implementation written in Python. It has speed and memory usage advantages compared to Cython.

https://www.pypy.org || pypy , pypy3

• Jython — An implementation of the Python language written in Java. It can be used to embed Python scripting into Java programs or use Java libraries in Python programs.

https://www.jython.org/ || jython

• micropython — Python for microcontrollers. It includes a small subset of the Python standard library and is optimized to run on microcontrollers and in constrained environments.

https://micropython.org/ || micropythonAUR

• IronPython — An implementation of the Python programming language which is tightly integrated with .NET. It can use .NET libraries and allows .NET programs to use Python libraries.

https://ironpython.net || ironpython-gitAUR

More implementations exist. Some, such as Stackless, Pyston and Cinder are used internally at large technology companies. Others are historically notable but are no longer maintained due to improvements in the most popular implementations.

Alternative shells

The python package includes an interactive Python shell/REPL which can be launched with the python command. The following shells are also available:

Old versions

Old versions of Python are available via the AUR and may be useful for historical curiosity, old applications that do not run on current versions, or for testing Python programs intended to run on a distribution that comes with an older version:

Extra modules/libraries for old versions of Python may be found on the AUR by searching for python , e.g. searching for python26 for 2.6 modules.

Package management

There are several ways to install Python packages on Arch Linux:

• Official repositories and AUR — A large number of popular packages are available in the Arch repositories. This is the preferred way to install system-wide packages.
• pip(1) — The official package installer for Python. You can use pip to install packages from the Python Package Index and other indexes.

https://pip.pypa.io/ || python-pip

• pipx — Closely related to pip, but creates, for the user running it, an isolated environment for each application and its associated packages, preventing conflicts with system packages. Focused on packages that can be run from the command line directly as applications. You can use pipx to install packages from the Python Package Index and other indexes.

https://pypa.github.io/pipx/ || python-pipx

• Anaconda — An open source package management system and environment management system, originally created for Python programs. You can use Conda to install packages from the Anaconda repositories.

https://docs.conda.io/projects/conda/ || anacondaAUR

• Miniconda — A lightweight alternative to Anaconda which installs the package manager but does not install scientific computing packages by default.

https://docs.conda.io/en/latest/miniconda.html || miniconda3AUR

When installing packages using pip, it is recommended to use a virtual environment to prevent conflicts with system packages in /usr . Alternatively, pip install —user can be used to install packages into the user scheme instead of /usr . pipx and Conda integrate environment management into their workflows.

See the Python Packaging User Guide for the official best practices for package management.

Historically, easy_install (part of python-setuptools ) was used to install packages distributed as Eggs. easy_install and Eggs have been replaced with pip and Wheels. See pip vs easy_install and Wheel vs Egg for more information.

Widget bindings

The following widget toolkit bindings are available:

To use these with Python, you may also need to install the associated widget toolkit packages (e.g. tk must also be installed to use Tkinter).

Tips and tricks

Virtual environment

Python provides tools to create isolated virtual environments into which packages may be installed without conflicting with other virtual environments or the system packages. Virtual environments can also run applications with different versions of Python on the same system.

Tab completion in Python shell

Tab completion is available in the interactive shell by default. Note that the readline completer will only complete names in the global namespace. You can use python-jedi for a richer tab completion experience [1].

Источник

Python package guidelines

This document covers standards and guidelines on writing PKGBUILDs for Python software.

Contents

Package naming

For Python 3 library modules, use python-modulename . Also use the prefix if the package provides a program that is strongly coupled to the Python ecosystem (e.g. pip or tox). For other applications, use only the program name.

The same applies to Python 2 except that the prefix (if needed) is python2- .

Architecture

A Python package that contains C extensions using the ext_modules keyword in setup.py , is architecture-dependent. Otherwise it is most likely architecture-independent.

Source

Download URLs linked from the PyPI website include an unpredictable hash that needs to be fetched from the PyPI website each time a package must be updated. This makes them unsuitable for use in a PKGBUILD. PyPI provides an alternative stable scheme: PKGBUILD#source source=() array should use the following URL templates:

Source package https://files.pythonhosted.org/packages/source/$<_name::1>/$_name/$_name-$pkgver.tar.gz Pure Python wheel package https://files.pythonhosted.org/packages/py2.py3/$<_name::1>/$_name/$<_name>-$pkgver-py2.py3-none-any.whl (Bilingual – Python 2 and Python 3 compatible) https://files.pythonhosted.org/packages/py3/$<_name::1>/$_name/$<_name>-$pkgver-py3-none-any.whl (Python 3 only) Note that the distribution name can contain dashes, while its representation in a wheel filename cannot (they are converted to underscores). Arch specific wheel package in this example for source_x86_64=(‘. ‘) . Also _py=cp38 can be used to not repeat the python version: https://files.pythonhosted.org/packages/$_py/$<_name::1>/$_name/$<_name>-$pkgver-$_py-$<_py>m-manylinux1_x86_64.whl

Note that a custom _name variable is used instead of pkgname since python packages are generally prefixed with python- . This variable can generically be defined as follows:

Installation methods

Python packages are generally installed using language-specific tools, such as pip or easy_install (deprecated), which are comparable to dedicated package managers in that they are designed to fetch the source files from an online repository (usually PyPI, the Python Package Index) and track the relevant files (for a detailed comparison between the two, see pip vs easy_install).

However, for managing Python packages from within PKGBUILD s, the standard-provided distutils proves to be the most convenient solution since it uses the downloaded source package’s setup.py and easily installs files under $pkgdir/usr/lib/python

distutils

A distutils PKGBUILD is usually quite simple:

• python is replaced with the proper binary, python or python2
• —root=»$pkgdir» prevents trying to directly install in the host system instead of inside the package file, which would result in a permission error
• —optimize=1 compiles optimized bytecode files (.pyo for Python 2, .opt-1.pyc for Python 3) so they can be tracked by pacman instead of being created on the host system on demand
• Adding —skip-build optimizes away the unnecessary attempt to re-run the build steps already run in the build() function, if that is the case.

setuptools

The Python packaging scene has largely migrated from distutils to setuptools, which is actively developed and functions as a drop-in replacement import in setup.py . The main difference for packagers is that setuptools is packaged separately from Python itself, and must be specified as a makedepends :

If the resulting package includes executables which import the pkg_resources module, then setuptools must be additionally specified as a depends in the split package_*() functions; alternatively, if the PKGBUILD only installs the Python package for a single version of Python, setuptools should be moved from makedepends to depends .

Some packages try to use setuptools and fall back to distutils if setuptools could not be imported. In this case, setuptools should be added as a makedepends , so that the resulting python metadata is better.

If a package needs setuptools to build due to including executables (which is not supported by distutils), but only imports distutils, then building will raise a warning, and the resulting package will be broken (it will not contain the executables):

An upstream bug should be reported. To work around the problem, an undocumented setuptools feature can be used:

If you need to use pip (provided by python-pip and python2-pip ), e.g. for installing wheel packages, remember to pass the following flags:

/.config>/pip.conf that may be appending arbitrary flags to pip.
—isolated ignores environment variables (and again

/.config>/pip/pip.conf ) that may otherwise also be appending arbitrary flags to pip.

—ignore-installed is necessary until https://github.com/pypa/pip/issues/3063 is resolved (otherwise pip skips the install in the presence of an earlier —user install).

—no-deps ensures, that dependencies do not get packaged together with the main package.

pip does not know how to generate .pyo (or for 3.5 and later, .pyc) files (see https://github.com/pypa/pip/issues/2209). In order to generate them manually after pip has installed the module, run:

Build-time 2to3 translation

Most Python projects target either Python 2 or Python 3, or target both using compatibility layers like six. However, some use the deprecated 2to3 keyword in setuptools to heuristically convert the source code from Python 2 to Python 3 at build time. As a result, the same source directories cannot be used to build both Python 2 and Python 3 split packages.

For packages that do this, we need a prepare() function that copies the source before it is built. Then the Python 2 and Python 3 packages can be converted and built independently without overriding each other.

pyproject.toml (PEP 517)

If an upstream only provides a pyproject.toml (PEP 517) in their source tarball, most of the time it is possible to convert it to a setup.py and use #setuptools as usual.

Add python-dephell to makedepends and generate the setuptools integration in prepare() :

setuptools without a setup.py

The PEP 517 build-backend setuptools.build_meta does not need a setup.py file to exist, and if one does not exist, a minimal one is implied (most such projects will, however, include the minimal one in the upstream tarball):

This breaks compatibility with non-PEP 517 installation methods, such as Linux distributions (as of June 2020, there is no option other than pip). Painful hacks are needed to build; either create the minimal setup.py yourself, or use

An example of this is ansible-lint.

This page will eventually be updated once a non-gross way is found.

Check

Most python projects providing a testsuite use nosetests or pytest to run tests with test in the name of the file or directory containing the testsuite. In general, simply running nosetests or pytest is enough to run the testsuite.

If there is a compiled C extension, the tests need to be run using a $PYTHONPATH , that reflects the current major and minor version of Python in order to find and load it.

Some projects provide setup.py entry points for running the test. This works for both pytest and nosetests .

Tips and tricks

Discovering detached PGP signatures on PyPi

If detached PGP signatures for a given Python sdist tarball exist, they should be used to verify the tarball. However, the signature files do not show up directly in the files download section of any given project on pypi.org. To discover the sdist tarballs and their potential signature files, it is possible to use this service to get an overview per project: https://pypi.debian.net/

Using site-packages

Sometimes during building, testing or installation it is required to refer to the system’s site-packages directory. To not hardcode the directory, use a call to the system Python version to retrieve the path and store it in a local variable:

Источник