python / peps Public

PEP: 631 Title: Dependency specification in pyproject.toml based on PEP 508 Author: Ofek Lev <ofekmeister@gmail.com> Sponsor: Paul Ganssle <paul@ganssle.io> Discussions-To: https://discuss.python.org/t/5018 Status: Accepted Type: Standards Track Content-Type: text/x-rst Created: 20-Aug-2020 Post-History: 20-Aug-2020 Resolution: https://discuss.python.org/t/how-to-specify-dependencies-pep-508-strings-or-a-table-in-toml/5243/38

Abstract

This PEP specifies how to write a project's dependencies in a pyproject.toml file for packaging-related tools to consume using the :pep:`fields defined in PEP 621 <621#dependencies-optional-dependencies>`.

Entries

All dependency entries MUST be valid :pep:`PEP 508 strings <508>`.

Build backends SHOULD abort at load time for any parsing errors.

from packaging.requirements import InvalidRequirement, Requirement

...

try:
    Requirement(entry)
except InvalidRequirement:
    # exit

Specification

dependencies

• Format: array of strings
• Related core metadata:
  • Requires-Dist

Every element MUST be an entry.

[project]
dependencies = [
  'PyYAML ~= 5.0',
  'requests[security] < 3',
  'subprocess32; python_version < "3.2"',
]

optional-dependencies

• Format: table
• Related core metadata:
  • Provides-Extra
  • Requires-Dist

Each key is the name of the provided option, with each value being the same type as the dependencies field i.e. an array of strings.

[project.optional-dependencies]
tests = [
  'coverage>=5.0.3',
  'pytest',
  'pytest-benchmark[histogram]>=3.2.1',
]

Example

This is a real-world example port of what docker-compose defines.

[project]
dependencies = [
  'cached-property >= 1.2.0, < 2',
  'distro >= 1.5.0, < 2',
  'docker[ssh] >= 4.2.2, < 5',
  'dockerpty >= 0.4.1, < 1',
  'docopt >= 0.6.1, < 1',
  'jsonschema >= 2.5.1, < 4',
  'PyYAML >= 3.10, < 6',
  'python-dotenv >= 0.13.0, < 1',
  'requests >= 2.20.0, < 3',
  'texttable >= 0.9.0, < 2',
  'websocket-client >= 0.32.0, < 1',

  # Conditional
  'backports.shutil_get_terminal_size == 1.0.0; python_version < "3.3"',
  'backports.ssl_match_hostname >= 3.5, < 4; python_version < "3.5"',
  'colorama >= 0.4, < 1; sys_platform == "win32"',
  'enum34 >= 1.0.4, < 2; python_version < "3.4"',
  'ipaddress >= 1.0.16, < 2; python_version < "3.3"',
  'subprocess32 >= 3.5.4, < 4; python_version < "3.2"',
]

[project.optional-dependencies]
socks = [ 'PySocks >= 1.5.6, != 1.5.7, < 2' ]
tests = [
  'ddt >= 1.2.2, < 2',
  'pytest < 6',
  'mock >= 1.0.1, < 4; python_version < "3.4"',
]

Implementation

Parsing

from packaging.requirements import InvalidRequirement, Requirement

def parse_dependencies(config):
    dependencies = config.get('dependencies', [])
    if not isinstance(dependencies, list):
        raise TypeError('Field `project.dependencies` must be an array')

    for i, entry in enumerate(dependencies, 1):
        if not isinstance(entry, str):
            raise TypeError(f'Dependency #{i} of field `project.dependencies` must be a string')

        try:
            Requirement(entry)
        except InvalidRequirement as e:
            raise ValueError(f'Dependency #{i} of field `project.dependencies` is invalid: {e}')

    return dependencies

def parse_optional_dependencies(config):
    optional_dependencies = config.get('optional-dependencies', {})
    if not isinstance(optional_dependencies, dict):
        raise TypeError('Field `project.optional-dependencies` must be a table')

    optional_dependency_entries = {}

    for option, dependencies in optional_dependencies.items():
        if not isinstance(dependencies, list):
            raise TypeError(
                f'Dependencies for option `{option}` of field '
                '`project.optional-dependencies` must be an array'
            )

        entries = []

        for i, entry in enumerate(dependencies, 1):
            if not isinstance(entry, str):
                raise TypeError(
                    f'Dependency #{i} of option `{option}` of field '
                    '`project.optional-dependencies` must be a string'
                )

            try:
                Requirement(entry)
            except InvalidRequirement as e:
                raise ValueError(
                    f'Dependency #{i} of option `{option}` of field '
                    f'`project.optional-dependencies` is invalid: {e}'
                )
            else:
                entries.append(entry)

        optional_dependency_entries[option] = entries

    return optional_dependency_entries

Metadata

def construct_metadata_file(metadata_object):
    """
    https://packaging.python.org/specifications/core-metadata/
    """
    metadata_file = 'Metadata-Version: 2.1\n'

    ...

    if metadata_object.dependencies:
        # Sort dependencies to ensure reproducible builds
        for dependency in sorted(metadata_object.dependencies):
            metadata_file += f'Requires-Dist: {dependency}\n'

    if metadata_object.optional_dependencies:
        # Sort extras and dependencies to ensure reproducible builds
        for option, dependencies in sorted(metadata_object.optional_dependencies.items()):
            metadata_file += f'Provides-Extra: {option}\n'
            for dependency in sorted(dependencies):
                if ';' in dependency:
                    metadata_file += f'Requires-Dist: {dependency} and extra == "{option}"\n'
                else:
                    metadata_file += f'Requires-Dist: {dependency}; extra == "{option}"\n'

    ...

    return metadata_file

Copyright

This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.