Permalink
Cannot retrieve contributors at this time
| PEP: 389 | |
| Title: argparse - New Command Line Parsing Module | |
| Version: $Revision$ | |
| Last-Modified: $Date$ | |
| Author: Steven Bethard <steven.bethard@gmail.com> | |
| Status: Final | |
| Type: Standards Track | |
| Content-Type: text/x-rst | |
| Created: 25-Sep-2009 | |
| Python-Version: 2.7, 3.2 | |
| Post-History: 27-Sep-2009, 24-Oct-2009 | |
| Acceptance | |
| ========== | |
| This PEP was approved by Guido on python-dev on February 21, 2010 [17]_. | |
| Abstract | |
| ======== | |
| This PEP proposes inclusion of the argparse [1]_ module in the Python | |
| standard library in Python 2.7 and 3.2. | |
| Motivation | |
| ========== | |
| The argparse module is a command line parsing library which provides | |
| more functionality than the existing command line parsing modules in | |
| the standard library, getopt [2]_ and optparse [3]_. It includes | |
| support for positional arguments (not just options), subcommands, | |
| required options, options syntaxes like "/f" and "+rgb", zero-or-more | |
| and one-or-more style arguments, and many other features the other | |
| two lack. | |
| The argparse module is also already a popular third-party replacement | |
| for these modules. It is used in projects like IPython (the Scipy | |
| Python shell) [4]_, is included in Debian testing and unstable [5]_, | |
| and since 2007 has had various requests for its inclusion in the | |
| standard library [6]_ [7]_ [8]_. This popularity suggests it may be | |
| a valuable addition to the Python libraries. | |
| Why aren't getopt and optparse enough? | |
| ====================================== | |
| One argument against adding argparse is that thare are "already two | |
| different option parsing modules in the standard library" [9]_. The | |
| following is a list of features provided by argparse but not present | |
| in getopt or optparse: | |
| * While it is true there are two *option* parsing libraries, there | |
| are no full command line parsing libraries -- both getopt and | |
| optparse support only options and have no support for positional | |
| arguments. The argparse module handles both, and as a result, is | |
| able to generate better help messages, avoiding redundancies like | |
| the ``usage=`` string usually required by optparse. | |
| * The argparse module values practicality over purity. Thus, argparse | |
| allows required options and customization of which characters are | |
| used to identify options, while optparse explicitly states "the | |
| phrase 'required option' is self-contradictory" and that the option | |
| syntaxes ``-pf``, ``-file``, ``+f``, ``+rgb``, ``/f`` and ``/file`` | |
| "are not supported by optparse, and they never will be". | |
| * The argparse module allows options to accept a variable number of | |
| arguments using ``nargs='?'``, ``nargs='*'`` or ``nargs='+'``. The | |
| optparse module provides an untested recipe for some part of this | |
| functionality [10]_ but admits that "things get hairy when you want | |
| an option to take a variable number of arguments." | |
| * The argparse module supports subcommands, where a main command | |
| line parser dispatches to other command line parsers depending on | |
| the command line arguments. This is a common pattern in command | |
| line interfaces, e.g. ``svn co`` and ``svn up``. | |
| Why isn't the functionality just being added to optparse? | |
| ========================================================= | |
| Clearly all the above features offer improvements over what is | |
| available through optparse. A reasonable question then is why these | |
| features are not simply provided as patches to optparse, instead of | |
| introducing an entirely new module. In fact, the original development | |
| of argparse intended to do just that, but because of various fairly | |
| constraining design decisions of optparse, this wasn't really | |
| possible. Some of the problems included: | |
| * The optparse module exposes the internals of its parsing algorithm. | |
| In particular, ``parser.largs`` and ``parser.rargs`` are guaranteed | |
| to be available to callbacks [11]_. This makes it extremely | |
| difficult to improve the parsing algorithm as was necessary in | |
| argparse for proper handling of positional arguments and variable | |
| length arguments. For example, ``nargs='+'`` in argparse is matched | |
| using regular expressions and thus has no notion of things like | |
| ``parser.largs``. | |
| * The optparse extension APIs are extremely complex. For example, | |
| just to use a simple custom string-to-object conversion function, | |
| you have to subclass ``Option``, hack class attributes, and then | |
| specify your custom option type to the parser, like this:: | |
| class MyOption(Option): | |
| TYPES = Option.TYPES + ("mytype",) | |
| TYPE_CHECKER = copy(Option.TYPE_CHECKER) | |
| TYPE_CHECKER["mytype"] = check_mytype | |
| parser = optparse.OptionParser(option_class=MyOption) | |
| parser.add_option("-m", type="mytype") | |
| For comparison, argparse simply allows conversion functions to be | |
| used as ``type=`` arguments directly, e.g.:: | |
| parser = argparse.ArgumentParser() | |
| parser.add_option("-m", type=check_mytype) | |
| But given the baroque customization APIs of optparse, it is unclear | |
| how such a feature should interact with those APIs, and it is | |
| quite possible that introducing the simple argparse API would break | |
| existing custom Option code. | |
| * Both optparse and argparse parse command line arguments and assign | |
| them as attributes to an object returned by ``parse_args``. | |
| However, the optparse module guarantees that the ``take_action`` | |
| method of custom actions will always be passed a ``values`` object | |
| which provides an ``ensure_value`` method [12]_, while the argparse | |
| module allows attributes to be assigned to any object, e.g.:: | |
| foo_object = ... | |
| parser.parse_args(namespace=foo_object) | |
| foo_object.some_attribute_parsed_from_command_line | |
| Modifying optparse to allow any object to be passed in would be | |
| difficult because simply passing the ``foo_object`` around instead | |
| of a ``Values`` instance will break existing custom actions that | |
| depend on the ``ensure_value`` method. | |
| Because of issues like these, which made it unreasonably difficult | |
| for argparse to stay compatible with the optparse APIs, argparse was | |
| developed as an independent module. Given these issues, merging all | |
| the argparse features into optparse with no backwards | |
| incompatibilities seems unlikely. | |
| Deprecation of optparse | |
| ======================= | |
| Because all of optparse's features are available in argparse, the | |
| optparse module will be deprecated. However, because of the | |
| widespread use of optparse, the deprecation strategy contains only | |
| documentation changes and warnings that will not be visible by | |
| default: | |
| * Python 2.7+ and 3.2+ -- The following note will be added to the | |
| optparse documentation: | |
| The optparse module is deprecated and will not be developed | |
| further; development will continue with the argparse module. | |
| * Python 2.7+ -- If the Python 3 compatibility flag, ``-3``, is | |
| provided at the command line, then importing optparse will issue a | |
| DeprecationWarning. Otherwise no warnings will be issued. | |
| * Python 3.2+ -- Importing optparse will issue a | |
| PendingDeprecationWarning, which is not displayed by default. | |
| Note that no removal date is proposed for optparse. | |
| Updates to getopt documentation | |
| =============================== | |
| The getopt module will not be deprecated. However, its documentation | |
| will be updated to point to argparse in a couple of places. At the | |
| top of the module, the following note will be added: | |
| The getopt module is a parser for command line options whose API | |
| is designed to be familiar to users of the C getopt function. | |
| Users who are unfamiliar with the C getopt function or who would | |
| like to write less code and get better help and error messages | |
| should consider using the argparse module instead. | |
| Additionally, after the final getopt example, the following note will | |
| be added: | |
| Note that an equivalent command line interface could be produced | |
| with less code by using the argparse module:: | |
| import argparse | |
| if __name__ == '__main__': | |
| parser = argparse.ArgumentParser() | |
| parser.add_argument('-o', '--output') | |
| parser.add_argument('-v', dest='verbose', action='store_true') | |
| args = parser.parse_args() | |
| # ... do something with args.output ... | |
| # ... do something with args.verbose .. | |
| Deferred: string formatting | |
| =========================== | |
| The argparse module supports Python from 2.3 up through 3.2 and as a | |
| result relies on traditional ``%(foo)s`` style string formatting. It | |
| has been suggested that it might be better to use the new style | |
| ``{foo}`` string formatting [13]_. There was some discussion about | |
| how best to do this for modules in the standard library [14]_ and | |
| several people are developing functions for automatically converting | |
| %-formatting to {}-formatting [15]_ [16]_. When one of these is added | |
| to the standard library, argparse will use them to support both | |
| formatting styles. | |
| Rejected: getopt compatibility methods | |
| ====================================== | |
| Previously, when this PEP was suggesting the deprecation of getopt | |
| as well as optparse, there was some talk of adding a method like:: | |
| ArgumentParser.add_getopt_arguments(options[, long_options]) | |
| However, this method will not be added for a number of reasons: | |
| * The getopt module is not being deprecated, so there is less need. | |
| * This method would not actually ease the transition for any getopt | |
| users who were already maintaining usage messages, because the API | |
| above gives no way of adding help messages to the arguments. | |
| * Some users of getopt consider it very important that only a single | |
| function call is necessary. The API above does not satisfy this | |
| requirement because both ``ArgumentParser()`` and ``parse_args()`` | |
| must also be called. | |
| Out of Scope: Various Feature Requests | |
| ====================================== | |
| Several feature requests for argparse were made in the discussion of | |
| this PEP: | |
| * Support argument defaults from environment variables | |
| * Support argument defaults from configuration files | |
| * Support "foo --help subcommand" in addition to the currently | |
| supported "foo subcommand --help" | |
| These are all reasonable feature requests for the argparse module, | |
| but are out of the scope of this PEP, and have been redirected to | |
| the argparse issue tracker. | |
| Discussion: sys.stderr and sys.exit | |
| =================================== | |
| There were some concerns that argparse by default always writes to | |
| ``sys.stderr`` and always calls ``sys.exit`` when invalid arguments | |
| are provided. This is the desired behavior for the vast majority of | |
| argparse use cases which revolve around simple command line | |
| interfaces. However, in some cases, it may be desirable to keep | |
| argparse from exiting, or to have it write its messages to something | |
| other than ``sys.stderr``. These use cases can be supported by | |
| subclassing ``ArgumentParser`` and overriding the ``exit`` or | |
| ``_print_message`` methods. The latter is an undocumented | |
| implementation detail, but could be officially exposed if this turns | |
| out to be a common need. | |
| References | |
| ========== | |
| .. [1] argparse | |
| (http://code.google.com/p/argparse/) | |
| .. [2] getopt | |
| (http://docs.python.org/library/getopt.html) | |
| .. [3] optparse | |
| (http://docs.python.org/library/optparse.html) | |
| .. [4] argparse in IPython | |
| (http://mail.scipy.org/pipermail/ipython-dev/2009-April/005102.html) | |
| .. [5] argparse in Debian | |
| (http://packages.debian.org/search?keywords=argparse) | |
| .. [6] 2007-01-03 request for argparse in the standard library | |
| (https://mail.python.org/pipermail/python-list/2007-January/472276.html) | |
| .. [7] 2009-06-09 request for argparse in the standard library | |
| (http://bugs.python.org/issue6247) | |
| .. [8] 2009-09-10 request for argparse in the standard library | |
| (https://mail.python.org/pipermail/stdlib-sig/2009-September/000342.html) | |
| .. [9] Fredrik Lundh response to [6]_ | |
| (https://mail.python.org/pipermail/python-list/2007-January/1086892.html) | |
| .. [10] optparse variable args | |
| (http://docs.python.org/library/optparse.html#callback-example-6-variable-arguments) | |
| .. [11] parser.largs and parser.rargs | |
| (http://docs.python.org/library/optparse.html#how-callbacks-are-called) | |
| .. [12] take_action values argument | |
| (http://docs.python.org/library/optparse.html#adding-new-actions) | |
| .. [13] use {}-formatting instead of %-formatting | |
| (http://bugs.python.org/msg89279) | |
| .. [14] transitioning from % to {} formatting | |
| (https://mail.python.org/pipermail/python-dev/2009-September/092326.html) | |
| .. [15] Vinay Sajip's %-to-{} converter | |
| (http://gist.github.com/200936) | |
| .. [16] Benjamin Peterson's %-to-{} converter | |
| (http://bazaar.launchpad.net/~gutworth/+junk/mod2format/files) | |
| .. [17] Guido's approval | |
| (https://mail.python.org/pipermail/python-dev/2010-February/097839.html) | |
| Copyright | |
| ========= | |
| This document has been placed in the public domain. | |
| .. | |
| Local Variables: | |
| mode: indented-text | |
| indent-tabs-mode: nil | |
| sentence-end-double-space: t | |
| fill-column: 70 | |
| coding: utf-8 | |
| End: |