Permalink
Cannot retrieve contributors at this time
| PEP: 204 | |
| Title: Range Literals | |
| Version: $Revision$ | |
| Last-Modified: $Date$ | |
| Author: thomas@python.org (Thomas Wouters) | |
| Status: Rejected | |
| Type: Standards Track | |
| Content-Type: text/x-rst | |
| Created: 14-Jul-2000 | |
| Python-Version: 2.0 | |
| Post-History: | |
| Introduction | |
| ============ | |
| This PEP describes the "range literal" proposal for Python 2.0. | |
| This PEP tracks the status and ownership of this feature, slated | |
| for introduction in Python 2.0. It contains a description of the | |
| feature and outlines changes necessary to support the feature. | |
| This PEP summarizes discussions held in mailing list forums, and | |
| provides URLs for further information, where appropriate. The CVS | |
| revision history of this file contains the definitive historical | |
| record. | |
| List ranges | |
| =========== | |
| Ranges are sequences of numbers of a fixed stepping, often used in | |
| for-loops. The Python for-loop is designed to iterate over a | |
| sequence directly:: | |
| >>> l = ['a', 'b', 'c', 'd'] | |
| >>> for item in l: | |
| ... print item | |
| a | |
| b | |
| c | |
| d | |
| However, this solution is not always prudent. Firstly, problems | |
| arise when altering the sequence in the body of the for-loop, | |
| resulting in the for-loop skipping items. Secondly, it is not | |
| possible to iterate over, say, every second element of the | |
| sequence. And thirdly, it is sometimes necessary to process an | |
| element based on its index, which is not readily available in the | |
| above construct. | |
| For these instances, and others where a range of numbers is | |
| desired, Python provides the ``range`` builtin function, which | |
| creates a list of numbers. The ``range`` function takes three | |
| arguments, *start*, *end* and *step*. *start* and *step* are | |
| optional, and default to 0 and 1, respectively. | |
| The ``range`` function creates a list of numbers, starting at | |
| *start*, with a step of *step*, up to, but not including *end*, so | |
| that ``range(10)`` produces a list that has exactly 10 items, the | |
| numbers 0 through 9. | |
| Using the ``range`` function, the above example would look like | |
| this:: | |
| >>> for i in range(len(l)): | |
| ... print l[i] | |
| a | |
| b | |
| c | |
| d | |
| Or, to start at the second element of ``l`` and processing only | |
| every second element from then on:: | |
| >>> for i in range(1, len(l), 2): | |
| ... print l[i] | |
| b | |
| d | |
| There are several disadvantages with this approach: | |
| - Clarity of purpose: Adding another function call, possibly with | |
| extra arithmetic to determine the desired length and step of the | |
| list, does not improve readability of the code. Also, it is | |
| possible to "shadow" the builtin ``range`` function by supplying a | |
| local or global variable with the same name, effectively | |
| replacing it. This may or may not be a desired effect. | |
| - Efficiency: because the ``range`` function can be overridden, the | |
| Python compiler cannot make assumptions about the for-loop, and | |
| has to maintain a separate loop counter. | |
| - Consistency: There already is a syntax that is used to denote | |
| ranges, as shown below. This syntax uses the exact same | |
| arguments, though all optional, in the exact same way. It seems | |
| logical to extend this syntax to ranges, to form "range | |
| literals". | |
| Slice Indices | |
| ============= | |
| In Python, a sequence can be indexed in one of two ways: | |
| retrieving a single item, or retrieving a range of items. | |
| Retrieving a range of items results in a new object of the same | |
| type as the original sequence, containing zero or more items from | |
| the original sequence. This is done using a "range notation":: | |
| >>> l[2:4] | |
| ['c', 'd'] | |
| This range notation consists of zero, one or two indices separated | |
| by a colon. The first index is the *start* index, the second the | |
| *end*. When either is left out, they default to respectively the | |
| start and the end of the sequence. | |
| There is also an extended range notation, which incorporates | |
| *step* as well. Though this notation is not currently supported | |
| by most builtin types, if it were, it would work as follows:: | |
| >>> l[1:4:2] | |
| ['b', 'd'] | |
| The third "argument" to the slice syntax is exactly the same as | |
| the *step* argument to ``range()``. The underlying mechanisms of the | |
| standard, and these extended slices, are sufficiently different | |
| and inconsistent that many classes and extensions outside of | |
| mathematical packages do not implement support for the extended | |
| variant. While this should be resolved, it is beyond the scope of | |
| this PEP. | |
| Extended slices do show, however, that there is already a | |
| perfectly valid and applicable syntax to denote ranges in a way | |
| that solve all of the earlier stated disadvantages of the use of | |
| the ``range()`` function: | |
| - It is clearer, more concise syntax, which has already proven to | |
| be both intuitive and easy to learn. | |
| - It is consistent with the other use of ranges in Python | |
| (e.g. slices). | |
| - Because it is built-in syntax, instead of a builtin function, it | |
| cannot be overridden. This means both that a viewer can be | |
| certain about what the code does, and that an optimizer will not | |
| have to worry about ``range()`` being "shadowed". | |
| The Proposed Solution | |
| ===================== | |
| The proposed implementation of range-literals combines the syntax | |
| for list literals with the syntax for (extended) slices, to form | |
| range literals:: | |
| >>> [1:10] | |
| [1, 2, 3, 4, 5, 6, 7, 8, 9] | |
| >>> [:5] | |
| [0, 1, 2, 3, 4] | |
| >>> [5:1:-1] | |
| [5, 4, 3, 2] | |
| There is one minor difference between range literals and the slice | |
| syntax: though it is possible to omit all of *start*, *end* and | |
| *step* in slices, it does not make sense to omit *end* in range | |
| literals. In slices, *end* would default to the end of the list, | |
| but this has no meaning in range literals. | |
| Reference Implementation | |
| ======================== | |
| The proposed implementation can be found on SourceForge [1]_. It | |
| adds a new bytecode, ``BUILD_RANGE``, that takes three arguments from | |
| the stack and builds a list on the bases of those. The list is | |
| pushed back on the stack. | |
| The use of a new bytecode is necessary to be able to build ranges | |
| based on other calculations, whose outcome is not known at compile | |
| time. | |
| The code introduces two new functions to ``listobject.c``, which are | |
| currently hovering between private functions and full-fledged API | |
| calls. | |
| ``PyList_FromRange()`` builds a list from start, end and step, | |
| returning NULL if an error occurs. Its prototype is:: | |
| PyObject * PyList_FromRange(long start, long end, long step) | |
| ``PyList_GetLenOfRange()`` is a helper function used to determine the | |
| length of a range. Previously, it was a static function in | |
| ``bltinmodule.c``, but is now necessary in both ``listobject.c`` and | |
| ``bltinmodule.c`` (for ``xrange``). It is made non-static solely to avoid | |
| code duplication. Its prototype is:: | |
| long PyList_GetLenOfRange(long start, long end, long step) | |
| Open issues | |
| =========== | |
| - One possible solution to the discrepancy of requiring the *end* | |
| argument in range literals is to allow the range syntax to | |
| create a "generator", rather than a list, such as the ``xrange`` | |
| builtin function does. However, a generator would not be a | |
| list, and it would be impossible, for instance, to assign to | |
| items in the generator, or append to it. | |
| The range syntax could conceivably be extended to include tuples | |
| (i.e. immutable lists), which could then be safely implemented | |
| as generators. This may be a desirable solution, especially for | |
| large number arrays: generators require very little in the way | |
| of storage and initialization, and there is only a small | |
| performance impact in calculating and creating the appropriate | |
| number on request. (TBD: is there any at all? Cursory testing | |
| suggests equal performance even in the case of ranges of length | |
| 1) | |
| However, even if idea was adopted, would it be wise to "special | |
| case" the second argument, making it optional in one instance of | |
| the syntax, and non-optional in other cases ? | |
| - Should it be possible to mix range syntax with normal list | |
| literals, creating a single list? E.g.:: | |
| >>> [5, 6, 1:6, 7, 9] | |
| to create:: | |
| [5, 6, 1, 2, 3, 4, 5, 7, 9] | |
| - How should range literals interact with another proposed new | |
| feature, "list comprehensions" [2]_? Specifically, should it be | |
| possible to create lists in list comprehensions? E.g.:: | |
| >>> [x:y for x in (1, 2) y in (3, 4)] | |
| Should this example return a single list with multiple ranges:: | |
| [1, 2, 1, 2, 3, 2, 2, 3] | |
| Or a list of lists, like so:: | |
| [[1, 2], [1, 2, 3], [2]_, [2, 3]] | |
| However, as the syntax and semantics of list comprehensions are | |
| still subject of hot debate, these issues are probably best | |
| addressed by the "list comprehensions" PEP. | |
| - Range literals accept objects other than integers: it performs | |
| ``PyInt_AsLong()`` on the objects passed in, so as long as the | |
| objects can be coerced into integers, they will be accepted. | |
| The resulting list, however, is always composed of standard | |
| integers. | |
| Should range literals create a list of the passed-in type? It | |
| might be desirable in the cases of other builtin types, such as | |
| longs and strings:: | |
| >>> [ 1L : 2L<<64 : 2<<32L ] | |
| >>> ["a":"z":"b"] | |
| >>> ["a":"z":2] | |
| However, this might be too much "magic" to be obvious. It might | |
| also present problems with user-defined classes: even if the | |
| base class can be found and a new instance created, the instance | |
| may require additional arguments to ``__init__``, causing the | |
| creation to fail. | |
| - The ``PyList_FromRange()`` and ``PyList_GetLenOfRange()`` functions need | |
| to be classified: are they part of the API, or should they be | |
| made private functions? | |
| Rejection | |
| ========= | |
| After careful consideration, and a period of meditation, this | |
| proposal has been rejected. The open issues, as well as some | |
| confusion between ranges and slice syntax, raised enough questions | |
| for Guido not to accept it for Python 2.0, and later to reject the | |
| proposal altogether. The new syntax and its intentions were deemed | |
| not obvious enough. | |
| [ TBD: Guido, amend/confirm this, please. Preferably both; this | |
| is a PEP, it should contain *all* the reasons for rejection | |
| and/or reconsideration, for future reference. ] | |
| Copyright | |
| ========= | |
| This document has been placed in the Public Domain. | |
| References | |
| ========== | |
| .. [1] http://sourceforge.net/patch/?func=detailpatch&patch_id=100902&group_id=5470 | |
| .. [2] PEP 202, List Comprehensions | |
| .. | |
| Local Variables: | |
| mode: indented-text | |
| indent-tabs-mode: nil | |
| End: |