peps/pep-0367.txt at main · python/peps

Cannot retrieve contributors at this time

594 lines (443 sloc) 19.8 KB

	PEP: 367
	Title: New Super
	Version: $Revision$
	Last-Modified: $Date$
	Author: Calvin Spealman <ironfroggy@gmail.com>,
	Tim Delaney <timothy.c.delaney@gmail.com>
	Status: Superseded
	Type: Standards Track
	Content-Type: text/x-rst
	Created: 28-Apr-2007
	Python-Version: 2.6
	Post-History: `28-Apr-2007 <https://mail.python.org/pipermail/python-dev/2007-April/072807.html>`__,
	`29-Apr-2007 <https://mail.python.org/pipermail/python-dev/2007-April/072835.html>`__,
	`29-Apr-2007 <https://mail.python.org/pipermail/python-dev/2007-April/072858.html>`__,
	`14-May-2007 <https://mail.python.org/pipermail/python-dev/2007-May/073127.html>`__

	Numbering Note
	==============

	This PEP has been renumbered to :pep:`3135`. The text below is the last
	version submitted under the old number.

	Abstract
	========

	This PEP proposes syntactic sugar for use of the ``super`` type to automatically
	construct instances of the super type binding to the class that a method was
	defined in, and the instance (or class object for classmethods) that the method
	is currently acting upon.

	The premise of the new super usage suggested is as follows::

	super.foo(1, 2)

	to replace the old::

	super(Foo, self).foo(1, 2)

	and the current ``__builtin__.super`` be aliased to ``__builtin__.__super__``
	(with ``__builtin__.super`` to be removed in Python 3.0).

	It is further proposed that assignment to ``super`` become a ``SyntaxError``,
	similar to the behaviour of ``None``.


	Rationale
	=========

	The current usage of super requires an explicit passing of both the class and
	instance it must operate from, requiring a breaking of the DRY (Don't Repeat
	Yourself) rule. This hinders any change in class name, and is often considered
	a wart by many.


	Specification
	=============

	Within the specification section, some special terminology will be used to
	distinguish similar and closely related concepts. "super type" will refer to
	the actual builtin type named "super". A "super instance" is simply an instance
	of the super type, which is associated with a class and possibly with an
	instance of that class.

	Because the new ``super`` semantics are not backwards compatible with Python
	2.5, the new semantics will require a ``__future__`` import::

	from __future__ import new_super

	The current ``__builtin__.super`` will be aliased to ``__builtin__.__super__``.
	This will occur regardless of whether the new ``super`` semantics are active.
	It is not possible to simply rename ``__builtin__.super``, as that would affect
	modules that do not use the new ``super`` semantics. In Python 3.0 it is
	proposed that the name ``__builtin__.super`` will be removed.

	Replacing the old usage of super, calls to the next class in the MRO (method
	resolution order) can be made without explicitly creating a ``super``
	instance (although doing so will still be supported via ``__super__``). Every
	function will have an implicit local named ``super``. This name behaves
	identically to a normal local, including use by inner functions via a cell,
	with the following exceptions:

	1. Assigning to the name ``super`` will raise a ``SyntaxError`` at compile time;

	2. Calling a static method or normal function that accesses the name ``super``
	will raise a ``TypeError`` at runtime.

	Every function that uses the name ``super``, or has an inner function that
	uses the name ``super``, will include a preamble that performs the equivalent
	of::

	super = __builtin__.__super__(<class>, <instance>)

	where ``<class>`` is the class that the method was defined in, and
	``<instance>`` is the first parameter of the method (normally ``self`` for
	instance methods, and ``cls`` for class methods). For static methods and normal
	functions, ``<class>`` will be ``None``, resulting in a ``TypeError`` being
	raised during the preamble.

	Note: The relationship between ``super`` and ``__super__`` is similar to that
	between ``import`` and ``__import__``.

	Much of this was discussed in the thread of the python-dev list, "Fixing super
	anyone?" [1]_.


	Open Issues
	-----------


	Determining the class object to use
	'''''''''''''''''''''''''''''''''''

	The exact mechanism for associating the method with the defining class is not
	specified in this PEP, and should be chosen for maximum performance. For
	CPython, it is suggested that the class instance be held in a C-level variable
	on the function object which is bound to one of ``NULL`` (not part of a class),
	``Py_None`` (static method) or a class object (instance or class method).


	Should ``super`` actually become a keyword?
	'''''''''''''''''''''''''''''''''''''''''''

	With this proposal, ``super`` would become a keyword to the same extent that
	``None`` is a keyword. It is possible that further restricting the ``super``
	name may simplify implementation, however some are against the actual keyword-
	ization of super. The simplest solution is often the correct solution and the
	simplest solution may well not be adding additional keywords to the language
	when they are not needed. Still, it may solve other open issues.


	Closed Issues
	-------------

	super used with __call__ attributes
	'''''''''''''''''''''''''''''''''''

	It was considered that it might be a problem that instantiating super instances
	the classic way, because calling it would lookup the __call__ attribute and
	thus try to perform an automatic super lookup to the next class in the MRO.
	However, this was found to be false, because calling an object only looks up
	the __call__ method directly on the object's type. The following example shows
	this in action.

	::

	class A(object):
	def __call__(self):
	return '__call__'
	def __getattribute__(self, attr):
	if attr == '__call__':
	return lambda: '__getattribute__'
	a = A()
	assert a() == '__call__'
	assert a.__call__() == '__getattribute__'

	In any case, with the renaming of ``__builtin__.super`` to
	``__builtin__.__super__`` this issue goes away entirely.


	Reference Implementation
	========================

	It is impossible to implement the above specification entirely in Python. This
	reference implementation has the following differences to the specification:

	1. New ``super`` semantics are implemented using bytecode hacking.

	2. Assignment to ``super`` is not a ``SyntaxError``. Also see point #4.

	3. Classes must either use the metaclass ``autosuper_meta`` or inherit from
	the base class ``autosuper`` to acquire the new ``super`` semantics.

	4. ``super`` is not an implicit local variable. In particular, for inner
	functions to be able to use the super instance, there must be an assignment
	of the form ``super = super`` in the method.

	The reference implementation assumes that it is being run on Python 2.5+.

	::

	#!/usr/bin/env python
	#
	# autosuper.py

	from array import array
	import dis
	import new
	import types
	import __builtin__
	__builtin__.__super__ = __builtin__.super
	del __builtin__.super

	# We need these for modifying bytecode
	from opcode import opmap, HAVE_ARGUMENT, EXTENDED_ARG

	LOAD_GLOBAL = opmap['LOAD_GLOBAL']
	LOAD_NAME = opmap['LOAD_NAME']
	LOAD_CONST = opmap['LOAD_CONST']
	LOAD_FAST = opmap['LOAD_FAST']
	LOAD_ATTR = opmap['LOAD_ATTR']
	STORE_FAST = opmap['STORE_FAST']
	LOAD_DEREF = opmap['LOAD_DEREF']
	STORE_DEREF = opmap['STORE_DEREF']
	CALL_FUNCTION = opmap['CALL_FUNCTION']
	STORE_GLOBAL = opmap['STORE_GLOBAL']
	DUP_TOP = opmap['DUP_TOP']
	POP_TOP = opmap['POP_TOP']
	NOP = opmap['NOP']
	JUMP_FORWARD = opmap['JUMP_FORWARD']
	ABSOLUTE_TARGET = dis.hasjabs

	def _oparg(code, opcode_pos):
	return code[opcode_pos+1] + (code[opcode_pos+2] << 8)

	def _bind_autosuper(func, cls):
	co = func.func_code
	name = func.func_name
	newcode = array('B', co.co_code)
	codelen = len(newcode)
	newconsts = list(co.co_consts)
	newvarnames = list(co.co_varnames)

	# Check if the global 'super' keyword is already present
	try:
	sn_pos = list(co.co_names).index('super')
	except ValueError:
	sn_pos = None

	# Check if the varname 'super' keyword is already present
	try:
	sv_pos = newvarnames.index('super')
	except ValueError:
	sv_pos = None

	# Check if the cellvar 'super' keyword is already present
	try:
	sc_pos = list(co.co_cellvars).index('super')
	except ValueError:
	sc_pos = None

	# If 'super' isn't used anywhere in the function, we don't have anything to do
	if sn_pos is None and sv_pos is None and sc_pos is None:
	return func

	c_pos = None
	s_pos = None
	n_pos = None

	# Check if the 'cls_name' and 'super' objects are already in the constants
	for pos, o in enumerate(newconsts):
	if o is cls:
	c_pos = pos

	if o is __super__:
	s_pos = pos

	if o == name:
	n_pos = pos

	# Add in any missing objects to constants and varnames
	if c_pos is None:
	c_pos = len(newconsts)
	newconsts.append(cls)

	if n_pos is None:
	n_pos = len(newconsts)
	newconsts.append(name)

	if s_pos is None:
	s_pos = len(newconsts)
	newconsts.append(__super__)

	if sv_pos is None:
	sv_pos = len(newvarnames)
	newvarnames.append('super')

	# This goes at the start of the function. It is:
	#
	# super = __super__(cls, self)
	#
	# If 'super' is a cell variable, we store to both the
	# local and cell variables (i.e. STORE_FAST and STORE_DEREF).
	#
	preamble = [
	LOAD_CONST, s_pos & 0xFF, s_pos >> 8,
	LOAD_CONST, c_pos & 0xFF, c_pos >> 8,
	LOAD_FAST, 0, 0,
	CALL_FUNCTION, 2, 0,
	]

	if sc_pos is None:
	# 'super' is not a cell variable - we can just use the local variable
	preamble += [
	STORE_FAST, sv_pos & 0xFF, sv_pos >> 8,
	]
	else:
	# If 'super' is a cell variable, we need to handle LOAD_DEREF.
	preamble += [
	DUP_TOP,
	STORE_FAST, sv_pos & 0xFF, sv_pos >> 8,
	STORE_DEREF, sc_pos & 0xFF, sc_pos >> 8,
	]

	preamble = array('B', preamble)

	# Bytecode for loading the local 'super' variable.
	load_super = array('B', [
	LOAD_FAST, sv_pos & 0xFF, sv_pos >> 8,
	])

	preamble_len = len(preamble)
	need_preamble = False
	i = 0

	while i < codelen:
	opcode = newcode[i]
	need_load = False
	remove_store = False

	if opcode == EXTENDED_ARG:
	raise TypeError("Cannot use 'super' in function with EXTENDED_ARG opcode")

	# If the opcode is an absolute target it needs to be adjusted
	# to take into account the preamble.
	elif opcode in ABSOLUTE_TARGET:
	oparg = _oparg(newcode, i) + preamble_len
	newcode[i+1] = oparg & 0xFF
	newcode[i+2] = oparg >> 8

	# If LOAD_GLOBAL(super) or LOAD_NAME(super) then we want to change it into
	# LOAD_FAST(super)
	elif (opcode == LOAD_GLOBAL or opcode == LOAD_NAME) and _oparg(newcode, i) == sn_pos:
	need_preamble = need_load = True

	# If LOAD_FAST(super) then we just need to add the preamble
	elif opcode == LOAD_FAST and _oparg(newcode, i) == sv_pos:
	need_preamble = need_load = True

	# If LOAD_DEREF(super) then we change it into LOAD_FAST(super) because
	# it's slightly faster.
	elif opcode == LOAD_DEREF and _oparg(newcode, i) == sc_pos:
	need_preamble = need_load = True

	if need_load:
	newcode[i:i+3] = load_super

	i += 1

	if opcode >= HAVE_ARGUMENT:
	i += 2

	# No changes needed - get out.
	if not need_preamble:
	return func

	# Our preamble will have 3 things on the stack
	co_stacksize = max(3, co.co_stacksize)

	# Conceptually, our preamble is on the `def` line.
	co_lnotab = array('B', co.co_lnotab)

	if co_lnotab:
	co_lnotab[0] += preamble_len

	co_lnotab = co_lnotab.tostring()

	# Our code consists of the preamble and the modified code.
	codestr = (preamble + newcode).tostring()

	codeobj = new.code(co.co_argcount, len(newvarnames), co_stacksize,
	co.co_flags, codestr, tuple(newconsts), co.co_names,
	tuple(newvarnames), co.co_filename, co.co_name,
	co.co_firstlineno, co_lnotab, co.co_freevars,
	co.co_cellvars)

	func.func_code = codeobj
	func.func_class = cls
	return func

	class autosuper_meta(type):
	def __init__(cls, name, bases, clsdict):
	UnboundMethodType = types.UnboundMethodType

	for v in vars(cls):
	o = getattr(cls, v)
	if isinstance(o, UnboundMethodType):
	_bind_autosuper(o.im_func, cls)

	class autosuper(object):
	__metaclass__ = autosuper_meta

	if __name__ == '__main__':
	class A(autosuper):
	def f(self):
	return 'A'

	class B(A):
	def f(self):
	return 'B' + super.f()

	class C(A):
	def f(self):
	def inner():
	return 'C' + super.f()

	# Needed to put 'super' into a cell
	super = super
	return inner()

	class D(B, C):
	def f(self, arg=None):
	var = None
	return 'D' + super.f()

	assert D().f() == 'DBCA'

	Disassembly of B.f and C.f reveals the different preambles used when ``super``
	is simply a local variable compared to when it is used by an inner function.

	::

	>>> dis.dis(B.f)

	214 0 LOAD_CONST 4 (<type 'super'>)
	3 LOAD_CONST 2 (<class '__main__.B'>)
	6 LOAD_FAST 0 (self)
	9 CALL_FUNCTION 2
	12 STORE_FAST 1 (super)

	215 15 LOAD_CONST 1 ('B')
	18 LOAD_FAST 1 (super)
	21 LOAD_ATTR 1 (f)
	24 CALL_FUNCTION 0
	27 BINARY_ADD
	28 RETURN_VALUE

	::

	>>> dis.dis(C.f)

	218 0 LOAD_CONST 4 (<type 'super'>)
	3 LOAD_CONST 2 (<class '__main__.C'>)
	6 LOAD_FAST 0 (self)
	9 CALL_FUNCTION 2
	12 DUP_TOP
	13 STORE_FAST 1 (super)
	16 STORE_DEREF 0 (super)

	219 19 LOAD_CLOSURE 0 (super)
	22 LOAD_CONST 1 (<code object inner at 00C160A0, file "autosuper.py", line 219>)
	25 MAKE_CLOSURE 0
	28 STORE_FAST 2 (inner)

	223 31 LOAD_FAST 1 (super)
	34 STORE_DEREF 0 (super)

	224 37 LOAD_FAST 2 (inner)
	40 CALL_FUNCTION 0
	43 RETURN_VALUE

	Note that in the final implementation, the preamble would not be part of the
	bytecode of the method, but would occur immediately following unpacking of
	parameters.


	Alternative Proposals
	=====================

	No Changes
	----------

	Although its always attractive to just keep things how they are, people have
	sought a change in the usage of super calling for some time, and for good
	reason, all mentioned previously.

	- Decoupling from the class name (which might not even be bound to the
	right class anymore!)
	- Simpler looking, cleaner super calls would be better

	Dynamic attribute on super type
	-------------------------------

	The proposal adds a dynamic attribute lookup to the super type, which will
	automatically determine the proper class and instance parameters. Each super
	attribute lookup identifies these parameters and performs the super lookup on
	the instance, as the current super implementation does with the explicit
	invocation of a super instance upon a class and instance.

	This proposal relies on sys._getframe(), which is not appropriate for anything
	except a prototype implementation.


	super(__this_class__, self)
	---------------------------

	This is nearly an anti-proposal, as it basically relies on the acceptance of
	the __this_class__ PEP, which proposes a special name that would always be
	bound to the class within which it is used. If that is accepted, __this_class__
	could simply be used instead of the class' name explicitly, solving the name
	binding issues [2]_.

	self.__super__.foo(\*args)
	--------------------------

	The __super__ attribute is mentioned in this PEP in several places, and could
	be a candidate for the complete solution, actually using it explicitly instead
	of any super usage directly. However, double-underscore names are usually an
	internal detail, and attempted to be kept out of everyday code.

	super(self, \args) or __super__(self, \args)
	----------------------------------------------

	This solution only solves the problem of the type indication, does not handle
	differently named super methods, and is explicit about the name of the
	instance. It is less flexible without being able to enacted on other method
	names, in cases where that is needed. One use case this fails is where a base-
	class has a factory classmethod and a subclass has two factory classmethods,
	both of which needing to properly make super calls to the one in the base-
	class.

	super.foo(self, \*args)
	-----------------------

	This variation actually eliminates the problems with locating the proper
	instance, and if any of the alternatives were pushed into the spotlight, I
	would want it to be this one.

	super or super()
	----------------

	This proposal leaves no room for different names, signatures, or application
	to other classes, or instances. A way to allow some similar use alongside the
	normal proposal would be favorable, encouraging good design of multiple
	inheritance trees and compatible methods.

	super(\p, \\*kw)
	------------------

	There has been the proposal that directly calling ``super(p, *kw)`` would
	be equivalent to calling the method on the ``super`` object with the same name
	as the method currently being executed i.e. the following two methods would be
	equivalent:

	::

	def f(self, p, *kw):
	super.f(p, *kw)

	::

	def f(self, p, *kw):
	super(p, *kw)

	There is strong sentiment for and against this, but implementation and style
	concerns are obvious. Guido has suggested that this should be excluded from
	this PEP on the principle of KISS (Keep It Simple Stupid).



	History
	=======
	29-Apr-2007 - Changed title from "Super As A Keyword" to "New Super"
	- Updated much of the language and added a terminology section
	for clarification in confusing places.
	- Added reference implementation and history sections.

	06-May-2007 - Updated by Tim Delaney to reflect discussions on the python-3000
	and python-dev mailing lists.

	References
	==========

	.. [1] Fixing super anyone?
	(https://mail.python.org/pipermail/python-3000/2007-April/006667.html)

	.. [2] PEP 3130: Access to Module/Class/Function Currently Being Defined (this)
	(https://mail.python.org/pipermail/python-ideas/2007-April/000542.html)


	Copyright
	=========

	This document has been placed in the public domain.



	..
	Local Variables:
	mode: indented-text
	indent-tabs-mode: nil
	sentence-end-double-space: t
	fill-column: 70
	coding: utf-8
	End: