Permalink
Cannot retrieve contributors at this time
| PEP: 231 | |
| Title: __findattr__() | |
| Version: $Revision$ | |
| Last-Modified: $Date$ | |
| Author: barry@python.org (Barry Warsaw) | |
| Status: Rejected | |
| Type: Standards Track | |
| Content-Type: text/x-rst | |
| Created: 30-Nov-2000 | |
| Python-Version: 2.1 | |
| Post-History: | |
| Introduction | |
| ============ | |
| This PEP describes an extension to instance attribute lookup and | |
| modification machinery, which allows pure-Python implementations | |
| of many interesting programming models. This PEP tracks the | |
| status and ownership of this feature. It contains a description | |
| of the feature and outlines changes necessary to support the | |
| feature. This PEP summarizes discussions held in mailing list | |
| forums, and provides URLs for further information, where | |
| appropriate. The CVS revision history of this file contains the | |
| definitive historical record. | |
| Background | |
| ========== | |
| The semantics for Python instances allow the programmer to | |
| customize some aspects of attribute lookup and attribute | |
| modification, through the special methods ``__getattr__()`` and | |
| ``__setattr__()`` [1]_. | |
| However, because of certain restrictions imposed by these methods, | |
| there are useful programming techniques that can not be written in | |
| Python alone, e.g. strict Java Bean-like [2]_ interfaces and Zope | |
| style acquisitions [3]_. In the latter case, Zope solves this by | |
| including a C extension called ExtensionClass [5]_ which modifies | |
| the standard class semantics, and uses a metaclass hook in | |
| Python's class model called alternatively the "Don Beaudry Hook" | |
| or "Don Beaudry Hack" [6]_. | |
| While Zope's approach works, it has several disadvantages. First, | |
| it requires a C extension. Second it employs a very arcane, but | |
| truck-sized loophole in the Python machinery. Third, it can be | |
| difficult for other programmers to use and understand (the | |
| metaclass has well-known brain exploding properties). And fourth, | |
| because ExtensionClass instances aren't "real" Python instances, | |
| some aspects of the Python runtime system don't work with | |
| ExtensionClass instances. | |
| Proposals for fixing this problem have often been lumped under the | |
| rubric of fixing the "class/type dichotomy"; that is, eliminating | |
| the difference between built-in types and classes [7]_. While a | |
| laudable goal itself, repairing this rift is not necessary in | |
| order to achieve the types of programming constructs described | |
| above. This proposal provides an 80% solution with a minimum of | |
| modification to Python's class and instance objects. It does | |
| nothing to address the type/class dichotomy. | |
| Proposal | |
| ======== | |
| This proposal adds a new special method called ``__findattr__()`` with | |
| the following semantics: | |
| * If defined in a class, it will be called on all instance | |
| attribute resolutions instead of ``__getattr__()`` and | |
| ``__setattr__()``. | |
| * ``__findattr__()`` is never called recursively. That is, when a | |
| specific instance's ``__findattr__()`` is on the call stack, further | |
| attribute accesses for that instance will use the standard | |
| ``__getattr__()`` and ``__setattr__()`` methods. | |
| * ``__findattr__()`` is called for both attribute access ('getting') | |
| and attribute modification ('setting'). It is not called for | |
| attribute deletion. | |
| * When called for getting, it is passed a single argument (not | |
| counting 'self'): the name of the attribute being accessed. | |
| * When called for setting, it is called with third argument, which | |
| is the value to set the attribute to. | |
| * ``__findattr__()`` methods have the same caching semantics as | |
| ``__getattr__()`` and ``__setattr__()``; i.e. if they are present in the | |
| class at class definition time, they are used, but if they are | |
| subsequently added to a class later they are not. | |
| Key Differences with the Existing Protocol | |
| ========================================== | |
| ``__findattr__()``'s semantics are different from the existing | |
| protocol in key ways: | |
| First, ``__getattr__()`` is never called if the attribute is found in | |
| the instance's ``__dict__``. This is done for efficiency reasons, and | |
| because otherwise, ``__setattr__()`` would have no way to get to the | |
| instance's attributes. | |
| Second, ``__setattr__()`` cannot use "normal" syntax for setting | |
| instance attributes, e.g. "self.name = foo" because that would | |
| cause recursive calls to ``__setattr__()``. | |
| ``__findattr__()`` is always called regardless of whether the | |
| attribute is in ``__dict__`` or not, and a flag in the instance object | |
| prevents recursive calls to ``__findattr__()``. This gives the class | |
| a chance to perform some action for every attribute access. And | |
| because it is called for both gets and sets, it is easy to write | |
| similar policy for all attribute access. Further, efficiency is | |
| not a problem because it is only paid when the extended mechanism | |
| is used. | |
| Related Work | |
| ============ | |
| PEP 213 [9]_ describes a different approach to hooking into | |
| attribute access and modification. The semantics proposed in PEP | |
| 213 can be implemented using the ``__findattr__()`` hook described | |
| here, with one caveat. The current reference implementation of | |
| ``__findattr__()`` does not support hooking on attribute deletion. | |
| This could be added if it's found desirable. See example below. | |
| Examples | |
| ======== | |
| One programming style that this proposal allows is a Java | |
| Bean-like interface to objects, where unadorned attribute access | |
| and modification is transparently mapped to a functional | |
| interface. E.g. | |
| :: | |
| class Bean: | |
| def __init__(self, x): | |
| self.__myfoo = x | |
| def __findattr__(self, name, *args): | |
| if name.startswith('_'): | |
| # Private names | |
| if args: setattr(self, name, args[0]) | |
| else: return getattr(self, name) | |
| else: | |
| # Public names | |
| if args: name = '_set_' + name | |
| else: name = '_get_' + name | |
| return getattr(self, name)(*args) | |
| def _set_foo(self, x): | |
| self.__myfoo = x | |
| def _get_foo(self): | |
| return self.__myfoo | |
| b = Bean(3) | |
| print b.foo | |
| b.foo = 9 | |
| print b.foo | |
| A second, more elaborate example is the implementation of both | |
| implicit and explicit acquisition in pure Python:: | |
| import types | |
| class MethodWrapper: | |
| def __init__(self, container, method): | |
| self.__container = container | |
| self.__method = method | |
| def __call__(self, *args, **kws): | |
| return self.__method.im_func(self.__container, *args, **kws) | |
| class WrapperImplicit: | |
| def __init__(self, contained, container): | |
| self.__contained = contained | |
| self.__container = container | |
| def __repr__(self): | |
| return '<Wrapper: [%s | %s]>' % (self.__container, | |
| self.__contained) | |
| def __findattr__(self, name, *args): | |
| # Some things are our own | |
| if name.startswith('_WrapperImplicit__'): | |
| if args: return setattr(self, name, *args) | |
| else: return getattr(self, name) | |
| # setattr stores the name on the contained object directly | |
| if args: | |
| return setattr(self.__contained, name, args[0]) | |
| # Other special names | |
| if name == 'aq_parent': | |
| return self.__container | |
| elif name == 'aq_self': | |
| return self.__contained | |
| elif name == 'aq_base': | |
| base = self.__contained | |
| try: | |
| while 1: | |
| base = base.aq_self | |
| except AttributeError: | |
| return base | |
| # no acquisition for _ names | |
| if name.startswith('_'): | |
| return getattr(self.__contained, name) | |
| # Everything else gets wrapped | |
| missing = [] | |
| which = self.__contained | |
| obj = getattr(which, name, missing) | |
| if obj is missing: | |
| which = self.__container | |
| obj = getattr(which, name, missing) | |
| if obj is missing: | |
| raise AttributeError, name | |
| of = getattr(obj, '__of__', missing) | |
| if of is not missing: | |
| return of(self) | |
| elif type(obj) == types.MethodType: | |
| return MethodWrapper(self, obj) | |
| return obj | |
| class WrapperExplicit: | |
| def __init__(self, contained, container): | |
| self.__contained = contained | |
| self.__container = container | |
| def __repr__(self): | |
| return '<Wrapper: [%s | %s]>' % (self.__container, | |
| self.__contained) | |
| def __findattr__(self, name, *args): | |
| # Some things are our own | |
| if name.startswith('_WrapperExplicit__'): | |
| if args: return setattr(self, name, *args) | |
| else: return getattr(self, name) | |
| # setattr stores the name on the contained object directly | |
| if args: | |
| return setattr(self.__contained, name, args[0]) | |
| # Other special names | |
| if name == 'aq_parent': | |
| return self.__container | |
| elif name == 'aq_self': | |
| return self.__contained | |
| elif name == 'aq_base': | |
| base = self.__contained | |
| try: | |
| while 1: | |
| base = base.aq_self | |
| except AttributeError: | |
| return base | |
| elif name == 'aq_acquire': | |
| return self.aq_acquire | |
| # explicit acquisition only | |
| obj = getattr(self.__contained, name) | |
| if type(obj) == types.MethodType: | |
| return MethodWrapper(self, obj) | |
| return obj | |
| def aq_acquire(self, name): | |
| # Everything else gets wrapped | |
| missing = [] | |
| which = self.__contained | |
| obj = getattr(which, name, missing) | |
| if obj is missing: | |
| which = self.__container | |
| obj = getattr(which, name, missing) | |
| if obj is missing: | |
| raise AttributeError, name | |
| of = getattr(obj, '__of__', missing) | |
| if of is not missing: | |
| return of(self) | |
| elif type(obj) == types.MethodType: | |
| return MethodWrapper(self, obj) | |
| return obj | |
| class Implicit: | |
| def __of__(self, container): | |
| return WrapperImplicit(self, container) | |
| def __findattr__(self, name, *args): | |
| # ignore setattrs | |
| if args: | |
| return setattr(self, name, args[0]) | |
| obj = getattr(self, name) | |
| missing = [] | |
| of = getattr(obj, '__of__', missing) | |
| if of is not missing: | |
| return of(self) | |
| return obj | |
| class Explicit(Implicit): | |
| def __of__(self, container): | |
| return WrapperExplicit(self, container) | |
| # tests | |
| class C(Implicit): | |
| color = 'red' | |
| class A(Implicit): | |
| def report(self): | |
| return self.color | |
| # simple implicit acquisition | |
| c = C() | |
| a = A() | |
| c.a = a | |
| assert c.a.report() == 'red' | |
| d = C() | |
| d.color = 'green' | |
| d.a = a | |
| assert d.a.report() == 'green' | |
| try: | |
| a.report() | |
| except AttributeError: | |
| pass | |
| else: | |
| assert 0, 'AttributeError expected' | |
| # special names | |
| assert c.a.aq_parent is c | |
| assert c.a.aq_self is a | |
| c.a.d = d | |
| assert c.a.d.aq_base is d | |
| assert c.a is not a | |
| # no acquisition on _ names | |
| class E(Implicit): | |
| _color = 'purple' | |
| class F(Implicit): | |
| def report(self): | |
| return self._color | |
| e = E() | |
| f = F() | |
| e.f = f | |
| try: | |
| e.f.report() | |
| except AttributeError: | |
| pass | |
| else: | |
| assert 0, 'AttributeError expected' | |
| # explicit | |
| class G(Explicit): | |
| color = 'pink' | |
| class H(Explicit): | |
| def report(self): | |
| return self.aq_acquire('color') | |
| def barf(self): | |
| return self.color | |
| g = G() | |
| h = H() | |
| g.h = h | |
| assert g.h.report() == 'pink' | |
| i = G() | |
| i.color = 'cyan' | |
| i.h = h | |
| assert i.h.report() == 'cyan' | |
| try: | |
| g.i.barf() | |
| except AttributeError: | |
| pass | |
| else: | |
| assert 0, 'AttributeError expected' | |
| C++-like access control can also be accomplished, although less | |
| cleanly because of the difficulty of figuring out what method is | |
| being called from the runtime call stack:: | |
| import sys | |
| import types | |
| PUBLIC = 0 | |
| PROTECTED = 1 | |
| PRIVATE = 2 | |
| try: | |
| getframe = sys._getframe | |
| except ImportError: | |
| def getframe(n): | |
| try: raise Exception | |
| except Exception: | |
| frame = sys.exc_info()[2].tb_frame | |
| while n > 0: | |
| frame = frame.f_back | |
| if frame is None: | |
| raise ValueError, 'call stack is not deep enough' | |
| return frame | |
| class AccessViolation(Exception): | |
| pass | |
| class Access: | |
| def __findattr__(self, name, *args): | |
| methcache = self.__dict__.setdefault('__cache__', {}) | |
| missing = [] | |
| obj = getattr(self, name, missing) | |
| # if obj is missing we better be doing a setattr for | |
| # the first time | |
| if obj is not missing and type(obj) == types.MethodType: | |
| # Digusting hack because there's no way to | |
| # dynamically figure out what the method being | |
| # called is from the stack frame. | |
| methcache[obj.im_func.func_code] = obj.im_class | |
| # | |
| # What's the access permissions for this name? | |
| access, klass = getattr(self, '__access__', {}).get( | |
| name, (PUBLIC, 0)) | |
| if access is not PUBLIC: | |
| # Now try to see which method is calling us | |
| frame = getframe(0).f_back | |
| if frame is None: | |
| raise AccessViolation | |
| # Get the class of the method that's accessing | |
| # this attribute, by using the code object cache | |
| if frame.f_code.co_name == '__init__': | |
| # There aren't entries in the cache for ctors, | |
| # because the calling mechanism doesn't go | |
| # through __findattr__(). Are there other | |
| # methods that might have the same behavior? | |
| # Since we can't know who's __init__ we're in, | |
| # for now we'll assume that only protected and | |
| # public attrs can be accessed. | |
| if access is PRIVATE: | |
| raise AccessViolation | |
| else: | |
| methclass = self.__cache__.get(frame.f_code) | |
| if not methclass: | |
| raise AccessViolation | |
| if access is PRIVATE and methclass is not klass: | |
| raise AccessViolation | |
| if access is PROTECTED and not issubclass(methclass, | |
| klass): | |
| raise AccessViolation | |
| # If we got here, it must be okay to access the attribute | |
| if args: | |
| return setattr(self, name, *args) | |
| return obj | |
| # tests | |
| class A(Access): | |
| def __init__(self, foo=0, name='A'): | |
| self._foo = foo | |
| # can't set private names in __init__ | |
| self.__initprivate(name) | |
| def __initprivate(self, name): | |
| self._name = name | |
| def getfoo(self): | |
| return self._foo | |
| def setfoo(self, newfoo): | |
| self._foo = newfoo | |
| def getname(self): | |
| return self._name | |
| A.__access__ = {'_foo' : (PROTECTED, A), | |
| '_name' : (PRIVATE, A), | |
| '__dict__' : (PRIVATE, A), | |
| '__access__': (PRIVATE, A), | |
| } | |
| class B(A): | |
| def setfoo(self, newfoo): | |
| self._foo = newfoo + 3 | |
| def setname(self, name): | |
| self._name = name | |
| b = B(1) | |
| b.getfoo() | |
| a = A(1) | |
| assert a.getfoo() == 1 | |
| a.setfoo(2) | |
| assert a.getfoo() == 2 | |
| try: | |
| a._foo | |
| except AccessViolation: | |
| pass | |
| else: | |
| assert 0, 'AccessViolation expected' | |
| try: | |
| a._foo = 3 | |
| except AccessViolation: | |
| pass | |
| else: | |
| assert 0, 'AccessViolation expected' | |
| try: | |
| a.__dict__['_foo'] | |
| except AccessViolation: | |
| pass | |
| else: | |
| assert 0, 'AccessViolation expected' | |
| b = B() | |
| assert b.getfoo() == 0 | |
| b.setfoo(2) | |
| assert b.getfoo() == 5 | |
| try: | |
| b.setname('B') | |
| except AccessViolation: | |
| pass | |
| else: | |
| assert 0, 'AccessViolation expected' | |
| assert b.getname() == 'A' | |
| Here's an implementation of the attribute hook described in PEP | |
| 213 (except that hooking on attribute deletion isn't supported by | |
| the current reference implementation). | |
| :: | |
| class Pep213: | |
| def __findattr__(self, name, *args): | |
| hookname = '__attr_%s__' % name | |
| if args: | |
| op = 'set' | |
| else: | |
| op = 'get' | |
| # XXX: op = 'del' currently not supported | |
| missing = [] | |
| meth = getattr(self, hookname, missing) | |
| if meth is missing: | |
| if op == 'set': | |
| return setattr(self, name, *args) | |
| else: | |
| return getattr(self, name) | |
| else: | |
| return meth(op, *args) | |
| def computation(i): | |
| print 'doing computation:', i | |
| return i + 3 | |
| def rev_computation(i): | |
| print 'doing rev_computation:', i | |
| return i - 3 | |
| class X(Pep213): | |
| def __init__(self, foo=0): | |
| self.__foo = foo | |
| def __attr_foo__(self, op, val=None): | |
| if op == 'get': | |
| return computation(self.__foo) | |
| elif op == 'set': | |
| self.__foo = rev_computation(val) | |
| # XXX: 'del' not yet supported | |
| x = X() | |
| fooval = x.foo | |
| print fooval | |
| x.foo = fooval + 5 | |
| print x.foo | |
| # del x.foo | |
| Reference Implementation | |
| ======================== | |
| The reference implementation, as a patch to the Python core, can be | |
| found at this URL: | |
| http://sourceforge.net/patch/?func=detailpatch&patch_id=102613&group_id=5470 | |
| References | |
| ========== | |
| .. [1] http://docs.python.org/reference/datamodel.html#customizing-attribute-access | |
| .. [2] http://www.javasoft.com/products/javabeans/ | |
| .. [3] http://www.digicool.com/releases/ExtensionClass/Acquisition.html | |
| .. [5] http://www.digicool.com/releases/ExtensionClass | |
| .. [6] http://www.python.org/doc/essays/metaclasses/ | |
| .. [7] http://www.foretec.com/python/workshops/1998-11/dd-ascher-sum.html | |
| .. [8] http://docs.python.org/howto/regex.html | |
| .. [9] PEP 213, Attribute Access Handlers, Prescod | |
| http://www.python.org/dev/peps/pep-0213/ | |
| Rejection | |
| ========= | |
| There are serious problems with the recursion-protection feature. | |
| As described here it's not thread-safe, and a thread-safe solution | |
| has other problems. In general, it's not clear how helpful the | |
| recursion-protection feature is; it makes it hard to write code | |
| that needs to be callable inside ``__findattr__`` as well as outside | |
| it. But without the recursion-protection, it's hard to implement | |
| ``__findattr__`` at all (since ``__findattr__`` would invoke itself | |
| recursively for every attribute it tries to access). There seems | |
| to be no good solution here. | |
| It's also dubious how useful it is to support ``__findattr__`` both | |
| for getting and for setting attributes -- ``__setattr__`` gets called | |
| in all cases already. | |
| The examples can all be implemented using ``__getattr__`` if care is | |
| taken not to store instance variables under their own names. | |
| Copyright | |
| ========= | |
| This document has been placed in the Public Domain. | |
| .. | |
| Local Variables: | |
| mode: indented-text | |
| indent-tabs-mode: nil | |
| End: |