Python OG

Classes

Classes provide a means of bundling data and functionality together.

  • Python add very little syntactic and semantics feature when comes to Class Mechanism
  • It is a mixture of C++ and Modula-3
  • It fully support everything you know in OOP
  • As Python is a dynamic language, Python Classes can be created at runtime, and can be modified further after creation.
  • Since no universal accepted terminology for classes, terms of C++, Smalltake and Module-3 are used.

9.1. A Word About Names and Objects

Objects have individuality, and multiple names (in multiple scopes) can be bound to the same object. This is known as aliasing in other languages. Sometimes it behave like pointers in some respects.

9.2. Python Scopes and Namespaces

Knowledges of Scopes and Namespaces is very useful for understanding Python Classes.

9.2.1 Namespaces

  • A namespace is a mapping from names to objects.
  • Most namespaces are currently implemented as Python dictionaries.
  • there is absolutely no relation between names in different namespaces
    1
    2
    3
    4
    5
    6
    7
    8
    9
    >>> a = 1
    >>> def tell():
    ...     a = 2
    ...     print(a)
    ... 
    >>> tell()
    2
    >>> a
    1
  • the word attribute is used to refer any name followed by a dot, like z.real, real is an attribute of the object z.
  • From this perspective, in the expression modname.funcname, modname is a module object and funcname is an attribute of it. Then the attributes of the module and the global names defined in the module share the same namespace.
  • Attributes may be read-only or writable. 
1
2
3
modname.the_answer = 42
del modname.the_answer
# these statements are valid

  • Namespaces are created at different moments and have different lifetimes.

    Namespace start end
    containing the built-in names when the interpreter start the interpreter exit
    The global namespace for a module when the module definition is read in(when the module is imported) the interpreter quits, normally
    The local namespace for a function the function is called The function returns or raise unhandled exception

  • The statements executed by the top-level invocation of the interpreter, are considered part of a module called __main__, so they have their own global namespace.

9.2.2 Scope

  • Scope —- a textual region of a Python program where a namespace is directly accessible.

  • directly accessible —- an unqualified reference to a name attempts to find the name in the namespace.

  • Maybe we can say that Namespace is logical while that Scope is physical or literally textual

  • At any time during execution, there are 3 or 4 nested scopes whose namespaces are directly accessible:

    • the innermost scope, which is searched first, contains the local names(Local)
    • the scopes of any enclosing functions, which are searched starting with the nearest enclosing scope, contains non-local, but also non-global names(Enclosing)
    • the next-to-last scope contains the current module’s global names(Global)
    • the outermost scope (searched last) is the namespace containing built-in names(Built-in)

  • The searching process is like this:

    1
    Local -> Enclosing -> Global -> Built-in
    1
    2
    import builtins
    print(dir(builtins)) # print all built-in names
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    # here is global
    a = "global a"

    def outter():
        # local for outter, enclosing for inner
        a = "outter a"

        def inner():
            # local for inner
            a = "inner a"
            print(a)
        
        inner()
        print(a)

    outter()
    print(a)
    '''
    inner a
    outter a
    global a
    '''

  • The global statement can be used to indicate that particular variables live in the global scope and should be rebound there

  • the nonlocal statement indicates that particular variables live in an enclosing scope and should be rebound there.

  • in order to keep things under control, you should try to avoid use these two statements ALAP

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    # here is global
    a = "global a"

    def outter():
        # local for outter, enclosing for inner
        # nonlocal a  # no binding for a because outter do not have an enclosing scope
        a = "outter a"

        def inner():
            # local for inner
            nonlocal a # bind the a of next line to outter's a
            a = "inner a"
            print(a)
        
        inner()
        print(a)

    outter()
    print(a)
    '''
    inner a
    inner a
    global a
    '''
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    # here is global
    a = "global a"

    def outter():
        # local for outter, enclosing for inner
        # nonlocal a  # no binding for a because outter do not have an enclosing scope
        global a # bind the a of next line to global's a
        a = "outter a"

        def inner():
            # local for inner
            # nonlocal a # the outter's a has been rebound to global's a, so there's no outter's a any more
            a = "inner a"
            print(a)
        
        inner()
        print(a)

    outter()
    print(a)
    '''
    inner a
    outter a
    outter a
    '''

  • In fact, all operations that introduce new names use the local scope: in particular, import statements and function definitions bind the module or function name in the local scope.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    # here is global
    def func():
        import string
        print(string.ascii_letters)
    func()
    print(string.ascii_letters)
    '''
    abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ
    Traceback (most recent call last):
      File "/Users/******/test/ptest.py", line 6, in <module>
        print(string.ascii_letters)
    NameError: name 'string' is not defined
    '''

    9.3. A First Look at Classes

a little bit of new syntax, three new object types, and some new semantics.

9.3.1. Class Definition Syntax

  • Classes can be defined in form of

    1
    2
    3
    4
    5
    6
    7
    8
    class ClassName:
      # statements
    or
    class ClassName(BaseClass):
      # statements
    or
    class ClassName():
      # statements
    1
    2
    3
    4
    5
    >>> class myc():
    ...     pass
    ... 
    >>> myc
    <class '__main__.myc'>

  • Class definitions, like function definitions must be executed before they have any effect, so you can even place the definition in a if claus.

  • When a class definition is entered, a new namespace is created, and used as the local scope

9.3.2. Class Objects

Class objects support two kinds of operations: attribute references and instantiation.

  • Attribute references use the standard syntax used for all attribute references in Python: obj.name.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    >>> class A():
    ...     '''class A'''
    ...     a = 0
    ... 
    >>> dir(A)
    ['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'a']
    >>> A.__doc__
    'class A'
    >>> A.a
    0

  • Class instantiation uses function notation. The following statement creates a new instance of the class and assigns this object to the local variable x.

    1
    2
    3
    >>> b = A()
    >>> type(b)
    <class '__main__.A'>

  • The instantiation operation (“calling” a class object) creates an empty object. To initialize it, define __init__() function, which will be automatically invoked for newly-created class instance.

  • initialization function can have arguments. Under this circumstance, arguments given to the class instantiation operator are passed on to __init__()

  • the self argument represent the instantiated class instance, it can be changed to other names, but you’d better not do that. It works like *this in c++

  • just like C++ constructor

    1
    2
    3
    4
    5
    6
    >>> class C():
    ...     def __init__(self):
    ...             print("class C is instantiated")
    ... 
    >>> c = C()
    class C is instantiated

    9.3.3. Instance Objects

The only operations understood by instance objects are attribute references. There are two kinds of valid attribute names: data attributes and methods.

  • data attributes correspond to data members in C++.

  • Data attributes need not be declared; like local variables, they spring into existence when they are first assigned to.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    >>> dir(c)
    ['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__']
    >>> c.newmember = 'hhh'
    >>> dir(c)
    ['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'newmember']
    >>> c.newmember
    'hhh'
    >>> del c.newmember
    >>> dir(c)
    ['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__']

  • A method is a function that “belongs to” an object. 

    1
    2
    instance.f # a method object
    ClassName.f # a function object

    9.3.4. Method Objects

  • the special thing about methods is that the instance object(the self) is passed as the first argument of the function

    the call instance.f() is exactly equivalent to ClassName.f(instance)

  • The implementation: method = ObjPointer + function (ObjPointer just like this in C++)

9.3.5. Class and Instance Variables

Generally speaking, instance variables are for data unique to each instance and class variables are for attributes and methods shared by all instances of the class

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
class Dog:

    kind = 'canine'         # class variable shared by all instances

    def __init__(self, name):
        self.name = name    # instance variable unique to each instance

>>> d = Dog('Fido')
>>> e = Dog('Buddy')
>>> d.kind                  # shared by all dogs
'canine'
>>> e.kind                  # shared by all dogs
'canine'
>>> d.name                  # unique to d
'Fido'
>>> e.name                  # unique to e
'Buddy'

Mutable variable like lists will be shared by all instances !!!

9.4. Random Remarks

  • classes are not usable to implement pure abstract data types. In fact, nothing in Python makes it possible to enforce data hiding, in order to be convenient.(If you really want to completely hide something, add extensions written in C to C Python implementation can achieve this.)

  • There is no shorthand for referencing data attributes (or other methods!) from within methods.(nonlocal won’t work here)

    If you want to access a class variable, use ClassName.attribute or self.attribute in your methods

  • Any function object that is a class attribute defines a method for instances of that class.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    # Function defined outside the class
    def f1(self, x, y):
        return min(x, x+y)

    class C:
        f = f1

        def g(self):
            return 'hello world'
        h = g
    # f , g , h  are all methods of class C
    # this technique is useless unless you want to confuse your code readers

  • Methods may call other methods by using method attributes of the self argument

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    class Bag:
        def __init__(self):
            self.data = []

        def add(self, x):
            self.data.append(x)

        def addtwice(self, x):
            self.add(x)
            self.add(x)

  • Methods may reference global names in the same way as ordinary functions.

  • Each value is an object, and therefore has a class (also called its type). It is stored as object.__class__. This is what type() will give you

    1
    2
    >>> type(a) == a.__class__
    True

    9.5. Inheritance

to derive a class from a base class, the syntax is easy

1
2
class subclass(BaseClassName):
  # statements

  • The name BaseClassName must be defined in a scope containing the derived class definition.

  • This is also very useful : class DerivedClassName(modname.BaseClassName):

  • if a requested attribute is not found in the class, the search proceeds to look in the base class recursively

  • Method references are resolved descending the derived chain, too.

  • Derived classes may override methods of their base classes. In another word, all methods are virtual from c++ programmers’ perspective.(Be careful! This can destroy your code!!!)

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    class base():
        def __init__(self):
            self.a = 0

        def addone(self):
            print('addone of base called')
            self.a += 1

        def addtwo(self):
            self.addone()
            self.addone()


    class derive(base):
        def addone(self):
            print('addone of derive called')
        
        def addthree(self):
            self.addone()
            self.addone()
            self.addone()

    instance = derive()
    print(instance.a)
    instance.addtwo()
    print(instance.a)
    instance.addthree()
    print(instance.a)
    '''
    0
    addone of derive called
    addone of derive called
    0
    addone of derive called
    addone of derive called
    addone of derive called
    0
    '''
    print(help(derive))
    '''
    Help on class derive in module __main__:

    class derive(base)
     |  Method resolution order:
     |      derive
     |      base
     |      builtins.object
     |  
     |  Methods defined here:
     |  
     |  addone(self)
     |  
     |  addthree(self)
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from base:
     |  
     |  __init__(self)
     |      Initialize self.  See help(type(self)) for accurate signature.
     |  
     |  addtwo(self)
     |  
     |  ----------------------------------------------------------------------
    '''

  • There is a simple way to call the base class method directly: just call BaseClassName.methodname(self, arguments), you can use this to extend a method safely. (Note that this only works if the base class is accessible as BaseClassName in the global scope.)

    1
    2
    3
    4
    5
    6
    7
    8
    class derive(base):
        def addone(self):
            print('addone of derive called')
        
        def addthree(self):
            base.addone(self)
            base.addone(self)
            base.addone(self)

  • Python has two built-in functions that work with inheritance:

    • Use isinstance() to check an instance’s type: isinstance(obj, int) will be True only if obj.__class__ is int or some class derived from int.
    • Use issubclass() to check class inheritance: issubclass(bool, int) is True since bool is a subclass of int. However, issubclass(float, int) is False since float is not a subclass of int.

9.5.1. Multiple Inheritance

the syntax of mutiple inheritance is:

1
2
3
4
5
6
class DerivedClassName(Base1, Base2, Base3):
    <statement-1>
    .
    .
    .
    <statement-N>

  • For most purposes, in the simplest cases, you can think of the search for attributes inherited from a parent class as depth-first, left-to-right, not searching twice in the same class where there is an overlap in the hierarchy. 

    1
    2
    3
    4
    5
    6
    |	derived
    |	├── Base1
    |	│   └── base
    |	├── Base2
    |	│   └── base(ignored)
    ↓	└── Base3

  • However, in fact, the method resolution order changes dynamically to support cooperative calls to super().

  • dynamically ordering is used to implement the “Dimond inheritance”

9.6. Private Variables

“Private” instance variables that cannot be accessed except from inside an object Don’t Exist in Python.

  • there is a convention that name prefixed with a _ should be treated as a private one.

  • Python name mangling

    • In name mangling process any identifier with two leading underscore and one trailing underscore is textually replaced with _classname__identifier where classname is the name of the current class.

    • ```python
      class Student: 

      def __init__(self, name): 
    self.__name = name

      s1 = Student(“Santhosh”)
      print(dir(s1))
      ‘’’
      [‘Student__name’, ‘class‘, ‘delattr‘, ‘dict‘, ‘dir‘, ‘doc‘, ‘eq‘, ‘format‘, ‘ge‘, ‘getattribute‘, ‘gt‘, ‘hash‘, ‘init‘, ‘__init_subclass_‘, ‘le‘, ‘lt‘, ‘module‘, ‘ne‘, ‘new‘, ‘reduce‘, ‘__reduce_ex__’, ‘repr‘, ‘setattr‘, ‘sizeof‘, ‘str‘, ‘subclasshook‘, ‘weakref‘]
      ‘’’

      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15

      ## 9.7. Odds and Ends

      > Sometimes it is useful to have a data type similar to the Pascal “record” or C “struct”, bundling together a few named data items. An empty class definition will do nicely:

      ```python
      class Employee:
          pass

      john = Employee()  # Create an empty employee record

      # Fill the fields of the record
      john.name = 'John Doe'
      john.dept = 'computer lab'
      john.salary = 1000

  • A piece of Python code that expects a particular abstract data type can often be passed a class that emulates the methods of that data type instead.

9.8. Iterators

The use of iterators pervades and unifies Python.

1
2
for item in iterator:
  ......

  • What is behind the scenes ?

    • the for statement calls iter() on the container object.
    • The function returns an iterator object that defines the method __next__() which accesses elements in the container one at a time.
    • When there are no more elements, __next__() raises a StopIteration exception which tells the for loop to terminate.
    • You can call the __next__() method using the next() built-in function
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    >>> s = 'abc'
    >>> it = iter(s)
    >>> it
    <iterator object at 0x00A1DB50>
    >>> next(it)
    'a'
    >>> next(it)
    'b'
    >>> next(it)
    'c'
    >>> next(it)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
        next(it)
    StopIteration

  • Then you can define your iterable class

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    class Reverse:
        """Iterator for looping over a sequence backwards."""
        def __init__(self, data):
            self.data = data
            self.index = len(data)

        def __iter__(self):
            return self

        def __next__(self):
            if self.index == 0:
                raise StopIteration
            self.index = self.index - 1
            return self.data[self.index]

  • do iteration manually

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    >>> ls = [1,2,3,4,5]
    >>> it = ls.__iter__()
    >>> it.__next__()
    1
    >>> it.__next__()
    2
    >>> next(it)
    3
    >>> it.__next__()
    4
    >>> it.__next__()
    5
    >>> it.__next__()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    StopIteration

    9.9. Generators

Generators are a simple and powerful tool for creating iterators.

  • They are written like regular functions but use the yield statement whenever they want to return data.

  • Each time next() is called on it, the generator resumes where it left off 

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    def reverse(data):
        for index in range(len(data)-1, -1, -1):
            yield data[index]
    >>>
    >>> for char in reverse('golf'):
    ...     print(char)
    ...
    f
    l
    o
    g

  • for generators, __iter__() and __next__() methods are created automatically.

  • local variables and execution state are automatically saved between calls in generators

  • when generators terminate, they automatically raise StopIteration.

9.10. Generator Expressions

  • a syntax similar to list comprehensions but with parentheses instead of square brackets.

  • more memory friendly than equivalent list comprehensions.

    1
    2
    3
    >>> a = (i for i in range(10))
    >>> a.__class__
    <class 'generator'>

Supplement

watch the videos

  • You can use instance.__dict__to check all the names your instance have.(the namespace of instance)

  • you can use decorator to make a ClassMethod which automatically take the class as its first argument(cls is the same with self in ordinary methods)

    1
    2
    3
    4
    class Class:
      @classmethod
      def namespace(cls):
        print(cls.__dict__)

    This can be used to define alternative constructors!

    1
    2
    3
    4
    @classmethod
    def from_string(cls, s):
      x,y,z = s.split('-')
      return cls(x,y,z)

  • static functions = ordinary functions have some logical connection with the class, decorator @staticmethod is used

    1
    2
    3
    @staticmethod
    def f():
      .....

  • To make things quick clear, you can use help(obj) with any obj which confuses you.

  • in subclass, use super().__init__(arg1,arg2 , ...) to invoke the constructor of parent class

    1
    2
    3
    4
    5
    class Developer(Employee):
      def __init__(self , name , email , dev_lang):
        super().__init__(name , email)
        # Employee.__init__(self , name , email) will do the same
        self.devlang= dev_lang

  • Special methods : repr(instance) will automatically invoke classname.__repr__(), and str(instance) will automatically invoke classname.__str__()

  • you can use special methods(usually in form of __xxx()__) to customize or overload operations

    1
    2
    3
    class Employee:
      def __add__(self , other):
        return self.salary + other.salary

    see more special methods at reference

  • the attribute decorator allow us to define a function works like a property, which can be accessed by instance.property(like a getter in Java), the result is the return value.

    1
    2
    3
    @property
    def attr(self):
      return ........

  • Since we have a getter, we may need a setter also. We can do this by using the decorator @PropertyName.setter

    1
    2
    3
    4
    @attr.setter
    def attr(self , argus):
      self.argus = argus
      ...

  • to do clean up, we can use @PropertyName.deleter decorator

    1
    2
    3
    4
    5
    6
    @attr.deleter
    def attr(self): # no other argus are needed.
      self.argus = None
      ...
      
    del instance.attr  # automatically invoke deleter

  • if you want to learn the decorators @