python-function

Posted on Edited on In ,

Advanced function feature

In python, function is an object that means you can assign it to a variable or pass it as a parameter! we can also return a function inside another function, cool!

Assign functions to a variable

1
2
3
4
5
def greet(name):
    return "hello " + name

greet_someone = greet
greet_someone("John")

Function can return other function

1
2
3
4
5
6
7
8
def compose_greet_func(name):
    def get_message():
        return "Hello " + name

    return get_message

greet = compose_greet_func("jason")
greet()

if nothing returns in function, None returned by default

descriptor(class method)

In general, a descriptor is an object attribute with "binding behavior" whose access has been overridden by method in the descriptor protocol. Those methods are __get__(), __set__(), and __delete__(). If any of those methods is defined for an object, it is said to be a descriptor

Calling property() is a succinct way of building a data descriptor that triggers function call upon access to an attribute.

define a property
property(fget=None, fset=None, fdel=None, doc=None) -> property attribute we can also use decorator @property and @xx.setter to define a property, must define @property first!!!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
class Person:

    #descriptor
    def __init__(self, name):
        self._name = name
        
    def get_name(self):
        print("get_name() is called")
        return self._name

    def set_name(self, name):
        print('set_name() is called with name:', name)
        self._name = name
        
    # declare name as a property, obj.name
    # but inside class, it's function.
    # @property is getter which must define first!!!   
    @property
    def name(self):
        print('name getter is called')
        return self._name
    
    @name.setter
    # name must be same between getter and setter.
    def name(self, name):
        print('name setter is called')
        self._name = name

    # bname: a property of person(fget() fset(), property can be different names)
    # but if you use decorator to define a propterty, in that case fget, fset, property must be same name!!!
    bname = property(get_name, set_name) # declare a property 
    
p1 = Person("jason")
print(p1.bname)
p1.bname = "kk"

print(p1.name)
get_name() is called
jason
set_name() is called with name: kk
name getter is called
kk

function closure

if a function defines another function in its scope and returns it, it’s a closure

1
2
3
4
5
6
7
def welcome(name):
    def wrapper():
        print("wrapper is called with: ", name)
    return wrapper # here we return a function

wf = welcome("jason")
wf() 
wrapper is called with:  jason

decorator

Python has built-in decorator like property, to apply a decorator to a function by placing it just before the function definition. like 

1
2
3
4
5
6
7
8
@xxx
def test():
    pass

# with parameter(s)
@xxx(arg)
def test()
    pass

Decorator is needed when you want do something before a function run, place it anywhere that needs it.

Decorator can be a class or function, if it is class, must be callable, has __call__ function defined.

below are the ways to create a custom decorator

function as a decorator.

decorator is applied when you define that function, later on when you call it, the wrapped function is called actually.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
def log(text):
    print('we got parameter for decorator: ', text)
    def myDecorator(f): # passed in user function.
        def wrapper(*args, **argv): # parameters for user functioin
            # *args for dynamic position parameters
            # args is a list ['a', 'b']
            f(*args, **argv) # here *args, separate the list == f('a', 'b')
        return wrapper
    # return a function object, when it's called, call wrapped f() actually!
    return myDecorator

@log("param1") # parameter is used by decorator, not wrapper function!!!
def aFunction(name):
    print ("inside aFunction ", name)
aFunction("jason")
we got parameter for decorator:  param1
inside aFunction  jason

class as a decorator

decorator is applied when you define that function, later on when you call it, the wrapped function is called actually.

**Always use call as decorator, as it’s easy to understand.

class decorator without parameter

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
class cDecorator:
    'doc of class decorator'
    def __init__(self, f): # fixed format
        # must passed function
        print('decorator is initialized')
        self.f = f
    def __call__(self, *argc, **argv): # argc is a list, argv is a dict, fixed format
        # do extra here
        print('decorator is called')
        self.f(*argc, **argv) # get args from wrap function, then pass it orginal.

        
# here is NOT @cDecorator()
# this happend when function is declared!!!
# wrapped = cDecorator(bFunction)--->__init__()
@cDecorator
def bFunction(msg, name="josh"):
    print ("inside bFunction: {0} {1}".format(msg, name))

# repalced with wrapped function
# wrapped("hi", name="josh")--->__call__()
bFunction('hi')
print(bFunction.__doc__)
decorator is initialized
decorator is called
inside bFunction: hi josh
doc of class decorator

class decorator with parameter

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
class cDecorator:
    "doc of class decorator"

    def __init__(self, msg):  # fixed format
        # must passed function
        print("decorator is initialized", msg)
        self.msg = msg

    def __call__(self, fn):  # argc is a list, argv is a dict, fixed format
        print("wrapper parameter:", self.msg)

        def wrapper(*args, **argv):
            print("call original function")
            fn(*args, **argv)

        return wrapper


# wrapped = cDecorator("cool")(bFunction) -->__init__(), then __call__()
@cDecorator("cool")
def bFunction(msg, name="josh"):
    print("inside bFunction: {0} {1}".format(msg, name))

# wrapped("hi")
bFunction("hi")
print(bFunction.__doc__) # see below function wraps 
decorator is initialized cool
wrapper parameter: cool
call original function
inside bFunction: hi josh
None

why use functools.wraps

when a function is decorated, getting the docstring and function signature return “wrapper”, the inner function, not the original function, this is a problem, we can use “functool.wraps” decorator to decorate our inner function, By applying this “wraps” decorator to our inner function, we copy over func name, docstring, and signature to our inner function, avoiding the issues.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
from functools import wraps

def log(text):
    def decorator(func):
        
        @wraps(func) # assign func.__name__ to inner function
        def wrapper(*args, **kw):
            print('%s call %s()' % (text, func.__name__))
            return func(*args, **kw)
        
        return wrapper
    
    return decorator

@log('execute')
def f(msg, name='josh'):
    print('{0} {1}'.format(msg, name))
f('hi')
print(f.__name__) # __name__ is not wrapper, but f great!
execute call f()
hi josh
f

Generator vs Iterator

generator function returns a generator object, generator function is function that yield(like return) value when next() call,it will execute until last yield point, most of time next() is not called explictly but implicitly like this for elm in list.

Iterator

An iterator is an object that implements the iterator protocol (don’t panic!). An iterator protocol is nothing but a specific class with __next__() in Python which further has the __next__() method.

While for iterable object, No need to __next__() but mostly have __iter__() and __getitem__(), __iter__() allows to convert the object into an iterator, __getitem__() will be called in iterator’s __next__() function.

iter(object) will call object.__iter__() while next(object) will call object.__next__()

Generator

  • Iterator is an object has __next__() method
  • generator is an object which implements __next__(), so that it’s an iterator!
  • generator function is a function that uses yield as return, when you call this function, it returns a generator object!

more details about these two, refer to iterator-generator-descriptor.

vs

Most uses case, you create a generator function which returns a generator object and for elm in generator_object

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
def f():
    yield 'start'
    yield 'middle'
    yield 'end'

gen = f()
# return generator object, then call next()  to 
# yield object(which has __next__() method provided by yield)

print(gen)
print(dir(gen))
print('use next(generator_object)')
print(next(gen))
print(next(gen))
print(next(gen))

# next(gen) exception happend!!!
print("use for/in to iterate generator")
for stage in f():
    print(stage)
<generator object f at 0x7f3e30235f90>
['__class__', '__del__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__lt__', '__name__', '__ne__', '__new__', '__next__', '__qualname__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'close', 'gi_code', 'gi_frame', 'gi_running', 'gi_yieldfrom', 'send', 'throw']
use next(generator_object)
start
middle
end
use for/in to iterate generator
start
middle
end

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
# List is iterable, but it's not an Iterator!
# but can be converted to an Iterator
class MyList:
    def __init__(self, data):
        self.__list = data[:]
        
    def __iter__(self):
        return MyIterator(self) # return Iterator
        
    def __getitem__(self, i):
        return self.__list[i]
    
    def __len__(self):
        return len(self.__list)

    
class MyIterator:
    def __init__(self, obj):
        self.__obj = obj
        self.__index = 0 # key: the current index
        self.__size = len(obj)
    
    # must __next__ as it's Iterator
    def __next__(self):
        if self.__index >= self.__size:
            # stop next by return StopIteration!!!
            raise StopIteration
        else:
            value = self.__obj[self.__index] # call obj.__getitem__()
            self.__index += 1
            return value

            
lt = [1, 2]
mlt = MyList(lt)
it = iter(mlt)
print(it)
print(next(it))
print(next(it))
<__main__.MyIterator object at 0x7f3e30278d00>
1
2