python-basics

Posted on Edited on In

Overview

Here are some basics about python3 that you should know. if your code is written with python2, convert it to python3 with 2to3 tool.

drawing 
1
2
$ 2to3 python2_file
$ 2to3 python2_dir

You can also run your app with virtual env, so that different python applications can use different python(python2 or python3) libraries

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
$sudo pip3 install virtualenv

#(python version >= 3.3)
$python3 -m venv ~/my_app_env

#(will copy python3 binary(default) and related libs(less) from system, my_app_env is a directory)

# if you want to inherit all system sites packages
$python3 -m venv ~/my_app_env --system-site-packages

$source ~/my_app_env/bin/activate
#(set python path as ~/my_app_env, run with virtual env)

(my_new_app)$ pip3 install flask
#(will install flask at ~/my_app_env (dir)!!! not system lib directory, so that each env can have separate flasks)

(my_new_app)$deactivate

.py, pyc, pyo

  • .py: this is normal input source code that you’ve written.
  • .pyc: this is the compiled bytecode. If you import a module(python file), python will build a *.pyc file that contains the bytecode for importing to make it easier (and faster). pycache/lib.cpython-39.pyc
  • .pyo: this is a another form of *.pyc file that was created while optimizations (-O) was on.

When the Python interpreter is invoked with the -O flag, optimized code is generated and stored in ‘.pyo’ files.

.pyo is gone, never use it

A program doesn’t run faster when it is read from a ‘.pyc’ or ‘.pyo’ file than when it is read from a ‘.py’ file,the only thing that’s faster about '.pyc' or '.pyo' files is the speed with which they are loaded.

When a script is run by giving its name on the command line, the bytecode for the script is never written to a '.pyc' or '.pyo' file automatically. Thus, the startup time of a script may be reduced by moving most of its code to a module and having a small bootstrap script that imports that module.

1
2
3
4
5
#generate pyc manually(pyc is generated automatically for module when it's imported by others)
$ python3 -m compileall test.py
$ ls __pycache__/
test.cpython-3.9.pyc
$ python3 __pycache__/test.cpython-3.9.pyc

generic methods for types

Each variable has a type, each type has some common methods and specific methods, here are generic rules you should know

methods for all types(str, list, tuple, range, dict, set)

  • dir(str) to get methods of type(str) for help, or detail for a specific one help(str.index).

  • item in s, item not in s, len(s), max/min(except dict, str), for item in s(s must be iterable)

  • NOTE: len(str) DO NOT count '\0'

methods for sequence type(str, list, tuple, range)

  • s[i], s[i:j](not include j), s[i:j:k], s + s1, s.count(x)
  • a, b, c = s (size of s must be 3)
  • call func(*s), pass each elem to func, like func(s[0], s[1], s[2])
  • the first index of sequence is 0

you can’t declare var with explicit type, the type of var depends on its value, can be changed at runtime!!!

naming convention

python uses underscores to write multiword names like get_name, no capital letter.

Files

  • python follows a convention where source files are all lower case with underscore separating multiple words, client_log.py
  • Compound file names are separated with _

Functions and Methods

  • private functions like this _do_work()
  • built-in function like this __init__()
  • If a name consists of multiple words, should write like this get_name().
  • function names are case-sensitive (car, Car and CAR are three different variables).

Variables

  • should not include the name of your type in the name of your variable’s name, tet_list
  • Generally, use relatively simple (short) name(lower case), (_ underscore for multiple worlds) long var.
    • user to u
    • user_id to uid
    • server_listener
    • lpcfg
  • If variable type is bool, its name should start with has, is, can or allow, etc.
  • Single letter represents index: i, j, k

Constants

  • Constant should be capitalized all letters. WORLD

class

  • type of class should be capitalized and camel case Class Person(object):
  • private function(_private_f()) and built-in(__eq__()) function can NOT be accessed!

import package

  • package(is dir with init.py under it) name should be lowercase. like import xxx.testhello, xxx is package name

printing

Printing is an easy way to show result, there are several ways to print variables.

  • print one line
1
2
3
4
print("hello"
      "jason")
print("hello \
       jason")
  • print multiple lines
1
2
3
print("hello \n jason")
print('''hello
      jason''')
  • show one parameter at last position
1
2
3
name = "jason"
print("hello", name) # a space is added between after 'hello' automatically
print("hello %s" % name) # no auto added space (python2 format)
  • show multiple parameters at different positions
1
2
3
4
5
6
7
8
9
10
11
# position parameters
msg = "welcome"
print("hello %s msg %s" % (name, msg)) # python2 format

# named parameters like {0} or {n}
# format index start from 0
print("hello {0}, msg {1}".format(name, msg))
# no need to write index if used with order
print("hello {}, msg {}".format(name, msg))
print("hello {1}, msg {0}".format(name, msg))
print("hello {n}, msg {m}".format(n=name, m=msg))
  • variable can be at any position
1
2
3
4
# print add space between different parts!!!
print(name, "hi") #has auto added space before 'hi'
print(name, "hi", name) # has auto added space before and after 'hi'
print(name, "hi %s" % name) # has auto added space before 'hi', python2 format
  • print with string.format()
1
2
print('{{ {0}'.format('hi'))
# {{ for escape
  • print with template which is less verbose than format() but need high python version
1
2
3
4
5
6
7
8
9
10
11
12
# use var directly in template with {}
name='jason'
print(f"hi{name}")

# print in multiple lines
print(f'''hi {name}
boy
''')

# more advanced, use var and function directly!!!
print(f'hi{2+3}')
print(f'hi{fun1(name)}')

Suggestion for python3 printing

  • print(‘{}’.format(var))
  • print(f”hello {name}”)

format() details

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
name='jason'
def msg(name):
    return 'hi ' + name
print(f'hi {name}')
print(f'hi {2+3}')
print(f'{msg(name)}')

# print in one lines
msg=(
    "hello "
    "world"
)
print(msg)
print("hello "
      "world")

# + is optional
print("hello " +
      "world")

# write at different lines with \ or () but print in one line
print("hello \
world in one line")

# print in multiple lines
print("hello \nworld")
print('''hello
world''')
hi jason
hi 5
hi jason
hello world
hello world
hello world
hello world in one line
hello 
world
hello
world

built-in APIS

Python(without library) provides many methods to make it easy to use. here is a list of that.

  • Complex
    1
    2
    3
    c1 = 4 + 3j
    c2 = complex(4, 3)
    c1*c2
  • Built-in function
    1
    2
    3
    4
    5
    6
    7
    8
    pow(2, 3)
    divmod(8, 4) # result: (2, 0)
    abs(-12)
    sum([1,2]) sum([False, True])
    max([1, 2])
    min([1,2])
    round(3.23, 1) # return 3.2
    round(3.26, 1) # return 3.3
  • built-in modules
    1
    2
    3
    4
    5
    6
    7
    8
    import math
    math.pi
    math.sin()
    math.cos()
    math.pow()
    math.log2()
    math.ceil(2.1) == 3   # rounds up to close integer
    math.floor(2.1) == 2  # rounds down to close integer
  • assert
    1
    2
    3
    a = 1
    assert a, 'bad value'  # assert true, never raise exception.
    assert 0, 'bad value'  # assert fail, exception with msg 'bad value'
  • argv
    1
    2
    3
    print(sys.argv) # argv is list that includes the app name and parameter
    # ./app.py -f xx.text
    # sys.argv = ['./app.py', '-f', 'xx.text']

constants

  • Python doesn’t have built-in constant types.
  • By convention, Python uses a variable whose name consist of capital letters to define a constant.

NOTE: someone can go and change the value of constant, even it’s unlikely but possible.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
MY_CONSTANT = "Whatever"


# trick way
class CONST(object):
    FOO = 1234

    def __setattr__(self, *_):
        pass

CONST = CONST()
print CONST.FOO    # 1234

CONST.FOO = 4321   # as we do thing for setter as above.
CONST.BAR = 5678

print CONST.FOO    # Still 1234!
print CONST.BAR    # Oops AttributeError

# or this way
class Const:
  @property
  def FOO(self):
    return "foo"

CONST = Const()  # You need an instance

if something == CONST.FOO:

variable

A variable is a label or a name given to a certain location in memory. This location holds the value you want your program to remember for use later on. What’s great in Python is that you do not have to explicitly state what the type of variable you want to define is - it can be of any type (string, integer, float, list, dict etc.). To create a new variable in Python, you simply use the assignment operator (=, a single equals sign) and assign the desired value to it, its type is determined by its value, but if you change the value, the type can be changed as well.

1
2
3
4
5
first_string_var = "First String"  
first_int_var = 1

total = 1 + 2 * 3
total = "hello"

Local Scope

A variable created inside a function belongs to the local scope of that function, and can only be used inside that function.

1
2
3
4
5
6
def print_number():
    first_num = 1
    # Print statement 1
    print("The first number defined is:", first_num)

print_number()
The first number defined is: 1

Enclosing Scope

Inner function can access var defined in outer function directly, while the reverse direction is not allowed.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
def outer():
    first_num = 1
    
    def inner():
        second_num = 2
        # Print statement 1 - Scope: Inner
        print("first_num from outer:", first_num)
        # Print statement 2 - Scope: Inner
        print("second_num from inner:", second_num)
    inner()
    # Print statement 3 - Scope: Outer
    
    # this will cause error if uncommented
    # print("second_num from inner: ", second_num)

outer()
first_num from outer: 1
second_num from inner: 2

nolocal keyword

The nonlocal keyword is useful in nested functions. It causes the variable to refer to the previously bound variable in the closest enclosing scope. In other words, it will prevent the variable from trying to bind locally first,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
def outer():
    first_num = 1
    
    def inner():
        # try without nonlocal
        # without nonlocal, you can still access first_num, but if you modify it, a new local variable is defined
        nonlocal first_num
        first_num = 0
        second_num = 1
        print("inner - second_num is:", second_num)
        
    inner()
    print("outer - first_num is:", first_num)

outer()
inner - second_num is: 1
outer - first_num is: 0

Global Scope

A variable created in the main body of the Python code is a global variable and belongs to the global scope.

Global variables are available from any scope, global and local.

If you need to create a global variable, but are stuck in the local scope, you can use the global keyword.

The global keyword makes the variable global which is defined in a function or refer to a predefined global variable

1
2
3
4
5
6
7
def gb(new_greeting):
    # define a global variable from a function or refer to an existing one if defined somewhere!!!
    global gt
    gt = new_greeting
    
gb("cool")
print(gt)
1
2
3
4
5
6
7
8
9
10
11
greeting = "Hello"

def greeting_world():
    world = "World"
    print(greeting, world)

def greeting_name(name):
    print(greeting, name)

greeting_world()
greeting_name("Samuel")
Hello World
Hello Samuel

Global keyword

To change the value of a global variable inside a function, refer to the variable by using the global keyword, it’s similar like nolocal keyword.

1
2
3
4
5
6
7
8
9
10
11
12
13
greeting = "Hello"

def change_greeting(new_greeting):
    # try without it
    global greeting
    greeting = new_greeting

def greeting_world():
    world = "World"
    print(greeting, world)

change_greeting("Hi")
greeting_world()
Hi World

types

operators

Arithmetic operators

Operator Meaning Example
+ Add two operands or unary plus x + y+ 2
- Subtract right operand from the left or unary minus x - y- 2
* Multiply two operands x * y
/ Divide left operand by the right one (always results into float) x / y
% Modulus - remainder of the division of left operand by the right x % y (remainder of x/y)
// Floor division - division that results into whole number adjusted to the left in the number line x // y
** Exponent - left operand raised to the power of right x**y (x to the power y)

Comparison operators

Operator Meaning Example
> Greater than - True if left operand is greater than the right x > y
< Less than - True if left operand is less than the right x < y
== Equal to - True if both operands are equal x == y
!= Not equal to - True if operands are not equal x != y
>= Greater than or equal to - True if left operand is greater than or equal to the right x >= y
<= Less than or equal to - True if left operand is less than or equal to the right x <= y

Logical operators

Operator Meaning Example
and True if both the operands are true x and y
or True if either of the operands is true x or y
not True if operand is false (complements the operand) not x

Bitwise operators

Operator Meaning Example
& Bitwise AND x & y = 0 (0000 0000)
Bitwise OR
~ Bitwise NOT ~x = -11 (1111 0101)
^ Bitwise XOR x ^ y = 14 (0000 1110)
>> Bitwise right shift x >> 2 = 2 (0000 0010)
<< Bitwise left shift x << 2 = 40 (0010 1000)

Assignment operators

Operator Example Equivalent to
= x = 5 x = 5
+= x += 5 x = x + 5
-= x -= 5 x = x - 5
*= x *= 5 x = x * 5
/= x /= 5 x = x / 5
%= x %= 5 x = x % 5
//= x //= 5 x = x // 5
**= x **= 5 x = x ** 5
&= x &= 5 x = x & 5
= x
^= x ^= 5 x = x ^ 5
>>= x >>= 5 x = x >> 5
<<= x <<= 5 x = x << 5

Membership operators

Operator Meaning Example
in True if value/variable is found in the sequence 5 in x
not in True if value/variable is not found in the sequence 5 not in x

NOTE: +, ==, != works for tuple, list, string (sequence type)as well.

Integer

Integer supports most operators like C language but with less difference, here are the operators it supports.

  • +, -, *, /, //
  • **, %
  • x or y (C ||), x and y (C &&), not x (C !)
  • <, <=, >, >=, ==, !=
  • x | y, x & y, x ^ y, x >> y, x << y
1
2
3
4
5
6
7
8
9
10
def integer_demo():
    print("convert str to int int('12') =", int('12'))
    print("convert int to string str(12) =", str(12))
    print("// only remains interger 5//1.5=", 5//1.5)
    # format float
    print("/  result is float 1/2=%0.2f 1/2=%0.4f"%(1/2, 1/2))
    print("%0.2f"%1.234)
    print(format(1.234, ".2f"))
    print("2**3 =", 2**3) # like pow(2,3)
    print("8%4 =", 8 % 4) # result: 0 while divmod(8,4)=(2,0)
1
integer_demo()
convert str to int int('12') = 12
convert int to string str(12) = 12
// only remains interger 5//1.5= 3.0
/  result is float 1/2=0.50 1/2=0.5000
1.23
1.23
2**3 = 8
8%4 = 0

string

string is a kind of sequence, like list, tuple, but its content can't be modified in place, in order to change its content, you must create a new one.

no char type in python, that means ‘’ and “” has the same meaning

Create a string

  • s = “hello”
  • s = ‘hello’
  • s = “hi” + “world”
  • s = r'hello\nworld' # raw literal string now \n is a normal char!!

Ops

  • index from 0, last index -1
  • len(str), no \0 is there but we count in C.
  • count(), index(), isdigit(), islower(), lower(), isupper(), upper(), isspace(), join(), replace(), find()(first matched one), split(), splitlines(), strip(), startwith(), endwith(), rfind()(last matched one)
  • join() each string element of iterable object, “-“.join([1,2]) will cause error as element is not string!!!
  • str.strip('hy') will strip h or y both from right and left not ‘hy’ as a whole
  • index() and find() return same if found(index), otherwise, exception ValueError for index, -1 returned for find()
  • slicing [2:5], [2:], [:5], [2:-1] all with default step 1 from left to right
  • slicing with steps, [2:5:2] ([start:end:step])
  • [::-1](reverse the string)
  • str.xxx() all such methods do NOT support regular pattern
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
def str_demo():
    print("raw string r\"a\\nb\" = ", r"a\nb")
    # very useful when use re as regular pattern may contain special char to python
    # use r before a string to suppress(r means raw literal)

    s1 = "hello boy"
    
    print("str='hello boy'")
    print("reverse s1 by s1[::-1]=", s1[::-1])
    print("count sub words, str.count('o') =", s1.count('o'))
    print("first element index 0, str[0] =", s1[0])
    print("last element index: str[-1] =", s1[-1])
    print("not include \\0 len(str) =", len(s1))
    print("a:b means [a, b)  str[:-1] =", s1[:-1])
    try:
        s1[2] = 'a'
    except:
        print("can't change the content of a string by str[2] = 'a'")

    print("create a copy of str by str[:]!")
    # newstr = s1[:]#a copy a str.

    print("str is: " + s1)
    print('s1.startswith("hello") =', s1.startswith("hello"))
    print('s1.endswith("bo") =', s1.endswith("bo"))
    if s1.isdigit():
        print("str is digital")
    elif s1.islower():
        print("str is lower")
    elif s1.isupper():
        print('str is upper')
    elif s1.isspace():
        print("str is space")
    else:
        print("str is", s1)

    idx = s1.find("hello")
    if idx != -1:
        print("find hello at %d in : %s" % (idx, s1))
    print("replace boy with girl, new str is: " + s1.replace("boy", "girl"))
    print("splitlines() == split('\\n'), split() will return a list, but s1 is unchanged s1 = %s s1.split(' ') = %s" % (s1, s1.split(' ')))
    print('''strip() removes characters(whitespace by default)
             from both left and right based on the argument set of character
             while rstrip() only from right''')
    print("before strip str =", s1)
    print("after strip('hy') str =", s1.strip('hy'))
    print("join() join each string elem in an iterable object with character provided, '-''.join('hello') =", '-'.join("hello"))
    print("'-'.join(['hello', 'boy'] =", '-'.join(["hello", "boy"]))
1
str_demo()
raw string r"a\nb" =  a\nb
str='hello boy'
reverse s1 by s1[::-1]= yob olleh
count sub words, str.count('o') = 2
first element index 0, str[0] = h
last element index: str[-1] = y
not include \0 len(str) = 9
a:b means [a, b)  str[:-1] = hello bo
can't change the content of a string by str[2] = 'a'
create a copy of str by str[:]!
str is: hello boy
s1.startswith("hello") = True
s1.endswith("bo") = False
str is lower
find hello at 0 in : hello boy
replace boy with girl, new str is: hello girl
splitlines() == split('\n'), split() will return a list, but s1 is unchanged s1 = hello boy s1.split(' ') = ['hello', 'boy']
strip() removes characters(whitespace by default)
             from both left and right based on the argument set of character
             while rstrip() only from right
before strip str = hello boy
after strip('hy') str = ello bo
join() join each string elem in an iterable object with character provided, '-''.join('hello') = h-e-l-l-o
'-'.join(['hello', 'boy'] = hello-boy

encoding and decoding

Python3’s str type is meant to represent human-readable text and can contain any Unicode character.

The bytes type, conversely, represents binary data or sequences of raw bytes, that do not intrinsically have an encoding attached to it.

Python 3 is all-in on Unicode and UTF-8 specifically. Here is what that means:

  • Python 3 source code is assumed to be UTF-8 by default. This means that you don’t need # -- coding: UTF-8 -- at the top of .py files in Python 3.

  • All text (str) is Unicode by default. Encoded Unicode text is represented as binary data in memory (bytes). The str type can contain any literal Unicode character, such as “Δv / Δt”, all of which will be stored as Unicode.

  • Python’s re module defaults to the re.UNICODE flag rather than re.ASCII. This means, for instance, that r”\w” matches Unicode word characters, not just ASCII letters.

  • The default encoding/decoding in str.encode() and bytes.decode() is UTF-8.

1
2
3
4
5
6
7
8
9
10
11
12
13
# The length of a single Unicode character as a Python str will always be 1
# no matter how many bytes it occupies.
# The length of the same character encoded to bytes
# will be anywhere between 1 and 4.

def encode():
    ibrow = "🤨"
    print('len of character:', len(ibrow))
    bs = ibrow.encode("utf-8")
    print('encoded with utf-8:', bs)
    print('len of bytes:', len(bs))
    
encode()
len of character: 1
encoded with utf-8: b'\xf0\x9f\xa4\xa8'
len of bytes: 4

list

List is a sequence, element can be any type even for one list. but for sort() method it requires same type of all elements.

To create a list

  • lv = list()
  • lv = [], lv == [] True, len(lv)==0
  • lv = [1, 2]
  • lv = list(‘abc’) same as lv = [‘a’, ‘b’, ‘c’]
  • lv = list(dt.items()), dt={‘a’:1} same as lv=[(‘a’, 1)]
  • lv = list(range(3)) same as lv = [0, 1, 2]
  • lv = [1, ‘a’, False, [3, 4]]
  • lv = [1] * 3 same as [1, 1, 1]
  • lv = old[:] # shadow copy, only the top level is copied
  • lv = [val+10 for val in arr if val >= 0]

Ops

  • append(value), count(value), extend(iterable), index(value, start=None, stop=None), insert(index, value), pop(index=None)(default from last), remove(value), reverse(), sort(key=None, reverse=False)
  • sort() needs all elements have the same type.
  • for i in range(len(lt)):lt[i]
  • for elem in lt: print(elem)
  • for i in range(len(lt) - 1, -1, -1):lt[i] loop from last

Note

  • reverse() and sort() are in place methods.
  • if you pass a list to a function, modification in that function of the list will take place in original.
  • pop() from last by default ----> pop(0) pop from head.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
import copy

def list_demo():
    la = ['1', 5, [3, 4], 2]
    print("list element can be any type: ", la)
    la.append(6)
    print("append only take one element, after list.append(6): ", la)

    la.extend([3, 4])
    print("after list.extend([3,4]) = ", la)
    la.insert(1, 10)
    print("after insert 10 at index 1, the previous index 1 move back, list.insert(1,10): ", la)
    
    # la.pop() remove the last
    # la.pop(0) remove the first one(from from index 0)
    
    la.pop(2) # same as del la[2] here 2 is index
    print("after pop() value at index 2, list=: ", la)
    la.remove(2) # here 2 is value but index
    print("after remove() element whose value is 2,\
           list.remove(2) now list is: ", la)

    lt = [2, 1, 4]
    # please note: to sort, make sure all elements has the same type!!!
    lt.sort() # take in place!!!
    print("[2, 1, 4] after sorted , now lt is %s",lt)
    lt.reverse()
    print("list.reverse() after reverse now list is: ", lt)
    
    # loop list with index
    for index, value in enumerate(lt):
        print("with index", (index, value))
    
    
    # another way to loop list with index
    for i in range(len(lt)):
        print("with index:", i, lt[i])
    for e in lt:
        print("without index:",e)
    
    print('reverse loop: ')
    # loop list from end to start
    for i in range(len(lt) - 1, -1, -1):
        print(i, lt[i])
    
    
    # shadow copy, only the top level is copied
    la = ['1', 2, [3, 4], 6]
    cla = la[:]
    print("shadow copy la: %s, cla: %s" %(la, cla))
    cla[1] += 10
    print("cla[1] += 10, now la:",la)
    cla[2].append(5)
    print("cla[2].append(5), la changes as well la:",la)
    
    # deep copy
    la = ['1', 2, [3, 4], 6]
    dla = copy.deepcopy(la)
    print("*"*60)
    print("deep copy la: %s, dla: %s" %(la, dla))
    dla[1] += 10
    print("dla[1] += 10, now la:",la)
    dla[2].append(5)
    print("dla[2].append(5), la changes as well la:",la)

1
list_demo()
list element can be any type:  ['1', 5, [3, 4], 2]
append only take one element, after list.append(6):  ['1', 5, [3, 4], 2, 6]
after list.extend([3,4]) =  ['1', 5, [3, 4], 2, 6, 3, 4]
after insert 10 at index 1, the previous index 1 move back, list.insert(1,10):  ['1', 10, 5, [3, 4], 2, 6, 3, 4]
after pop() value at index 2, list=:  ['1', 10, [3, 4], 2, 6, 3, 4]
after remove() element whose value is 2,           list.remove(2) now list is:  ['1', 10, [3, 4], 6, 3, 4]
[2, 1, 4] after sorted , now lt is %s [1, 2, 4]
list.reverse() after reverse now list is:  [4, 2, 1]
with index (0, 4)
with index (1, 2)
with index (2, 1)
with index: 0 4
with index: 1 2
with index: 2 1
without index: 4
without index: 2
without index: 1
reverse loop: 
2 1
1 2
0 4
shadow copy la: ['1', 2, [3, 4], 6], cla: ['1', 2, [3, 4], 6]
cla[1] += 10, now la: ['1', 2, [3, 4], 6]
cla[2].append(5), la changes as well la: ['1', 2, [3, 4, 5], 6]
************************************************************
deep copy la: ['1', 2, [3, 4], 6], dla: ['1', 2, [3, 4], 6]
dla[1] += 10, now la: ['1', 2, [3, 4], 6]
dla[2].append(5), la changes as well la: ['1', 2, [3, 4], 6]

pack/unpack list

unpack list to separate elements, while pack from separate elements to a list, this is mostly used for parameter passing.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
def list_unpack_pack():
    # unpack, must have the same elements number
    a, b, c = [1, 2, 3]
    a, *e = [1, 2, 3]
    
    
    def test_unpack(a, b, c):
        print(a, b, c)
    # *list in call to unpack list
    test_unpack(*[1, 2, 3]) # same element numbers

    
    # *list in definition to pack separate ones, must be used as the last argument.
    def test_pack(*lt):
        print(lt)
    test_pack(1, 2, 3)
    
list_unpack_pack()
1 2 3
(1, 2, 3)

dict

dict is not sequence type, but we can convert dict to a list.

to create a dict

  • dt = {}
  • dt = dict()
  • dt = {“a”:1, “b”:2}
  • lt = [(‘a’, 1), (‘b’, 2)] dt = dict(lt)

key without '' or "", treated as variable if not literal must define it firstly

1
2
3
4
5
6
# both are valid, but key 2 and '2' are different keys!!!!
dt = {2: 1}
dt = {'2': 1}

# must define var a firstly!!!, otherwise error!!!
dt = {a: 1}

Ops

  • dt[newkey] = 12(add a new element)

  • dt[key] (exception if no key), dt.key NOT support it s only valid for class

  • dt.get(key, default=None) # should use this always to avoid exception

  • dt.pop(key, default=None) # get its value and pop it.

  • del dt[key] # only delete it.

  • dt.keys()

  • dt.values()

  • dt.items() # each element is a tuple

  • dt.update(another_dict) # extend a dict or update existing elem

  • dt.copy() # shallow copy!!!

  • dt.clear() # reset dict

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
def dict_demo():
    dt = {"a": 1, "b": 2}
    # different with list as list is []
    print("dict now is:", dt)
    
    # add an element
    dt['c'] = 3
    print("add dt['c'] = 3 now dt is:", dt)
    dt['c'] += 1
    dt['d'] = [1, 2]
    dt['e'] = {"f": 6}
    print("""after added and modifed
           dt['c'] += 1
           dt['d'] = [1, 2]
           dt['e'] = {"f":6}
           now dict:""", dt)
    # access element, you can NOT access elem by dt.key
    print("access one element dt['e']['f'] =", dt['e']['f'])
    print("get a its value with dt.get():", dt.get('a', "doesn't exist"))
    print("get f its vlaue with dt.get():", dt.get("f",  "doesn't exist"))
    print("pop f from dict with dt.pop():", dt.pop("f", "doesn't exist"))
    print("now dic is: ", dt)
    
    # delete an element
    dt.pop('a') # delete key-value pair, can also be done: del dt['a']
    print("pop a from dict, dt.pop('a') now dict:", dt)
    print("all keys in dict is dt.keys() =", dt.keys())  # a list
    print("all values in dict is dt.values() =", dt.values())  # a list
    print("all items in dict is dt.items() =", dt.items()) # list, each element is a tuple
    
    # join two dicts if has duplicated key, use the later one(update one)
    dt.update({"c": 100, "g": 100})
    print('update dict with {"c": 100, "g": 100}, now dt =', dt)
    # iterate all elements with dict
    for it in dt.keys():
        print(it, "==", dt[it])
    dt.clear() # remove all elements
    print("dt is %s after clear()" % dt)
1
dict_demo()
dict now is: {'a': 1, 'b': 2}
add dt['c'] = 3 now dt is: {'a': 1, 'b': 2, 'c': 3}
after added and modifed
           dt['c'] += 1
           dt['d'] = [1, 2]
           dt['e'] = {"f":6}
           now dict: {'a': 1, 'b': 2, 'c': 4, 'd': [1, 2], 'e': {'f': 6}}
access one element dt['e']['f'] = 6
get a its value with dt.get(): 1
get f its vlaue with dt.get(): doesn't exist
pop f from dict with dt.pop(): doesn't exist
now dic is:  {'a': 1, 'b': 2, 'c': 4, 'd': [1, 2], 'e': {'f': 6}}
pop a from dict, dt.pop('a') now dict: {'b': 2, 'c': 4, 'd': [1, 2], 'e': {'f': 6}}
all keys in dict is dt.keys() = dict_keys(['b', 'c', 'd', 'e'])
all values in dict is dt.values() = dict_values([2, 4, [1, 2], {'f': 6}])
all items in dict is dt.items() = dict_items([('b', 2), ('c', 4), ('d', [1, 2]), ('e', {'f': 6})])
update dict with {"c": 100, "g": 100}, now dt = {'b': 2, 'c': 100, 'd': [1, 2], 'e': {'f': 6}, 'g': 100}
b == 2
c == 100
d == [1, 2]
e == {'f': 6}
g == 100
dt is {} after clear()

pack/unpack dict

unpack dict to separate elements, while pack from separate elements to a dict, this is only used for parameter passing, when unpack, the parameters must use the same ‘key’ as dict and same count of element.

1
2
3
4
5
6
7
8
9
10
11
12
dt = {"a":1, "b":2}
def test(a, b):
    print(a,b)
    
# unpack for call a fucntion
test(**dt)

# pack argument must be the last argument with **argv for dict, *argv for list
def test1(**argv):
    print(argv)
    
test1(a=1, b=2, c=3)
1 2
{'a': 1, 'b': 2, 'c': 3}

tuple

tuple is similar as list(element can be any type) except it can NOT be modified meaning its top level element can not be changed, but the embedded object can be modified, tuple has no special methods for itself. it’s less used actually.

Create a tuple

  • tp = 1, 2
  • tp = (1, 2)
  • tp = tuple([1, 2])

Ops

  • no special method for itself, has sequence type generic methods
1
2
3
4
5
6
7
8
9
10
11
12
def tuple_demo():
    tp = (1, 2, 'd', {'a': 1}, [3, 4])
    print("tuple is:", tp)
    print("tp[:-1]:", tp[:-1])
    print("tp + (6,):", tp + (6,))
    
    # the top level can NOT change but the embeded type can be modified if possible
    tp[3]['a'] = 2 # good
    print("change the value it points tuple is:", tp)
    
    # tp can NOT modified just means it can't change top level.
    # but the value it points can be modified
1
tuple_demo()
tuple is: (1, 2, 'd', {'a': 1}, [3, 4])
tp[:-1]: (1, 2, 'd', {'a': 1})
tp + (6,): (1, 2, 'd', {'a': 1}, [3, 4], 6)
change the value it points tuple is: (1, 2, 'd', {'a': 2}, [3, 4])

set

Set is like dict without value, only keys and it’s not sequence type but iterable and has its special methods

NO duplicate element in set.

Create a set

  • st = set() empty set, only one to create set not like list, tuple, dict
  • st = set(“abc”) == st={‘a’, ‘b’, ‘c’}
  • st = {‘a’, ‘b’, ‘c’}
  • st = set([‘a’, ‘b’, ‘c’]) # from list

Ops

  • s1.add()
  • s1.remove()
  • s1.update() #extend a set with another one
  • s1 | s2 并集
  • s1 & s2 交集
  • s1 - s2 差集
  • s1 ^ st2 对称差
  • s1.issubset(s2) s1 is subnet of s2
  • s2.issuperset(s1) s2 is subnet of s1
1
2
3
4
5
6
7
8
9
10
11
def set_demo():
    # {} not []
    st = {'a', 'b', 'c'}
    print("st =", st)
    st.add(3)
    print("after st.add(3) =", st)
    st.remove('a')
    print("after st.remove('a') =", st)
    # set is iterable, but not sequence.
    for elm in st:
        print(elm)
1
set_demo()
st = {'c', 'b', 'a'}
after st.add(3) = {3, 'c', 'b', 'a'}
after st.remove('a') = {3, 'c', 'b'}
3
c
b

range

When you call range() API, it returns an object of range type, but most of time, we convert range to list or tuple implicitly.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
def range_demo():
    # range(stop)------>[start, stop)
    rg = range(5) #[0, 4]
    print("range is:", rg)
    lt = list(rg) # convert range to list explicitly
    print("convert range to list:", lt)
    tp = tuple(rg)
    print(tp)
    for i in range(5): #convert range to tuple implicitly!
        print(i)
        
    # range(start, stop, step)------>[start, stop)
    # a list from large to small
    print(list(range(5, -1, -1)))
1
range_demo()
range is: range(0, 5)
convert range to list: [0, 1, 2, 3, 4]
(0, 1, 2, 3, 4)
0
1
2
3
4
[5, 4, 3, 2, 1, 0]

conversion between different types

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# string <--> integer
a = int('12')
b = str(12)
print(a, b)

# string <---> list
lt = list('abc2')
s = ''.join(lt)
print(lt, s)

# string <-->set
st = set('abc')
s = ''.join(st)
print(st, s)

# list <--->dict
lt = list({"a":1, "b":2}.items())
print(lt)
dt = dict(lt)
print(dt)

function

variable scope is like C, variable is local by default, if want to change value of global variable, must use keyword global x in function, you can access global variable in function without global keyword, but only for accessing, not modifying.

variable defined in a function is also visible to the function embedded in it.

argument can have default value when define it

1
2
def show(x, y=0):
    print(y)

when you call function with named parameter, the order can be any

1
2
3
show(1, 2)
show(y=2, x=1) ### order can be different with function definition!!!
show(1, y=23)

For dynamic args(the number of argument is not defined)

  • use *arg for list like parameters in function definition
  • **arg for dict like parameters in function definition
1
2
3
4
5
def test(*args):
    for i in args:
        print('arg: ', i)
test([1, 2])  # passed args [[1, 2]]
test(1, 2)    # passed args [1, 2]

Note: >=python3.5, python supports define function with type

1
2
def function_demo_python3_5(name: str='jason') -> bool:
    return True
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# no global keyword when defines it.
g_y = 100
def function_demo():
    x = 12
 
    def show_y():
        print('g_y: ', g_y)
        
    def show_v():
        x = 10  # modified here, define a local x, not impact x outside
        global g_y  # no need this line if only access y, not modify it!!!!
        g_y = 200
        print("x in show_v is: ", x)

        def show_sub_v():
            # x defined in show_v is visible in show_sub_v
            print("x in show_sub_v is: ", x)
        show_sub_v()
    show_v()
    show_y()
    print("out of show_v x is: ", x)
    print("out of show_v y is:", g_y)

    # dynamic arguments
    def show_arg(x, *argc, **argv):
        print("x=%d, argc=%s argv=%s" % (x, argc, argv))
    show_arg(1, 2, 3, name='jason', id=12)
1
function_demo()
x in show_v is:  10
x in show_sub_v is:  10
g_y:  200
out of show_v x is:  12
out of show_v y is: 200
x=1, argc=(2, 3) argv={'name': 'jason', 'id': 12}

flow control

Flow control controls the logic of statement, like C, python supports many flow control directives like while/else, for/else, if/elif/else, pass, break/continue, try/except/finally, raise

NO goto, switch/case in python.

if/elif/else

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
def if_demo():
    con = 1
    short_out = 'con is 1' if con == 1 else 'con is not 1' # shortcut for if/else at same line no ':' needed
    print(short_out)
    
    a = 12 if False else 23
    print("like: a = 12 if False else 23")
    print("a =", a)
    
    if 1:
        print("say hi")
    else:
        pass  # nothing
    
    if 1:
        print("say hi")  # simple
        
if_demo()
con is 1
like: a = 12 if False else 23
a = 23
say hi
say hi

while/else

1
2
3
4
5
6
7
8
def while_demo():
    i = 10
    while i:
        i -= 1
    else:
        print("loop is out but not by break statement inside while")

while_demo()
loop is out but not by break statement inside while

for/else

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
def for_demo():
    lt = [1, 2]
    if lt[0] == 1:
        print("equal")

    for i in [1, 2, 3, 4, 5, 6]:
        if i == 4:
            print("break from loop")
            break
    else:
        print("loop for is out but not by break statement inside for")
        
    lt1 = [1, 2, 3, 4]
    lines = '-'.join([str(m) for m in lt1])
    print(lines)
    
for_demo()
equal
break from loop
1-2-3-4

try/exception

1
2
3
4
try:
    fd = open('/sdaf/b.c', 'wb')
except Exception as error:
    print(error)

system env

Sometimes, you may want to get environment from system or set an environ for a process, use from os import environ

note: environ[‘PATH’] is different with os.path which is only used for search .py

1
2
3
4
5
6
7
8
9
def env_demo():
    from os import environ
    environ['HOST'] = "10.64.32.1"
    # set env
    print('get HOST env: ', environ.get('HOST', 'HOST default value'))
    for key, v in environ.items():
        print("env-> %s : %s" % (key, v))
        
    # environ['PATH'] si different with os.path which is only used for seerch .py!!!
1
env_demo()
get HOST env:  10.64.32.1
env-> http_proxy : http://10.226.136.231:3128
env-> ftp_proxy : http://10.226.136.231:3128
env-> PATH : /home/data/Anaconda3/envs/py3.9/bin:/opt/llvm/bin:/home/data/Anaconda3/envs/py3.9/bin:/home/data/Anaconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/home/go:/home/go/bin:/root/.yarn_pkg/bin:/usr/lib64:/usr/local/go/bin:/home/data/Anaconda3/envs/py3.9/libexec/git-core:/root/bin:/root/.yarn_pkg/bin:/home/go/bin:/home/go:/usr/local/go/bin
env-> PWD : /
env-> LANG : en_US.UTF-8
env-> https_proxy : http://10.226.136.231:3128
env-> SHLVL : 1
env-> _ : /usr/bin/env
env-> GOPROXY : https://goproxy.cn,direct
env-> GO111MODULE : on
env-> GOMODCACHE : /home/go/pkg/mod
env-> GOCACHE : /root/.cache/go-build
env-> GOPATH : /home/go
env-> PYDEVD_USE_FRAME_EVAL : NO
env-> JPY_PARENT_PID : 1811
env-> TERM : xterm-color
env-> CLICOLOR : 1
env-> PAGER : cat
env-> GIT_PAGER : cat
env-> MPLBACKEND : module://matplotlib_inline.backend_inline
env-> HOST : 10.64.32.1

small tips

Id of object

In python, each object has a unique identifier(memory address), to get it by id(var), a is b checks the id of them, for integer, string, if content is same, id is same as well, but this is not true for list, tuple, dict. for example:

1
2
3
4
5
6
7
8
9
a = 3
b = 3
id(a) == id(b) # True
a is b # True

a = [1]
b = [1]
a == b # True
a is b # False

while isinstance(var, str) to check if a var is an instance of a class

None () {} [] “”

None () {} [] “” are false when used a condition.

1
2
3
4
5
6
print("""None, (), {}, [], "", '' all are false but they are different types""")
print(bool(None), bool(()), bool([]), bool({}), bool(""))


msg = 'None == []' if None == [] else "None not equal []"
print(msg)
None, (), {}, [], "", '' all are false but they are different types
False False False False False
None not equal []

exec eval

  • exec is used to execute python statements which are stored in a string or file
  • eval is used to evaluate valid python expression which are stored in a string.
1
2
3
4
5
# execute python code from string
exec('print("hello")')

# eval python expression from string
print(eval('2*3'))
hello
6

try/except

1
2
3
4
5
6
7
8
try:
    1.0 / 0
except ZeroDivisionError:
    print('Error: Division by zero')
except TypeError:
    print('Error: Unsupported operation')
except Exception as error:  # catch all other exception
    print(error) # general error, unknown
Error: Division by zero

ref