go-advanced

Posted on Edited on In ,

Advanced

struct

tags

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
package main

import (
    "fmt"
    "reflect"
)

type Person struct {
    Name string `k1:"v1" k2:"v2"`
}

func main() {
    var p Person
    tp := reflect.TypeOf(p)
    for i := 0; i < tp.NumField(); i++ {
        f := tp.Field(i)
        fmt.Println(f.Tag.Get("k1"))
        fmt.Println(f.Tag.Get("k2"))
    }
}
main()
v1
v2

Rules for writing tag

  • whitespace, double quote ", colon : are special in tags.

  • Tag keys must not contain space (Unicode value 32), quote "(Unicode value 34) and : colon (Unicode value 58) characters.

  • To form a valid key-value pair, no space characters are allowed to follow the colon in the supposed key-value pair. So

    optional: "yes" doesn’t form key-value pairs.

  • different key-value pairs are separated by whitespace

  • space characters in tag values are important. So

    json:"author, omitempty",
    json:" author,omitempty" and
    json:"author,omitempty" are different from each other.

  • each struct field tag should present as a single line to be wholly meaningful.

Exported struct
If a struct type starts with a capital letter, then it is an exported type and it can be accessed from other packages. Similar the fields of a struct start with caps, they can be accessed from other packages.

Structs Equality
Structs are value types and are comparable if each of their fields are comparable. Two struct variables are considered equal if their corresponding fields(field with same name) are equal.

anonymous fields

It is possible to create structs with fields that contain only a type without the field name. These kinds of fields are called anonymous fields. Even though anonymous fields do not have an explicit name, by default the name of an anonymous field is the name of its type.

1
2
3
4
5
6
7
8
9
type Person struct {
    string
    int
}
// same as 
type Person struct {
    string string
    int int
}

Person struct has 2 fields with name string and int!!!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
package main

import (
    "fmt"
)

type Person struct {
    string
    int
}

func main() {
    p1 := Person{
        string: "naveen",
        int:    50,
    }
    fmt.Println(p1.string)
    fmt.Println(p1.int)
}

Fields that belong to an anonymous field which is also a struct are called promoted fields since they can be accessed as if they belong to the struct which holds the anonymous struct. promoted fields can be functions as well!!!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
package main

import (
    "fmt"
)

type Address struct {
    city  string
    state string
}

func (a *Address) City() string {
    return a.city
}

type Person struct {
    name    string
    age     int
    Address //anonymous struct field
}

type Personx struct {
    name     string
    age      int
    *Address //anonymous struct field *Address is new type.
}

func main() {
    p := Person{
        name: "Naveen",
        age:  50,
        Address: Address{
            city:  "Chicago",
            state: "Illinois",
        },
    }

    px := Personx{
        name: "Naveen",
        age:  50,
        Address: &Address{ // different ways for assigning
            city:  "Chicago",
            state: "Illinois",
        },
    }

    fmt.Println("Name:", p.name)
    fmt.Println("Age:", p.age)
    fmt.Println("City:", p.Address.city)  // can access this way as well
    fmt.Println("City:", p.Address.state) // can access this way as well
    fmt.Println("City:", p.city)          //city is promoted field
    fmt.Println("City:", p.City())        //City() function is promoted field
    fmt.Println("State:", p.state)        //state is promoted field

    // same way as above for accessing pointer
    fmt.Println("Name:", px.name)
    fmt.Println("Age:", px.age)
    fmt.Println("City:", px.Address.city)  // can access this way as well
    fmt.Println("City:", px.Address.state) // can access this way as well
    fmt.Println("City:", px.city)          //city is promoted field
    fmt.Println("City:", px.City())        //City() is promoted field
    fmt.Println("State:", px.state)        //state is promoted field
}

class

There is no class in Go, but you can bind functions with struct, hence it behaves like a class.
func (p Person) speak() string {} The receiver appears in its own argument list between the func keyword and the method name.

You can only declare a method with a receiver whose type is defined in the same package as the method. You cannot declare a method with a receiver whose type is defined in another package

  • Value receiver makes a copy of the type and pass it to the function. The function stack now holds an equal object but at a different location on memory. That means any changes done on the passed object will remain local to the method. The original object will remain unchanged.

  • Pointer receiver passes the address of a type to the function. The function stack has a reference to the original object. So any modifications on the passed object will modify the original object.

If you want to change the state of the receiver in a method, manipulating the value of it, use a pointer receiver. It’s not possible with a value receiver, which copies by value. Any modification to a value receiver is local to that copy. If you don’t need to manipulate the receiver value, use a value receiver(pointer receiver can be used as well).

The Pointer receiver avoids copying the value on each method call. This can be more efficient if the receiver is a large struct.

Value receivers are concurrency safe, while pointer receivers are not concurrency safe. Hence a programmer needs to take care of it.

RULES for receivers

  • Try to use same receiver type for all your methods as much as possible, not use both.
  • If state modification needed, use pointer receiver if not use value receiver(but can use pointer receive as well, good case for large struct).

Specific class(not interface) method supports

  • call pointer receiver on non-pointer object(which is converted to pointer automatically)
  • call value receiver on pointer object(which is converted to value object automatically)

Note: Above supports only work for class method, NOT normal function, as normal function with a pointer argument must take a pointer, normal function with a value argument must take a value object not a pointer

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
type Person struct {
    name string
}

// type T and type *T are different types but *T contains method of T!!!
func (p *Person) speak() string {
    return "Speak() called by " + p.name
}

func (p Person) say() string {
    return "say() called by " + p.name
}

func test() {
    p1 := Person{"Jack"}
    p1.speak() // (&p).speak() automatically done by Go, but speak() is not a method of Person!!
    
    p2 := &Person{"jason"}
    p2.say() //(*p2).say() automatically done by Go, but say() is a method of Person!!!
}

If you were to call p.speak(), the compiler would automatically change that to (&p).speak(). A similar conversion happens in the other direction if you have a method with a non-pointer receiver and you call it on a pointer, easier to use.

Call method on struct nil pointer, no exception in Go, nil return

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
import "fmt"

type Vertex struct {
  x, y int
  // this is no method delcare here!!!
}

// Bad way if want to update, but it's ok if not updating caller object
func (v Vertex) Update() { // value receiver
  v.x = 10
  v.y = 20
}

// Good way use (v *Vertex) to bind with struct Vertex
// bound with Vertex struct
func (v *Vertex) Scale(f int) { // pointer receiver
  v.x = v.x * f
  v.y = v.x * f
}

func classDemo() {
    // call pointer receiver on non-pointer object
    v1 := Vertex{1, 2}
    
    v1.Update() // copy of v1 is passed!!!
    fmt.Println(v1) // v1 is not changed at all
    
    v1.Scale(2) // pointer receiver (&v1).Scale() is called automatcially by Go
    fmt.Println(v1)
    
    
    // call value receiver on pointer object
    v2 := &Vertex{1, 2}
    
    v2.Update() // value receiver (*v2).Update() is called automatcially by Go
    fmt.Println(*v2) // v2 is not changed at all
    
    v2.Scale(2)
    fmt.Println(*v2)
}

classDemo()
{1 2}
{2 4}
{1 2}
{2 4}

Interface

Types implicitly satisfy an interface if they implement all required methods defined by that interface.

type T and type *T are different types but *T contains all methods of T, but the other side is not true, even not true you still can call *T method from T object like above(actually implicit conversion happened)

The method set of any other type T consists of all methods declared with receiver type T. The method set of the corresponding pointer type *T is the set of all methods declared with receiver *T or T (that is, it also contains the method set of T).

That means:

  • If a type T implements all methods of an interface using value receiver, then both value T and pointer of that type *T can be used while assigning to that interface variable or while passing to a function which accept an argument as that interface.

  • If a type T implements all methods of an interface using pointer receiver, then the only pointer of that type *T can be used while assigning to that interface variable or while passing to a function that accepts an argument as that interface.

1
2
3
4
5
6
7
8
9
type Humaner interface {
    speak() string // no func keyword at the beginning
}

// two common used interface var
var hi Humaner = struct_object
var hi Humaner = &struct_object

var hi *Humaner = &struct_object // Never see such way used!!!

interface {} is a special type which has no method, hence all types can be converted to it, it looks like void* in C but it’s not an pointer in Go, an string, int, object, &object can assign to it as well

1
2
3
4
5
6
7
8
9
10
11
12
13
type Person struct {
    Name string
}
func test(a interface{}) {
    var b interface {}
}

test(12)
test("hello")

// both are ok!!
test(Person{Name: "Jason"})
test(&Person{"Jason"})

Call method on interface nil pointer, runtime error!!!

More details, refer to inside interface

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
import (
    "fmt"
)

type Person struct {
    name string
}

type Humaner interface {
    speak() string
    say() string
}

func (p Person) say() string {
    return "say() called by " + p.name
}

func (p *Person) speak() string {
    return "speak() called by " + p.name
}

func speakSomething(h Humaner) {
    fmt.Println("area", h.speak())
}

func saySomething(h Humaner) {
    fmt.Println("area", h.say())
}

func demoInterface() {
    p := Person{"harsh"}
    
    //works
    speakSomething(&p)
    // works because *person has method implemented by person as well.
 
    // both do NOT work(compiler error) because type person does not implment speak() method
    // hence can't convert type person to human interface
    // speakSomething(p) compiler error
    // saySomething(p)   compiler error.
    
    //works, as for specific class p.speak() converted to (&p).speak() by Go automatically
    fmt.Println(p.speak())
    
    fmt.Println(p.say())
}

demoInterface()
area speak() called by harsh
speak() called by harsh
say() called by harsh

embedding and composing struct

Embedding old way used like C 

1
2
3
4
5
6
7
8
9
type User struct {
  Name string
}

type Admin struct {
  u User
  permissions map[string]string
}

Embedding go supported new way 

1
2
3
4
5
6
7
8
9
10
11
12
13
type User struct {
  Name string
}

type Admin struct {
  User  //anonymous fields
  /* all its methods are “promoted” to the Admin as well. 
   * That means one can reference the name of the user via admin.Name no intermediate call to the u needed, like Admin.u.Name
   // short way: Admin.Name
   // another way: Admin.User.Name 
   */ 
  permissions map[string]string
}

Composing types
which consists of embedding various types to create other types/interfaces

1
2
3
4
type ReadWriter interface {
    Reader //anonymous fields
    Writer //anonymous fields
}

What I can see from the above definition is that a ReadWriter is an interface which must contain all the functions defined on both Reader and Writer, which are defined elsewhere.

NOTE

  • There is no function signature in struct type like we did for C
  • If you embed a Interface in struct, that means you declare a Interface variable of the struct!!!

empty struct

Instance of empty struct struct{} in doesn’t occupy any memory. It is of zero byte. it’s used mostly in two cases:

  • Empty struct is a very good use case in a channel when you only want to use a channel for notification and not for actually passing in any data. but some one uses bool channel, which is accepted, but empty struct is better choice!!!

  • Implementation of Set data structure. A set is a data structure that holds elements without any particular order. An element only appears once in a set. We use map[keyType]struct{} for set. struct{} is only just to let us know if an element exists in the set or not.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
package main

import "fmt"

type Set map[string]struct{}

func (s Set) Add(key string) {
    // struct {} is type, while struct{}{} is an instance of this type.
    s[key] = struct{}{}
}

func (s Set) Delete(key string) {
    delete(s, key)
}

func (s Set) Has(key string) bool {
    _, ok := s[key]
    return ok
}

func main() {
    s := Set{}
    s.Add("a")
    
    // save slot for same key
    s.Add("a")
    s.Add("b")
    s.Delete("b")
    
    // no error even for no exist!!!
    s.Delete("c")
    fmt.Printf("set %v has 'a': %v\n", s, s.Has("a"))
    fmt.Printf("set %v has 'b': %v\n", s, s.Has("b"))
}

main()
set map[a:{}] has 'a': true
set map[a:{}] has 'b': false

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
package main

import "fmt"
import "time"

func worker(ch chan struct{}) {
    // send
    time.Sleep(time.Second * 1)
    fmt.Println("after 1s, sent notification, data is ready")
    ch <- struct{}{}
}

func main() {
    // channel buffer is zero
    // some one use ch := make(chan bool) but empty struct is better!!!
    ch := make(chan struct{})
    go worker(ch)

    fmt.Println("blocked due to no data")
    // read from channel, if no data block here
    <-ch
    fmt.Println("wake up after 1s as data is ready")
    close(ch)
}

main()
blocked due to no data
after 1s, sent notification, data is ready
wake up after 1s as data is ready

package

A package is a collection of source files in the same directory that are compiled together. Functions, types, variables, and constants defined in one source file are visible to all other source files within the same package.

one package per directory, you can NOT have multiple packages in same directory

create a runnable program
A standalone executable Go program must have package main declaration. If a program is part of the main package, then go build(go install) will create a binary file; which upon execution calls main function of the program, binary file is created only for man package

create a library
If a program is part of a package other than main, then a package archive file is created with go build(go install) command

Package declaration(package xxx at beginning of xx.go) which should be first line of code, file name can be different than package name. When you import a package, package declaration is used to create package reference variable.

Export name(var or method from a package)
A name is exported if it begins with a capital letter, exported means it can be accessed from other package.

package scope
A package scope is a region within a package where a declared variable(even it’s not exported) is accessible from within a package (across all the files in the package).

package init()
func init(){} is called by Go when a package is initialized. It does not take any arguments and does not return any value, hence func init(){} is a special function of xx.go file should be only one for a package.

Package alias
When you import a package, Go creates a variable using the package declaration of the package. If you are importing multiple packages with the same name, this will lead to a conflict, use alias to avoid conflict if happens.

1
2
3
import (
    log "fmt" // log is alias for fmt package.
)

Publish your package
Publish it on GitHub and you are good to go. If your package is executable, people can use it as a command-line tool else they can import it in a program and use it as a utility module.

inside import statement

1
2
3
4
import github.com/example/hello
// github.com/example is a path!!!
// hello is also a path under example/
// as you know package name can be different with dir who contains it, but most of time it's same.

Above statement essentially means that import package present at directory hello. It doesn’t mean import package hello, it import package under hello/, that also means package name can be different with its directory.

Note

  • Go does NOT allow multiple packages at same directory
  • import is not recursive, if you have packages under subdirectory, you should import that subdirectory as well.
  • dot import: If an explicit period (.) appears instead of a name, all the package’s exported identifiers declared in that package’s package block will be declared in the importing source file’s file block and must be accessed without a qualifier. import . "fmt", then use Println("hello"). but it’s not good way.

searching package

Path of searching packages depends on GO111MODULE is enabled or not but both way check standard library firstly

$GOROOT=/usr/local/go for standard library like fmt, path, cmd, buffio etc.
$GOPATH=/home/go for third-party library.

GO111MODULE=on you must have go.mod in your project to build and run!!!

  • $CURRENT_DIR/vendor is NOT checked anymore!!!
  • $GOROOT/pkg/{arch}/xxx.a precompiled
  • $GOROOT/src standard library for source code
  • $GOPATH/pkg/mod/xxx workspace for source code

GO111MODULE=off

  • $CURRENT_DIR/vendor NOT $CURRENT_DIR/vendor/src!!!
  • $GOROOT/pkg/{arch}/xxx.a precompiled
  • $GOROOT/src standard library for source code
  • $GOPATH/pkg/{arch}/xxx.a precompiled
  • $GOPATH/src/xxx workspace for source code
  • Must put your project at $GOPATH/src to make it build

when GO111MODULE=off, go get would fetch all the sources by using their import paths and store them in $GOPATH/src. There was no versioning storing a single git checkout of every package and the ‘master’ branch would represent a stable version of the package.

Go Modules (GO111MODULE=on) were introduced with Go 1.11, Go Modules stores tagged versions with go.mod keeping track of each package’s version

  • manually run GO111MODULE=on go get would fetch all the sources with tagged versions and saved it at $GOPATH/pkg/mod/
  • automatically run go get when you run go build or go install based on tagged version from go.mod of each module,you must have go.mod of each module

import package

import statement imports the package under that path, as one package per directory, hence only one package is imported for the path, most of time ,for easy to use, path and package name are same, but the path and package name can be different, if they are different, you need to know both package path and package name, while if they are same, you just need to know one, details refer to package and folder name

go env

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# check all env of go
$ go env
GOENV="/home/ubuntu/.config/go/env"
GO111MODULE="on"
GOBIN=""
GOMODCACHE="/home/ubuntu/go/pkg/mod"
GOPATH="/home/ubuntu/go"
GOROOT="/usr/local/go"
GOPROXY="https://goproxy.io,direct"
...
# set env
$ vi /home/ubuntu/.config/go/env
GOBIN=""
$ go env -w GOBIN=""

frequently used env

  • $GOBIN: bin dir of workspace which stores binary for application after go install, default $GOPATH/bin
  • $GOMODCACHE: mod(module) cache source code(xx.go) if mod is not standard library when GO111MODULE is on(mod with version), default $GOPATH/pkg/mod.
  • $GOPATH: working path, has bin/, src/, mod/, src is used to store download non-standard library(without version)
  • $GOROOT: Standard library of Go

workspace

A workspace is Go’s way to facilitate project management. A workspace, in a nutshell, is a directory on your system where Go looks for source code files, manages dependency packages and build distribution binary files

A workspace can have multiple applications, if different apps refer to same package, they share the same package files at this workspace.

You can have as many workspaces as you want, as long as you keep GOPATH environment variable pointed to the current working workspace directory.

A Go workspace directory must have three sub-directories src, pkg and bin, $GOPATH points to active workspace.

  • pkg:

    • The pkg directory contains Go package objects(get by go get). They are the compiled versions of the original package source code or source code at pkg/mod for GO111MODULE enabled.

  • bin:

    • The bin directory contains the binary executable files. These files are created by go install commands. go install command runs go build command internally and then outputs these files to the bin directory

  • src:

    • The src directory contains Go packages. A package in nutshell is a project directory containing Go source code (.go files). Any packages installed using GO111MODULE=off go get command will reside here as well (and its dependency packages).

module

Go code is grouped into packages, and packages are grouped into modules, a module can have several related packages but not at same directory, a module is logical groups to track dependencies of all packages in go.mod file, a module only needs one go.mod file at root directory, subdirectory does not need it all.

Go must provide all of their dependencies via either Go modules with a go.mod file, or a vendor directory, go.mod is created with go mod init example.com/greetngs and updated when run go mod tidy, go.mod only tracks the deps(write a record in it), the downloaded modules is saved at $GOMODCACHE.

In GO111MODULE=off, if a package or a parent folder(parent's parent ...) of a package contains folder named vendor it will be searched for dependencies using the vendor folder as an import path root. While vendor folders can be nested, in most cases it is not advised or needed. when GO111MODULE=off Any package in the vendor folder will be found before the standard library.

go.mod

useful command used within a module 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# initialize new module in current directory
# use example.com/greetings. 
# If you publish a module, this must be a path from which your module can be downloaded by Go tools. Mostly it's your code's repository.

$ go mod init example.com/greetings

# edit go.mod from command line
$ go mod edit


# edit source code, import packages etc, then run go mod tidy
# go mod tidy ensures that the go.mod file matches the source code in the module. It adds any missing module requirements necessary to build the current module's packages and dependencies, and it removes requirements on modules that don't provide any relevant packages. It also adds any missing entries to go.sum and removes unnecessary entries.

# add missing and remove unused modules, download package if not found locally
$ go mod tidy

# show all import information
$ go list -m -json all

project layout 

1
2
3
4
5
6
7
|-- prj
|   |-- greetings
|   |   |-- go.mod
|   |   `-- greetings.go
|   `-- hello
|       |-- go.mod
|       `-- hello.go

creating a module(library) used by others

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
$ mkdir greetings
$ cd greetings
$ go mod init example.com/greetings
go: creating new go.mod: module example.com/greetings

# go.mod file to track your code's dependencies, it's updated automatically.

# vi greatings.go

package greetings

import "fmt"

// Hello returns a greeting for the named person.
func Hello(name string) string {
    // Return a greeting that embeds the name in a message.
    message := fmt.Sprintf("Hi, %v. Welcome!", name)
    return message
}

# optional
$ go tool compile -pack greetings.go
greetings.a

$ go tool compile greetings.go
greetings.o

creating a module(runnable application)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
$ mkdir hello
$ cd hello
$ go mod init example.com/hello
go: creating new go.mod: module example.com/hello

# vi hello.go

package main

import (
    "fmt"

    "example.com/greetings" //By convention, the package name is the same as the last element of the import path
)

func main() {
    // Get a greeting message and print it.
    message := greetings.Hello("Gladys")
    fmt.Println(message)
}

# For production use, you’d publish the example.com/greetings module from its repository, go tools will download it from there

# For now, because you haven't published the module yet, you need to adapt the example.com/hello module so it can find the example.com/greetings code on your local file system.

# use the go mod edit command to edit the example.com/hello module to redirect Go tools from its module path (where the module isn't) to the local directory (where it is)

$ go mod edit -replace=example.com/greetings=../greetings
# add dependency in go.mod
$ go mod tidy

$ go build & ./hello  // build run, binary is written to disk
$ go run .           // build, run without writing binary to disk
$ go install & hello // build, install, and run

Importing packages from your module

1
2
3
4
5
6
7
8
9
10
11
12
13
14
(base) [root@centos go]# tree
.
|-- go.mod
|-- greet
|   `-- greet.go
`-- hello.go

1 directory, 3 files
(base) [root@centos go]# cat go.mod 
module example.com/hello

go 1.15

# module path: example.com/hello
1
2
3
4
5
6
7
8
9
package main

import (
    "example.com/hello/greet" // import this way: module_path/greet
)

func main() {
        greet.Say("hello")
}

vendor

It is a folder found in a module that stores a copy of all the code the module depends on. The code is used to compile the final executable when the go build command is run with GO111MODULE=off.

By default, there is no vendor folder at all, but you can create it with go mod vendor or govendor govendor tool, after this all deps are copied to vendor fold, that means you can build your project without downloading deps if you switch to another machine or the deps are deleted from Internet, you have a total copy of it.

It’s old way, should not use it anymore, refer to migrate to go mod to update your project.

exception

No exception in Go like python or C++, library should return a value and err if want caller check error.

In Go, there is a built-in error type which defines like this

1
2
3
type error interface {
    Error() string
}

So that any type who satisfies this interface implements Error() method can be used as error.

  • fmt.Errorf("error %d", 10) returns struct instance which implements such method
  • The fmt package formats an error value by calling its Errorf() method.
1
2
3
4
5
6
func great(a, b int)(int, error) {                                              
    if a > b {                                                                  
        return a, nil                                                           
    }                                                                           
    return b, fmt.Errorf(\"%v is not great than %v\", a, b)                     
}

recover from panic

defer function is called even panic happens.

When panic is called, including implicitly for run-time errors such as indexing a slice out of bounds or failing a type assertion, it immediately stops execution of the current function and begins unwinding the stack of the goroutine, running any deferred functions along the way. If that unwinding reaches the top of the goroutine's stack, the program dies.

A call to recover() stops the unwinding and returns the argument passed to panic. Because the only code that runs while unwinding is inside deferred functions, recover is only useful inside deferred functions.

One application of recover is to shut down a failing goroutine inside a server without killing the other executing goroutines.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
import "fmt"

func server() {
    works := [10]int{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
    
    for i := range works {
        go safelyDo(i)
    }
}

func do(i int) {
    if i % 2 == 0 {
        panic("panic when processing work")
    } else {
        fmt.Println("work is done")
    }
}


func safelyDo(i int) {
    defer func() {
        if err := recover(); err != nil {
            fmt.Println("work failed:", err)
        }
    }()
    do(i)
}

server()
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
package number

import "fmt"

func add(a, b int) (int, error) {
    if a < 0 || b < 0 {
        return 0, fmt.Errorf("not support negative adding")
    }
    
    return a + b, nil
}

func test() {
    v, err := add(1, 2)
    if err != nil {
        fmt.Println(err)
    } else {
        fmt.Println("1+2=", v)
    }
    
    v, err = add(-1, 2)
    if err != nil {
        fmt.Println(err)
    } else {
       fmt.Println("-1+2=", v)
    }
}
test()
1+2= 3
not support negative adding

cgo(call C in Go)

Cgo lets Go packages call C code. In order to use C code in Go, you first need to import a “pseudo-package”, “C” a special name interpreted by cgo as a reference to C’s name space, and include headers needed by C code when compiling with fixed format.

If the import of "C" is immediately preceded by a comment, that comment, called the preamble, is used as a header when compiling the C parts of the package by gcc!, The preamble may contain any C code, including function and variable declarations and definitions. These may then be referred to from Go code as though they were defined in the package “C”. All names declared in the preamble may be used, even if they start with a lower-case letter.

NOTE
CFLAGS, CPPFLAGS, CXXFLAGS, FFLAGS and LDFLAGS may be defined with pseudo #cgo directives within these comments to tweak the behavior of the C, C++ or Fortran compiler.

Note: No space line between import “C” and its header comment

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
/*
#include <stdio.h>
#include <xxx.h>
#include <xxx.c> 
*/
import "C"

// OR if needs to link other library or set FLAG

// #cgo CFLAGS: -DPNG_DEBUG=1
// #cgo linux CFLAGS: -DLINUX=1
// #cgo LDFLAGS: -lpng
// #include <stdio.h>
// #include <png.h>
import "C"

type mapping between C and go

The standard C numeric types are available under the names

  • C.char, C.schar (signed char), C.uchar (unsigned char)
  • C.short, C.ushort (unsigned short)
  • C.int, C.uint (unsigned int)
  • C.long, C.ulong (unsigned long), C.longlong (long long), C.ulonglong (unsigned long long)
  • C.float, C.double
  • The C type void is represented by Go’s unsafe.Pointer.*
  • The C types __int128_t and __uint128_t are represented by [16]byte.
  • C.struct_person{} refer to struct person{} defined in C
  • C.sizeof_struct_person get the len of struct person{} defined in C

Access struct

  • To access a struct, union, or enum type directly, prefix it with struct_, union_, or enum_, as in C.struct_stat.

sizeof

  • The size of any C type T is available as C.sizeof_T, like C.sizeof_struct_stat == sizeof(struct stat)

pass Go array to C function

  • n, err := C.f(&array[0]) // pass address of the first element.

Memory allocations made by C code are unknown to Go’s memory manager.

  • When you create a C string with C.CString (or any C memory allocation) you must remember to free the memory when you’re done with it by calling C.free.

write C within go file

1
2
3
4
5
6
7
8
9
10
11
12
package main

// #include <stdio.h>
//
// void hello() {
//     printf("hello\n");
// }
import "C"

func main() {
    C.hello()
}

write C out of go, compile as library, go import the library

sum.c

1
2
3
4
5
6
#include <stdio.h>

int sum(int a, int b, char* msg) {
    printf("say: '%s' adding\n", msg);
    return a + b;
}

sum.h

1
int sum(int a, int b, char* msg);

Then create shared library

1
2
3
4
$ gcc -fPIC -c sum.c
$ gcc -shared -o libsum.so sum.o
$ ls
libsum.so  sum.c  sum.h  sum.o  test.go

test.go

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
package main

// #cgo CFLAGS: -I./
// #cgo LDFLAGS: -L./ -lsum
// #include "sum.h"
// #include <stdlib.h>
import "C"

import (
    "fmt"
    "unsafe"
)

func main() {
    C.hello()

    a := C.int(1)
    b := C.int(2)
    // create memory without go track
    msg := C.CString("starting")
    c := C.sum(a, b, msg)
    // free it.
    C.free(unsafe.Pointer(msg)) //defined in stdlib.h

    fmt.Printf("sum(1,2)=%v\n", int(c))
}
1
2
3
4
$ go run test.go

say: '' adding
sum(1,2)=3

cgo guideline

Runtime

func NumCPU() int

NumCPU returns the number of logical CPUs usable by the current process.
The set of available CPUs is checked by querying the operating system at process startup. Changes to operating system CPU allocation after process startup are not reflected.

func GOMAXPROCS(n int) int

GOMAXPROCS sets the maximum number of CPUs that can be executing simultaneously and returns the previous setting. It defaults to the value of runtime.NumCPU. If n < 1, it does not change the current setting. This call will go away when the scheduler improves.

func Gosched()

Gosched yields the processor, allowing other goroutines to run. It does not suspend the current goroutine, so execution resumes automatically.
runtime library

concurrency(multiple threads)

The channel introduces many use cases in which channels are used to do data synchronizations among goroutines. In fact, channels are not the only synchronization techniques provided in Go. There are some other synchronization techniques supported by Go. For some specified circumstances, using the synchronization techniques other than channel are more efficient and readable than using channels.

ways to use channel, think chan bool as type

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
- var m chan bool // bool channel
- var m map[string] chan bool // each element of map is a bool channel.
- var m chan int // int channel
- var m []chan int // each element of the array is an int channel
- func gen(nums []int) <-chan int {} // reading channel as return value
- func gen(c chan int) int {} // channel as parameter
- var ch2 chan<- int// channel only for writing
- var ch3 <-chan int // channel only for reading

- ch4 := make(chan int)
- ch5 := <-chan int(ch4) // convert to unary channel for reading
- ch6 := chan<- int(ch4) // convert to unary channel for writing

// return a channel for reading
func gen(nums []int) <-chan int {
}

type Request struct {
    args        []int
    f           func([]int) int
    resultChan  chan int // channel var, need to initialize by make(chan int) later on
}

A goroutine is a lightweight 'thread' managed by the Go runtime. by default GO creates a pool of linux threads, the number of this pool equals to number of processor, goroutine runs on these threads.

go f(x, y, z)

The evaluation of f, x, y, and z happens in the current goroutine and the execution of f happens in the new goroutine.

goroutines run in the same address space, so access to shared memory must be synchronized. The sync package provides useful primitives.

communication between goroutines

Channels are a typed conduit through which you can send and receive values with the channel operator, <-

1
2
3
4
5
6
7
8
9
ch = make(chan int) // create a channel with buffer size 0, no buffer, which only access int as its message

// <- is an operator, NO operator ->!!!
// the left side is receiver, the right side is sender

ch <- v        // Send v to channel ch. blocked until data is read!!!
v := <- ch     // Receive from ch, and assign value to v blocked until data is sent!!!
v, ok := <- ch // check if channel is close, when closed, v is nil, ok is false, otherwise, v is value, ok is true
<- ch          // discard result

By default, send and receive block until the other side is ready. This allows goroutines to synchronize without explicit locks or condition variables.

Receivers always block until there is data to receive. If the channel is unbuffered(buffer size 0), the sender blocks until the receiver has received the value. If the channel has a buffer, the sender blocks only until if the buffer is full, this means waiting until some receiver has retrieved a value.

1
ch = make(chan int, 10) // buffer is 10, if no one receives, the 11th sending blocks!!!

Sends to a buffered channel block only when the buffer is full. Receives block when the buffer is empty

A sender can close a channel to indicate that no more values will be sent. Receivers can test whether a channel has been closed by assigning a second parameter to the receive expression: after
v, ok := <- ch

ok is false if there are no more values to receive and the channel is closed, ok is true if more values to receive, even it’s closed!!!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
package main

import "fmt"

func main() {
    c := make(chan int, 5)

    c <- 3
    c <- 4
    c <- 5

    d, ok := <-c
    fmt.Println(d, ok)
    close(c)
    d, ok = <-c
    fmt.Println(d, ok)
}

........
3 true
4 true

The loop for i := range c receives values from the channel repeatedly until it is closed.

Note:

  • Only the sender should close a channel, never the receiver
  • Sending on a closed channel will cause a panic.
  • Reading on closed channel, error happens, but not panic!!
  • For zero buffer channel, send and receive block until the other side is ready
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
package main

import (
    "fmt"
)
// chan int, type of channel
func fibonacci(n int, c chan int) {
    x, y := 0, 1
    for i := 0; i < n; i++ {
        c <- x // send x to channel
        x, y = y, x+y
    }
    close(c) // close the channel
}

func main() {
    c := make(chan int, 10)

    // pass channel to another routine
    go fibonacci(cap(c), c)
    for i := range c {
        // receives from channel until it's closed, blocked if no more data in the channel
        fmt.Println(i)
    }
}
main()
0
1
1
2
3
5
8
13
21
34

multiple channels

The select statement lets a goroutine wait on multiple communication operations.

A select blocks until one of its cases can run, then it executes that case. the default case in a select runs if no other case is ready(no event on channel), select only works for channel, not socket fd, if you want to monitor multiple fds for high performance use gnet

  • if no default case, select quits until one channel is ready!!! otherwise block for ever, so it's one time execution, if you need to select more time, put it in for loop!!!

It chooses one at random if multiple channels are ready

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
package main

import "fmt"

func fibonacci(c chan int, quit chan struct{}) {
    x, y := 0, 1
    for { // infinite loop
        select {
        // x is the result written to channel
        case c <- x:
            // go down when write to channel is done, wake from block.
            // go down only when x is read by reader as it's unbuffered channel
            x, y = y, x+y
    
        case _, ok:= <-quit: // run this case only when channel is ready successfully!!!
            fmt.Println(ok)
            return
        }
    }
}

func main() {
    c := make(chan int)
    quit := make(chan struct{})
    go func() {
        for i := 0; i < 10; i++ {
            v := <-c // full format v, ok: = <-c
            fmt.Println(v)
        }
        quit <- struct{}{} //write empty to signal channel quit
    }() // here call the unnamed function

    fibonacci(c, quit)
}
main()
0
1
1
2
3
5
8
13
21
34
true

Mutex

What we just want to make sure only one goroutine can access a shared variable at a time to avoid conflicts?

This concept is called mutual exclusion, and the conventional name for the data structure that provides it is mutex.

Go’s standard library provides mutual exclusion with sync.Mutex and its two methods:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
import "sync"
var mu sync.Mutex
mu.Lock()
...
mu.UnLock()


var mu sync.RWMutex
mu.RLock() // reader
...
mu.RUnLock()

mu.Lock() // writer
...
mu.UnLock()

We can define a block of code to be executed in mutual exclusion by surrounding it with a call to Lock and Unlock.

We can also use defer to ensure the mutex will be unlocked as in the Value method.

mutex is not associated with particular goroutine, it’s global and can be accessed by all goroutines, locked when the mutex lock bit is set.

NOTE

  • nested is not supported
    1
    2
    3
    m := sync.Mutex{}
    m.Lock()
    m.Lock()// block for ever here!!!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
package main

import (
    "fmt"
    "sync"
    "time"
)

func main() {
    var m sync.Mutex
    m.Lock()
    go func() {
        time.Sleep(time.Second)
        fmt.Println("Hi")
        m.Unlock() // make a notification
    }()
    // if m.Lock runs before m.Unlock in gorountine, it blocks as it's alrealdy locked above
    // as lock on unlocked mutext will block!!!
    m.Lock() // wait to be notified
    fmt.Println("Bye") // Byes is always after Hi printed!!!
}

main()
Hi
Bye

WaitGroup

Each sync.WaitGroup value maintains a counter internally. The initial value of the counter is zero.

The *WaitGroup type has three methods: Add(delta int), Done() and Wait().

  • we can call the wg.Add(delta) method to change the counter value maintained by wg.
  • the method call wg.Done() is totally equivalent to the method call wg.Add(-1).
  • if a call wg.Add(delta) (or wg.Done()) modifies the counter maintained by wg to negative, panic will happen.
  • when a goroutine calls wg.Wait(),
    • if the counter maintained by wg is already zero, then the call wg.Wait() can be viewed as a no-op.
    • otherwise (the counter is positive), the goroutine will enter blocking state. It will enter running state again (a.k.a., the call wg.Wait() returns) when another goroutine modifies the counter to zero, generally by calling wg.Done().

Generally, a WaitGroup value is used for the scenario that one goroutine waits until all of several other goroutines finish their respective jobs.

Note

  • The Wait() method can be called in multiple goroutines. When the counter becomes zero, all of them will be notified, in a broadcast way.

  • A WaitGroup value can be reused after one call to its Wait method returns. But please note that each Add method call with a positive delta that occurs when the counter is zero must happen before any Wait call starts, otherwise, data races may happen.

  • Must call Add() in main goroutine not the one runs the job.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
package main

import (
    "fmt"
    "math/rand"
    "sync"
    "time"
)

func main() {
    // set seed
    rand.Seed(time.Now().UnixNano())

    const N = 5
    var values [N]int32

    var wg sync.WaitGroup
    
    wg.Add(N)
    for i := 0; i < N; i++ {
        i := i
        go func() {
            // Int31n returns, as an int32, a non-negative pseudo-random number in the half-open interval [0,n) 
            // from the default Source. It panics if n <= 0. 
            values[i] = 50 + rand.Int31n(50) //
            fmt.Printf("Done:%v\n", i)
            wg.Done() // <=> wg.Add(-1)
        }()
    }

    wg.Wait()
    // All elements are guaranteed to be
    // initialized now.
    fmt.Printf("values:%v\n", values)
}
main()
Done:4
Done:2
Done:1
Done:3
Done:0
values:[63 93 53 59 60]

Once

A *sync.Once value has a Do(f func()) method, which takes a solo parameter with type func().

The code in the invoked argument function(doSomething()) is guaranteed to be executed before any once.Do() method call returns.
Generally, a Once value is used to ensure that a piece of code will be executed exactly once in concurrent programming.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
package main

import (
    "fmt"
    "log"
    "sync"
)

func main() {
    x := 0
    doSomething := func() {
        // only runs once!!!
        x++
        fmt.Println("Hello in once")
    }

    var wg sync.WaitGroup
    var once sync.Once
    for i := 0; i < 5; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            //Hello is guaranteed to be printed before all five world!!!
            once.Do(doSomething)
            
            // all below code runs after soSomething return
            fmt.Println("world!")
        }()
    }

    wg.Wait()
    fmt.Printf("x = %v\n", x) // x = 1
}
main()
Hello in once
world!
world!
world!
world!
world!
x = 1

Cond

The sync.Cond type provides an efficient way to do notifications among goroutines.

Each sync.Cond value holds a sync.Locker field with name L. The field value is often a value of type *sync.Mutex or *sync.RWMutex. So, in order to use Cron, you must have a mutex as well!!!

  • c.Wait() must be called when c.L is locked, otherwise, a c.Wait() will cause panic. A c.Wait() call will first push the current caller goroutine into the waiting goroutine queue maintained by c, then call c.L.Unlock() to unlock/unhold the lock c.L. then make the current caller goroutine enter blocking state. Once the caller goroutine is unblocked and enters running state again, c.L.Lock() will be called (in the resumed c.Wait() call) to try to lock and hold the lock c.L again, The c.Wait() call will exit after the c.L.Lock() call returns.

  • a c.Signal() call will unblock the first goroutine in (and remove it from) the waiting goroutine queue maintained by c, if the queue is not empty.

  • a c.Broadcast() call will unblock all the goroutines in (and remove them from) the waiting goroutine queue maintained by c, if the queue is not empty.

cond.Broadcast() and cond.Signal() are not required to be called when cond.L is locked. you can also call them after unlock.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
package main

import (
    "fmt"
    "math/rand"
    "sync"
    "time"
)

func main() {
    rand.Seed(time.Now().UnixNano())

    const N = 10
    var values [N]string

    // cond needs a mutex inside
    cond := sync.NewCond(&sync.Mutex{})

    for i := 0; i < N; i++ {
        d := time.Second * time.Duration(rand.Intn(5)) / 10
        go func(i int) {
            time.Sleep(d) // simulate a workload
            // Changes must be made when
            // cond.L is locked.
            cond.L.Lock()
            values[i] = string('a' + i)// 'a' is bytes(uint8!!!)
            //cond.Broadcast()
            cond.L.Unlock()

            // "cond.Broadcast()" can be put here
            // when cond.L lock is unlocked. so that the waked one can get lock right now.
            cond.Broadcast()
        }(i)
    }

    // This function must be called when
    // cond.L is locked.
    checkCondition := func() bool {
        fmt.Println(values)
        for i := 0; i < N; i++ {
            // we have the lock here.
            if values[i] == "" {
                // not all values are set.
                return false
            }
        }
        return true
    }

    cond.L.Lock()
    defer cond.L.Unlock()
    // not for loop.
    for !checkCondition() {
        // Must be called when cond.L is locked.
        cond.Wait() // when blockes, mutex is unlocked!!! when waked up, it gets lock again!!!
    }
}
main()
[         ]
[    e     ]
[ b   e     ]
[a b   e     ]
[a b c  e     ]
[a b c  e   h  ]
[a b c  e f  h  ]
[a b c d e f  h  ]
[a b c d e f  h  j]
[a b c d e f g h  j]
[a b c d e f g h i j]

atomic

go provides atomic operations Add、CompareAndSwap、Load、Store、Swap from "sync/atomic"

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
func AddInt32(addr *int32, delta int32) (new int32)
func AddUint32(addr *uint32, delta uint32) (new uint32)
func AddInt64(addr *int64, delta int64) (new int64)
func AddUint64(addr *uint64, delta uint64) (new uint64)
func AddUintptr(addr *uintptr, delta uintptr) (new uintptr)

func CompareAndSwapInt32(addr *int32, old, new int32) (swapped bool)
func CompareAndSwapInt64(addr *int64, old, new int64) (swapped bool)
func CompareAndSwapUint32(addr *uint32, old, new uint32) (swapped bool)
func CompareAndSwapUint64(addr *uint64, old, new uint64) (swapped bool)
func CompareAndSwapUintptr(addr *uintptr, old, new uintptr) (swapped bool)
func CompareAndSwapPointer(addr *unsafe.Pointer, old, new unsafe.Pointer) (swapped bool)

func LoadInt32(addr *int32) (val int32)
func LoadInt64(addr *int64) (val int64)
func LoadUint32(addr *uint32) (val uint32)
func LoadUint64(addr *uint64) (val uint64)
func LoadUintptr(addr *uintptr) (val uintptr)
func LoadPointer(addr *unsafe.Pointer) (val unsafe.Pointer)

func StoreInt32(addr *int32, val int32)
func StoreInt64(addr *int64, val int64)
func StoreUint32(addr *uint32, val uint32)
func StoreUint64(addr *uint64, val uint64)
func StoreUintptr(addr *uintptr, val uintptr)
func StorePointer(addr *unsafe.Pointer, val unsafe.Pointer)

func SwapInt32(addr *int32, new int32) (old int32)
func SwapInt64(addr *int64, new int64) (old int64)
func SwapUint32(addr *uint32, new uint32) (old uint32)
func SwapUint64(addr *uint64, new uint64) (old uint64)
func SwapUintptr(addr *uintptr, new uintptr) (old uintptr)
func SwapPointer(addr *unsafe.Pointer, new unsafe.Pointer) (old unsafe.Pointer)

reflection

from specific type to generic type

you can convert any type to interface {} just for parameter passing.

1
2
3
4
5
6
7
8
9
var a int = 10
var b *int = &a
var c string = "hello"

var i interface{}
i = a
i = b
i = c
// all are ok!!!

from interface to specific type

As any type can be converted to interface{} type, but how can we convert it back to particular type.

A type assertion provides access to an interface value’s underlying concrete value.

t := i.(T) If i does not hold a T, the statement will trigger a panic.

This statement asserts that the interface value i holds the concrete type T and assigns the underlying T value to the variable t.

To test whether an interface value holds a specific type, a type assertion can return two values: the underlying value and a boolean value that reports whether the assertion succeeded.

t, ok := i.(T)

If i holds a T(i is an instance of T), then t will be the underlying value and ok will be true.
If not, ok will be false and t will be the zero value of type T, and no panic occurs.

T is generic type, it can be int or *int(*int is also a type)

generic value, v, ok := i.(T) // i is var of interface{}

1
2
3
4
5
6
7
8
9
if v, ok := i.(int); ok {
    fmt.Printf("Twice %d is %d\n", v, v*2)
} else if v, ok := i.(*int); ok{
    fmt.Printf("Twice %d is %d\n", *v, *v*2)
} else if v, ok = i.(string); ok {
    fmt.Printf("%s is %d bytes long\n", v, len(v))
} else {
   fmt.Printf("I don't know about type %T!\n", v)
}

type switch(only valid for switch)
A type switch is like a regular switch statement, but the cases in a type switch specify types (not values), and those values are compared against the type of the value held by the given interface value.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
func do(i interface{}) {
    switch v := i.(type) { // i.(type), type is keyword here!!!
	case int:
		fmt.Printf("Twice %v is %v\n", v, v*2) // v is type int
	case string:
		fmt.Printf("%q is %v bytes long\n", v, len(v)) // v is type string
	default:
		fmt.Printf("I don't know about type %T!\n", v)
	}
}

func test() {
	do(21)
	do("hello")
	do(true)
}

Advanced feature between interface and specfic type

you can convert any type to interface {} just for parameter passing

but as interface {} has no method, hence you can NOT get more information about the type/value it behinds, so refection package gives advanced feature.

reflect implements run-time reflection, allowing a program to manipulate objects with arbitrary types. The typical use is to take a value with static type interface{} and extract its dynamic type information by calling TypeOf(), which returns a Type(an interface defined in reflect package).

A call to ValueOf() returns a Value(an interface defined in reflect package) representing the run-time data. Zero takes a Type and returns a Value representing a zero value for that type.

  • reflect.Type(type interface {}) returned by reflect.Typeof(any instance)

    • rType implements generic methods defined by reflect.Type
    • mapType, ptrType, structType(which embed rType) etc implement specific methods defined reflect.Type which are valid for its type, so specific type implements all methods defined by reflect.Type

  • reflect.Value(type Value struct {}) returned by reflect.Valueof(any instance)

    • Value has generic methods called by all values
    • Value has specific methods which are valid for specific type

Method of specific type

Kind Methods applicable
Int* Bits
Uint* Bits
Float* Bits
Complex* Bits
Array Elem, Len
Chan ChanDir, Elem
Func In, NumIn, Out, NumOut, IsVariadic
Map Key, Elem
Ptr Elem
Slice Elem
Struct Field, FieldByIndex, FieldByName,FieldByNameFunc, NumField

  • Type.Elem() returns a type’s element type.

  • Type.PtrTo() returns the pointer type with element t.

    • For example, if t represents type Foo, PtrTo(t) represents *Foo.

  • v.Indirect() returns the value that v points to(v is pointer Value)

    • v := reflect.Valueof(&Person{Name: "tom"}), vv := v.Indirect()

  • v.Addr() returns the pointer’s Value (v is object Value)

    • v := reflect.Valueof(Person{Name: "tom"}), vp: = v.Addr()

reflection rule

Ref 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
package main

import (
    "fmt"
    "reflect"
    "strconv"
)

type Person struct {
    Name string `k1:"v1" k2:"v2"`
    id   int
}

func (p *Person) Ident() string {
    return p.Name + " " + strconv.Itoa(p.id)
}

func (p Person) Say(msg string) string {
    return msg + p.Name
}

func main() {
    p := Person{Name: "tom", id: 10}
    
    
    /*********************************************************************/
    // typeof from interface
    rp := reflect.TypeOf(p)
    // rp.Elem() returns a type's element type.
    fmt.Printf(`p := Person{}
type :%v
size(byte): %v
numField: %v
numMethod: %v
align: %v
fieldalign: %v
`, rp.String(), rp.Size(), rp.NumField(), rp.NumMethod(), rp.Align(), rp.FieldAlign())
    // NOTE: Ident is not a method of Person
    // Go compiler auto converted p.Ident() to (&p).Ident()
    fmt.Println(p.Ident(), (&p).Ident()) //

    m, ok := rp.MethodByName("Say")
    if ok {
        fmt.Printf("method name: %v, type: %v, index: %v\n", m.Name, m.Type, m.Index)
    } else {
        fmt.Printf("Say method not found\n")
    }

    f, _ := rp.FieldByName("Name")
    fmt.Printf("field name: %v, type: %v, Tag: %v\n", f.Name, f.Type, f.Tag)
    // Type to Value: nv := reflect.New(rp)
    // if rp(map):    nv := reflect.MakeMap(rp)
    
    
    /*********************************************************************/
    // ValueOf from interface
    v := reflect.ValueOf(p)
    // Value to Type: rp := v.Type()
    rp = v.Type()
    fmt.Printf(`p := Person{}
type :%v
size(byte): %v
numField: %v
numMethod: %v
align: %v
fieldalign: %v
`, rp.String(), rp.Size(), rp.NumField(), rp.NumMethod(), rp.Align(), rp.FieldAlign())

    vName := v.FieldByName("Name")
    name := vName.String()
    // we know it's string has String() method, 
    // otherwise name := vName.Interface().(*NType)
    
    fmt.Println("name=", name)
    vSay := v.MethodByName("Say")
    if vSay.IsValid() {
        Say := vSay.Interface().(func(string) string)
        fmt.Println(Say("hello "))
    } else {
        fmt.Println("method Say not found")
    }
    
    /*******************************************************************************/
    // back to Person struct
    p = v.Interface().(Person)
    // both print Person, but v.Interface() can' not access field as it's interface!!!
    fmt.Println(p, v.Interface())
}
main()
p := Person{}
type :struct { Name string "k1:\"v1\" k2:\"v2\""; 𒀸id int }
size(byte): 24
numField: 2
numMethod: 0
align: 8
fieldalign: 8
tom 10 tom 10
Say method not found
field name: Name, type: string, Tag: k1:"v1" k2:"v2"
p := Person{}
type :struct { Name string "k1:\"v1\" k2:\"v2\""; 𒀸id int }
size(byte): 24
numField: 2
numMethod: 0
align: 8
fieldalign: 8
name= tom
method Say not found
{tom 10} {tom 10}

import cycle

Cyclic dependency is fundamentally a bad design and is a compile-time error in Golang(error: import cycle not allowed). we should change our design to solve this in either way.

Solve import cycle

  • put them in the same package
  • use interface to solve the import cycle issue
1
2
3
4
5
6
7
8
dep
|-- a
|   `-- a.go
|-- b
|   `-- b.go
|-- c
|   `-- c.go
`-- main.go
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
package a

import (
  "fmt"

  "github.com/mantishK/dep/b"
)

type A struct {
}

func (a A) PrintA() {
  fmt.Println(a)
}

func NewA() *A {
  a := new(A)
  return a
}

func RequireB() {
  o := b.NewB()
  o.PrintB()
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
package b

import (
  "fmt"

  "github.com/mantishK/dep/a"
)

type B struct {
}

func (b B) PrintB() {
  fmt.Println(b)
}

func NewB() *B {
  b := new(B)
  return b
}

func RequireA() {
  o := a.NewA()
  o.PrintA()// B wants to PrintA from A
}
1
2
3
4
5
6
7
8
9
10
package main

import (
    "github.com/mantishK/dep/a"
)

func main() {
    o := a.NewA()
    o.PrintA()
}

Soloution
we must introduce an interface in a new package say c. This interface will have all the methods that are in struct A and are accessed by struct B.

1
2
3
4
5
package c

type C interface {
    PrintA()
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
package a

import (
    "fmt"

    "github.com/mantishK/dep/b"
)

type A struct {
}

func (a A) PrintA() {
    fmt.Println("hello a")
}

func NewA() *A {
    a := new(A)
    return a
}

func (a A) PrintBFromA() {
    o := b.NewB(a)
    o.PrintB()
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
package b

import (
    "fmt"

    "github.com/mantishK/dep/c"
)

type B struct {
    c c.C
}

func (b B) PrintB() {
    fmt.Println("hello b")
}

func NewB(c c.C) *B {
    // pass interface(A) to instance b
    b := new(B)
    b.c = c
    return b
}

func (b B) PrintAFromB() {
    // access PrintA
    b.c.PrintA()
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
package main

import (
    "github.com/mantishK/dep/a"
    "github.com/mantishK/dep/b"
)

func main() {
    o := a.NewA()
    o.PrintA()
    o.PrintBFromA()

    b := b.NewB(o)
    b.PrintAFromB()
}

Deep copy

There is no deep copy built-in function provided by Go, if you want to deep copy, you have to copy it by your self, go only does shadow copy.

  • For channel, slice, dict, interface, pointer, assigned var points to same memory.
  • For struct, non-point, assigned var has its own memory, shadow copied.
  • Deep copy for map two ways.
    • Marshal –> then Unmarshal
    • iterate each item, then do copy, if item is a map, recursive deep is needed!!!
  • copy for slice, copy(dst, src), not recursive, not deep.

Go does the same thing as C language

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
a := []int{1,2,3}
b := a // b is copy of a, they points to same memory, same thing for map as well.
c := make([]int, len(a))
copy(c, a) // copy a to c, not support map, for map, you have to copy each by your self.

type Person struct {
    Name string
}

p1 := Person {Name: "Jack"}
p2 := p1 // p2 is copy of p1, they points to different memory.

ps := []Person{
    {
        Name:"Jack",
    },
}

for _, p := range ps {
    p.Name = "cool" // p is copy of array item.
}
// ps is not changed at all

Tips

Anonymous struct type

Most of time, if we frequently use a struct type, we should define a new type, use it for short like this

1
2
3
4
5
6
type Student struct {
    name string
    id int
}

var s1 Student = Student{"jason", 1}

But we still can use a unnamed struct if only use it for several times, like this

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import "fmt"

func test() {
    s1 := struct {
        name string
        id int
    }{"jason", 1}
    
    test1(s1)
    test2(&s1)
}

func test1(s struct {name string; id int}) {
     fmt.Println(s.name, s.id)
}
func test2(s *struct {name string; id int}) {
     fmt.Println(s.name, s.id)
}

Anonymous function

Anonymous function is a function without name, it’s mostly used in two cases

  • used as goruntine entry point
  • saved in function object, then call it later.
  • used as return value.
  • defer function.
1
2
3
4
5
6
7
8
9
10
11
func test {
    go func() {
        fmt.Println("hello go")
    }()
    
    
    f := func() {
        fmt.Println("hello go")
    }
    f()
}

Reader and Writer

The io package specifies the io.Reader interface, which represents the read end of a stream of data.

The Go standard library contains many implementations of this interface, including files, network connections, compressors, ciphers, and others.

The io.Reader interface has a Read method:

func (T) Read(b []byte) (n int, err error)

Read populates the given byte slice(should be create first with make()) with data and returns the number of bytes populated and an error value. It returns an io.EOF error when the stream ends.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
package main

import (
    "fmt"
    "io"
    "strings"
)

func main() {
    // return Reader interface
    r := strings.NewReader("Hello, Reader!")
    
    // buffer to store reading
    b := make([]byte, 8)
    for {
        n, err := r.Read(b) // must check err when reading
        fmt.Printf("n = %v err = %v b = %v\n", n, err, b)
        fmt.Printf("b[:n] = %q\n", b[:n])
        if err == io.EOF {
            break
        }
    }
}
main()
n = 8 err = <nil> b = [72 101 108 108 111 44 32 82]
b[:n] = "Hello, R"
n = 6 err = <nil> b = [101 97 100 101 114 33 32 82]
b[:n] = "eader!"
n = 0 err = EOF b = [101 97 100 101 114 33 32 82]
b[:n] = ""

never use break for each case in switch/select

As go break each matched case automatically, there is no need to use break for each case explicitly in case you want to break in the middle of the case.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
tick := time.NewTicker(2 * time.Second)                                         
var a int                                                                       
for {                                                                           
    select {                                                                    
    case <-tick.C:                                                              
        a++                                                                     
        if a > 2 {                                                              
            fmt.Println("break out")                                            
            break  // case is break, hence go next loop of for, for is not break!!!                                             
        }  
        
        fmt.Println("reach me")
    }                                                                                                                   
    fmt.Println("running")                                                 
}

Handle OS difference

Most of time, go code can shared by different OS like linux and windows etc, but in some case, we may need to handle special cases that depends on OS, hence we need to know the OS in Go code $GOOS gives the way for you to deal that.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
package main

import (
    "fmt"
    "runtime"
)

func main() {
    switch {
    case runtime.GOOS == "windows":
        fmt.Println("You are running on windows")
    case runtime.GOOS == "linux":
        fmt.Println("You are running on linux")
    default:
        fmt.Println("You are running on an OS that we do not support")
    }
}

HTTP over Unix

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
// A quick and dirty demo of talking HTTP over Unix domain sockets
package main

import (
    "fmt"
    "io"
    "net"
    "net/http"
    "os"
    "os/exec"
    "time"

    "github.com/codegangsta/martini"
)

// overriding ne.Dialer.Dial to force unix socket connection

var addr = "http.sock"

func fakeDial(_, _ string) (net.Conn, error) {
    return net.DialTimeout("unix", addr, time.Second*32)
}

var transport http.RoundTripper = &http.Transport{
    Dial: fakeDial,
}

func main() {
    m := martini.Classic()
    m.Get("/demo", handler)

    // normal unix listener
    listener, err := net.Listen("unix", addr)
    if err != nil {
        fmt.Println(err.Error())
        return
    }

    defer func() {
        cmd := exec.Command("/bin/rm", "-rf", "http.sock")
        cmd.Run()
    }()
    // as you can see server has no special setting for unix listener
    // serve http request on any listener(here unix listener)
    go http.Serve(listener, m) // http.Serve takes any net.Listener implementation

    time.Sleep(1 * time.Second)
    c := http.Client{}
    // client usage is different for tcp and unix transport!!!
    c.Transport = transport // use the unix dialer based on unix to send http request(no tcp as tcp+http)
    // as you can see the address is sock path not ip:port like tcp
    resp, err := c.Get("http://http.sock/demo")
    if err != nil {
        fmt.Println(err.Error())
        return
    }
    
    // io.copy from resp.Body to os.StdOut(Writer)
    io.Copy(os.Stdout, resp.Body)
}

func handler() (code int, body string) {
    code = 200
    body = "ok\n"
    return
}

WaitGroup vs Channel

Actually, there are designed for different scenarios, If you are dispatching one-off jobs to be run in parallel without needing to know the results of each job, then you can use a WaitGroup. But if you need to collect the results from the goroutines then you should use a channel. channel is designed to pass data while WaitGrop for wait jobs(which runs in goroutine) to finish.

waitgroup

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
package main

import (
    "fmt"
    "sync"
)

func work() {
    fmt.Println("job done")
}
func main() {
    var wg sync.WaitGroup

    wg.Add(1)
    go func() {
        defer wg.Done()
        work()
    }()

    wg.Wait()
    fmt.Println("got one job done")
}

notify channel

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
package main

import (
    "fmt"
)

func work(ch chan struct{}) {
    fmt.Println("job done")
    ch <- struct{}{}
}
func main() {
    ch := make(chan struct{})
    go func() {
        work(ch)
    }()

    <-ch
    fmt.Println("got one job done")
}

if you have several jobs to run, waitgroup is best choice to use!!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
package main

import (
    "fmt"
    "sync"
)

func work() {
    fmt.Println("job done")
}
func main() {
    var wg sync.WaitGroup

    wg.Add(2)
    go func() {
        defer wg.Done()
        work()
    }()

    go func() {
        defer wg.Done()
        work()
    }()

    wg.Wait()
    fmt.Println("got two jobs done")
}

main()
job done
job done
got two jobs done

do we need to close(channel) explicitly

No, It’s OK to leave a Go channel open forever and never close it. When the channel is no longer used(no read and write), it will be garbage collected.

It is only necessary to close a channel explicitly if the receiver is looking for a close. Closing the channel is a control signal on the channel indicating that no more data follows, or to notify goroutine to quit otherwise it may blocking for ever

Danger

  • closing a closed channel will panic, so it is dangerous to close a channel if the closers don’t know whether or not the channel is closed.
  • sending values to a closed channel will panic, so it is dangerous to send values to a channel if the senders don’t know whether or not the channel is closed.

Rule to close if required

  • don’t close a channel from the receiver side.
  • don’t close a channel if the channel has multiple concurrent senders.

In other words, we should only close a channel in a sender goroutine if the sender is the only sender of the channel. more solution to close a channel, refer to channel closing

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
package main

import (
    "fmt"
    "time"
)

var ch = make(chan struct{})

func test() {
    go func() {
        fmt.Println("goroutine blocks")
        <-ch
        fmt.Println("goroutine quit")
    }()

    fmt.Println("test return, but goroutine is blocking")
}
func main() {
    test()
    
    // if main is a daemon process, gorountine waits for signal while sender is quit
    // when sender quits, it should notify receive to quit as well, otherwise, it's blocks for ever
    
    time.Sleep(1 * time.Second)
    close(ch)
    time.Sleep(1 * time.Second)
}

main()
test return, but goroutine is blocking
goroutine blocks
goroutine quit

By default, struct pointer printed with its content if it’s not embeded, but if struct pointer as field of another struct, the pointer address is printed, As when call print, the String() of that type is called, but by default golang does not provide String() for struct pointer, but struct only, that’s why pointer address is printed, In order to print pinter content(not address), there are several ways we can use

  • iterate each pointer in struct, print it with fmt.Println("%s", *p)
  • with json.Marshel(), but only exported field are printed
  • For each pointer, implement its func (s *XX)String()string{} method
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
ackage main                                                                    
                                                                                
import (                                                                        
    "fmt"                                                                       
)                                                                               
                                                                                
type Class struct {                                                             
    Id int                                                                      
}                                                                               
                                                                                
// comment out to see what happens                                              
func (c *Class) String() string {                                               
    return fmt.Sprintf("%v", c.Id)                                              
}                                                                               
                                                                                
// comment out to see what happens                                              
func (p *Person) String() string {                                              
    return fmt.Sprintf("%v %v", p.Name, p.Cls)                                  
}                                                                               
                                                                                
type Person struct {                                                            
    Name string // string has default String()                                  
    Cls  *Class // no default String() for *Class                               
}                                                                               
                                                                                
func main() {                                                                   
    p1 := Person{Name: "tom", Cls: &Class{Id: 1}}                               
    p2 := Person{Name: "jack", Cls: &Class{Id: 2}}                              
    sp := []Person{p1, p2}                                                      
    fmt.Printf("%v\n", sp)                                                      
    fmt.Printf("%+v\n", sp)                                                     
                                                                                
    spp := []*Person{&p1, &p2} // Person has no default String()                
    fmt.Printf("%v\n", spp)                                                     
    fmt.Printf("%+v\n", spp)                                                    
}                      
1
2
3
4
[{tom 1} {jack 2}]
[{Name:tom Cls:1} {Name:jack Cls:2}]
[tom 1 jack 2]
[tom 1 jack 2]
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
package main

import (
    "encoding/json"
    "fmt"
)

type Class struct {
    Id int
}

type Person struct {
    Name string
    Cls  *Class
}

func main() {
    p1 := Person{Name: "tom", Cls: &Class{Id: 1}}
    p2 := Person{Name: "jack", Cls: &Class{Id: 2}}
    spp := []*Person{&p1, &p2}
    data, _ := json.Marshal(spp)
    fmt.Println(string(data))
}
main()
[{"Name":"tom","Cls":{"Id":1}},{"Name":"jack","Cls":{"Id":2}}]

block forever in go

1
2
3
4
5
6
7
8
9
10
11
12
13
14
func blockForever() {
    select{ }
}

func blockForever() {
    m := sync.Mutex{}
    m.Lock()
    m.Lock()
}

func blockForever() {
    c := make(chan struct{})
    <-c
}

block forever

Channel with timeout

Actually, there is no timeout parameter of API when reading/writing channel, so we use other way to do this.

Way1

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
func main() {
    ch := make(chan struct{}, 1)
    go func() {
        fmt.Println("do something...")
        time.Sleep(4*time.Second)
        ch<- struct{}{}
    }()
    
    select {
    case <-ch:
        fmt.Println("done")
    case <-time.After(3*time.Second):
        fmt.Println("timeout")
    }
}

Way2

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
ch := make(chan string)
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
defer cancel()
go func() {
    time.Sleep(time.Second * 4)

    ch <- "done"
}()

select {
case res := <-ch:
    fmt.Println(res)
case <-ctx.Done():
    fmt.Println("timout", ctx.Err())
}

Good reason to use context
Another advantage of using context is that it can take advantage of its natural transmission characteristics in multiple goroutines, so that all goroutines that pass the context can receive cancellation notifications at the same time. we can call cancel() once, while quits all gorotines who listen on it..

Ref