Golang 速查表 Cheatsheets
Golang
- Golang
- General
gb
- Go Tool
- import
- Scope
- Constants
- Variables
- Short Notation assignment for new variables
- Multiple Assignment
- Default values
- Bitwise operations 位运算符
- Types
- Basic Data Types
int
,float
,bool
string
- Unicode
- Iterate on symbols
- Iterate on bytes
- Composite Types
- Array
- Slice
- Map
- Control Flow
if
for
switch
- Function
- Anonymous Function
- Closure
- Recursive Functions
defer
panic()
init()
- Pointers
new
vs.make
- Heap vs. Stack
- Named basic types
- Struct (concrete type)
- Type methods
- Method Values and Expressions
- Interface (abstract type)
- Empty interface as a type
- Interface Value
- Interface with
nil
Pointer - Embed
- Error
- Type Assertion
- Switch Type Assertion
- Goroutines
- Overview
go run -race
go routine
& closure- Shadowing variable
- Channel
- Overview
- Buffered & unbuffered channels
- Read from channel w/
range
- Channel generator
- Sync go routines using channels
- Channel multiplexer
Select
- Packages
io
bytes
,strings
bufio
encoding
encoding/json
encoding/binary
strconv
fmt
http
context
net/rpc
net/http/httputil
html
sort
compress/gzip
ioutil
sync
time
runtime
flag
os
- Network
- TCP echo server (Async)
- HTTP Server
- JSON API Server
- Templates
text/template
html/template
- Inline
- Separate file
- Cookies
- DataBase
- MySQL
- PostgreSQL
- Memcached
- Redis
- MongoDB
- ORM
- Tests
- Simple
- Test HTTP handlers
- Mocks
- General
General
Binary file, that includes all dependencies.
GOPATH
├── bin/
├── pkg/
│ └── linux_amd64/
└── src/
├── github.com
│ └── oleglegun/
├── golang.org
└── gopkg.in
/*
Every package should have a package comment.
Package ... implements ...
*/
package main // executable main must contain function `main`
func main() {
println("Hello") // println works w/o any libs
}
// Do makes ...
func Do() {}
gb
Vendoring - save dependencies within repository.
vendor/src/my-lib
folder can keep all local vendoring files.
alias ggb='$GOPATH/bin/gb'
Files structure with gb
:
project
├── src/
└── vendor/
└── src/
└── github.com/
Command | Description |
---|---|
gb vendor fetch gopkg.in/telegram-bot-api.v4 | Download package to local vendor folder |
gb vendor update | Update all local packages |
gb vendor restore | Install all packages listed in manifest |
gb build | Build project from src/ |
All info about packages is located in
manifest
Go Tool
go run main.go # build + execute program
go build main.go; ./main # build + execute separately
go get -u github.com/user/repo
# cross compilation
env GOOS=linux GOARCH=amd64 go build -o ./bin/cmd ./src/
go doc
go doc ./dir
go doc ./dir.FuncName
godoc -http=:6060
goimports
formats code + manages insertion and removal of import
declarations as needed.
import
import "github.com/oleglegun/repo" // path relative to GOPATH (accessible only in current file)
import _ from "github.com/..." // import package for its side effects
func main() {
repo.Method() // Capitalized method == exported
}
In Go only Capitalized var
/const
/func
are exported (accessible
throughout the whole package (often folder)).
Scope
syntactic block
- sequence of statements enclosed in braceslexical block
- block that is not explicitly surrounded by braces
Scopes in Go:
- universe block (entire source code, lexical block)
- package level block
- file level block
for
,if
,switch
blocks- blocks for each
case
inswitch
/select
- each explicit syntactic block
In Go, package is a unit of encapsulation, not a type.
Constants
Only numbers, strings/characters and booleans can be
constants. Expression that defines constant must be constant, because it
evaluates at compile time. Can be used to set a fixed array size.
const i int64 = 10
const name = "Oleg"
const (
flag1 = true
flag2 = false
)
const x := math.Sin(0) // Wrong, Sin() needs to happen at run time
iota
- generator (auto-increment)
Excellent for enum.
iota
resets in each block ()
.
const (
one = iota + 1 // 1
two // 2
_ // skip 3
four // 4
)
const (
_ = iota * 2 // 0
i2 // 2
i4 // 4
_ // skip 6
i16 = iota * 4 // 16 - redefine
i32 // 20
)
Used for checking flags w/ mask.
// Masks
const (
flagUserIsActive = 1 << iota
flagUserIsLogged
flagUserIsSubscribed
)
var userFlags = '\x11'
println("User is active", userFlags & flagUserIsActive)
Variables
Short Notation assignment for new variables
Use for working w/ functions: age := getAge()
when we know the return
type and that there won't be any problems w/ dynamic type assignment.:=
can be used only within a function, not for package-level
variables.
age := 25
res1, err := do()
res2, err := do() // in := assignment `err` can be reused even if it has been already declared
Multiple Assignment
Multiple assignment should be used only when it helps readability
(initialization in for
loop).
var i, j, k int = 1, 2, 3
var s1, s2 string = "Hello", "World"
// 2nd variant - often used for global (package) variables
package main
var (
i int = 1
j = 2
s1 = "Hello"
)
// Tuple assignment (swap values)
// All of the right-hand side expressions are evaluated before any of the variables are updated
i, j = j, i
a[i], a[j] = a[j], a[i]
Default values
All variables in Go are initialized by default w/ nil
values. We can
define variable and instantly use it w/o value assignment. Go has
default values for all variables (unlike C
, where variable w/o value
can point to memory addr w/ trash).
int: 0
float: 0
string: ""
bool: false
*pointer : nil
Bitwise operations 位运算符
运算符 描述
% 取余
& 按位与
¦ 按位或
^ 按位异或
&^ 按位清除(AND NOT)
&^ 即是 AND NOT(x, y) = AND(x, NOT(Y)),如:
^ 作为二元运算符:按位异或 (XOR)
^ 作为一元运算符: 按位取反 (~)
procedure swap(var a,b:longint);
begin
a:=a xor b;
b:=a xor b;
a:=a xor b;
end;
package main
import "fmt"
func main() {
x := 0xDC // 11011100
y := 0xF0 // 11110000
z := x & ^y // 00001100 // y 中为 1 的位全部被清除为 0
m := x^y
n := ^y //
fmt.Printf("%08b", z)
fmt.Printf("%08b", m)
fmt.Printf("%08b", n)
}
var x uint8 = 1<<1 | 1<<5
var y uint8 = 1<<1 | 1<<2
fmt.Printf("%08b\n", x) // "00100010", the set {1, 5}
fmt.Printf("%08b\n", y) // "00000110", the set {1, 2}
fmt.Printf("%08b\n", x&y) // "00000010", the intersection {1}
fmt.Printf("%08b\n", x|y) // "00100110", the union {1, 2, 5}
fmt.Printf("%08b\n", x^y) // "00100100", XOR the symmetric difference {2, 5}
fmt.Printf("%08b\n", ^x) // "11011101", each bit is inverted
fmt.Printf("%08b\n", x&^y) // "00100000", AND NOT the difference {5}
for i := uint(0); i < 8; i++ {
if x&(1<<i) != 0 { // membership test
fmt.Println(i) // "1", "5"
} }
fmt.Printf("%08b\n", x<<1) // "01000100", the set {2, 6}
fmt.Printf("%08b\n", x>>1) // "00010001", the set {0, 4}
^x
- bitwise negation or complement (unary operation)
Types
- Basic types:
int
,float
,string
,bool
- Aggregate types:
array
,struct
- Reference types (indirect refer):
slice
,map
,pointer
,function
,channel
- Interface types
Struct
, array
and interface
may contain reference types.
Basic Data Types
int
, float
, bool
// INT
var i int = 10 // 32/64bit int, depends on platform
var autoInt = -10 // Auto assign type (int)
var bitInt int64 = 1<<32 - 1 // int8, int16, int32, int64
var uInt uint = 100 // unsigned 32/64bit int, depends on platform
var uBigInt uint64 = 1<<64 -1 // uint8, uint16, uint32, uint64
// FLOAT
var pi float64 = 3.14 // float32, float64 (precision)
// BOOL
var b = true // boolean
Use unsigned integer only when it is absolutely necessary
(hashing, cryptography, bit sets, parsing binary file format...
float64
is preferable for most purposes because float32
computations
accumulate error rapidly.
Short-circuit behavior
s != "" && s[0] == 'x' // if left is false => right will not be evaluated
string
All strings are in UTF-8.
String values are immutable.
Immutability means that it is safe for two copies of a string to share
the same underlying memory, making it cheap to copy strings of any
length. Similarly, a strings
and a substring likes[7:]
may
safely share the same data, so the substring operation is also cheap.
No new memory is allocated in either case.
var str1 string = "Hello\n\t" // double quotes for string literal
var str3 = `Hello
there` // Raw string literal (invisible symbols \t,\n)
// BINARY/'SYMBOL'
var binary byte = '\x28' // single quotes for symbols
Raw string literals are useful for RegExp, which tend to have lots
of backslashes.
var str1 = "Hello"
var str2 = ", world!"
str := []string{str1, str2}
// Length
println(len(str1)) // 5 - number of bytes
println(urf8.RuneCountInString(str1)) // 5 - number of runes
// Concatenation
// 1. `+` - Use only on small number of short strings: too many allocations
str3 := str1 + str2
// 2. strings.Join() - Use for large slices of strings
strings.Join(str, "")
// 3. bytes.Buffer - most efficient
var buffer bytes.Buffer
for _, str := range str {
buffer.WriteString(str)
}
fmt.Println(buffer.String())
if str1 == str2 {} // Equality
if str1 > str2 {} // Compare
Unicode
Unicode is embedded in Go.
1st Byte | 2nd Byte | 3rd Byte | 4th Byte | Free Bits | Maximum Unicode Value |
---|---|---|---|---|---|
0xxxxxxx | 7 | 007F hex (127) | |||
110xxxxx | 10xxxxxx | (5+6)=11 | 07FF hex (2047) | ||
1110xxxx | 10xxxxxx | 10xxxxxx | (4+6+6)=16 | FFFF hex (65535) | |
11110xxx | 10xxxxxx | 10xxxxxx | 10xxxxxx | (3+6+6+6)=21 | 10FFFF hex (1,114,111) |
rune
(Unicode code point) - is an alias for int32
and is equivalent
to int32
in all ways. It is used, by convention, to distinguish
character values from integer values.
var symbol rune = 's'
unicodeSymbol := ''
unicodeSymbolFromCodePoint := '\u2222'
A rune whose value < 256 may be written with a single hexadecimal escape\x40
, but for higher values, a \u
or \U
escape must be used.
Iterate on symbols
strRu := "Привет"
for index, rune := range strRu { // range loop performs UTF-8 decoding implicitly
fmt.Printf("%#U %d", rune, index)
println()
}
/*
U+041F 'П' 0
U+0440 'р' 2
U+0438 'и' 4
U+0432 'в' 6
U+0435 'е' 8
U+0442 'т' 10
*/
For unexpected input byte
DecodeRuneInString
orrange
loop
generate special Unicode replacement character\uFFFD
<?>.
Iterate on bytes
str := "Привет"
slice := []byte(str) // convert String to Byte array
for index, value := range slice {
fmt.Println(index, value)
}
Composite Types
Array
Is a low-level data type. Size is part of array type => can't change its
size. The types [10]int
and [20]int
are distinct.
Array is a value type. Assigning one array to another copies all the
elements. Passing array to a function - creates copy of it. Using a
pointer to an array is efficient and allows the called function to
mutate the caller's variable.
var arr1 [4]int // [0 0 0 0] create array + allocate memory for its elements
const size uint = 2
var arr2 [size]bool // [false false]
arr3 := [...]int{1, 2, 3} // [1 2 3], len = auto (3)
// Edit value of an element
arr3[2] = 10 // [1 2 10]
// Multidimensional Array
var arr4 [2][2]int // [[0 0] [0 0]], len = 2
Slice
Slice is an array w/ dynamic size. Slices hold references to an
underlying array, and if you assign one slice to another, both refer to
the same array.
Pointer points to the first element of the array that is reachable
through the slice, which is not necessarily the array's first element.
Passing a slice to a function permits the function to modify the
underlying array elements.
Slices unlike arrays are not comparable, we can compare only
[]byte
usingbytes.Equal()
struct {
length // length ≤ capacity
capacity // doubles on each length overflow
pointerToArray // pointer to the first element of a underlying array (fix size)
}
// Create slice
var sl1 []int // []
sl1 := []int{1, 2, 3} // [1 2 3]
bsl := []byte{0x0, 0xFF}
// Memory efficient creation (we know the len/cap beforehand)
sl2 := make([]int, 5) // [0 0 0 0 0] len = 5 cap = 5!!
sl3 := make([]int, 0, 10) // [] len = 0 cap = 10
// Length
len(sl1) // 3
// Capacity
cap(sl1) // 4
// Slicing (shrink)
sl := []int{1, 2, 3, 4, 5}
sl[1:3] // [2 3]
sl[1:] // [2 3 4 5]
sl[:3] // [1 2 3]
// Add element (grow)
sl1 = append(sl1, 100)
fmt.println(sl1) // [1 2 3 100]
// Concat another slice
sl := append(sl1, sl1...) // ... - spread
fmt.Println(sl) // [1 2 3 100 1 2 3 100]
// Copy slice
slCopy := make([]int, len(sl)) // new slice must have the same len
copy(slCopy, sl)
// Append slices
sl := append(sl1[:2], sl2[1:]...)
// Slice from Array
arr := [...]int{1, 2, 3}
sl := arr[:] // sl: [1 2 3]
arr[1] = 10 // sl points to the same addr as arr
fmt.println(sl) // [1 10 3]
Map
Map hold reference to an underlying data structure (created by make
).
When a map is passed to a function, it receives a copy of the reference
=> any changes will be visible through the caller's map reference too.
Map does not guarantee key order.
Keys of map can be of any comparable type. Value can be of any type at
all.
We can't take the address of a map element because growing a map might
cause rehashing of existing elements into new storage locations,
invalidating the address.
var m map[string]string // nil - points to nothing
// | | |
// type key value
// Nested Map
var mm map[string]map[string]string
// Initialize
var m map[string]string = map[string]string{} // Empty Interface {}
// Initialize w/ `make`
var m = make(map[string]string)
// Short notation for Initialization
m := map[string]string{}
// Add element
m["firstName"] = "Oleg" // String literal as key
m[varName] = "string" // Variable as key
// Read element
name = m["firstName"]
// Read nonexistent element
name = m["lastName"] // "" - returns default zero value for a type
// Check if key exists
_, ok := m["lastName"] // only check for key existence
lastName, ok := m["lastName"] // returns value + ok - bool
// Delete element
delete(m, "firstName")
, ok
- "comma ok" idiom
Control Flow
if
Checks strictly for bool
value. Doesn't support implicit type
conversion (1 ≠ true
).
if flag {
println("flag is true")
}
if a == 1 && flag {
println("success")
}
// Complex if using `;`
// firstName, ok are only accessible in 'if' scope
if firstName, ok := m["firstName"]; ok {
// | |
// can be `_` bool to check
println("Key exists")
} else {
println("Key doesn't exist")
}
for
// Infinite loop
for {
println("Works like `while true {}`")
break
}
// Conditional `for` == while
for i < 5 {
println(i)
i++
}
// Conditional `for` w/ `if`
for i < 10 {
if i%2 == 0 {
i++
continue
}
println(i)
i++
}
sl := []int{1, 3, 5}
// Standard `for`
for i := 0; i < len(sl); i++ {
println("Iteration")
}
// `for` w/ `range` for slice
for index, value := range sl {
println(index, "\b:", value)
}
// `for` w/ `range` for map
m := make(map[string]string)
m["firstName"] = "Oleg"
m["lastName"] = "Legun"
for key, value := range m {
println(key, ":", value)
}
switch
We don't fall through cases in Go - don't need break
.
m := make(map[string]string)
m["firstName"] = "Oleg"
switch m["firstName"] {
case "Oleg":
println("Name is set to 'Oleg'")
case "Igor":
println("Name is set to 'Igor'")
if true {
break // exit switch
}
println("Some text")
case "":
println("Name is empty")
fallthrough // fall to next case
default:
println("Please, set a valid 'firstName'")
}
Check condition directly in case
. Can substitute multiple if-else
.
switch { // tagless switch
case m["firstName"] == "Oleg":
println("First Name: Oleg")
case m["firstName"] == "":
println("First Name is empty")
}
To break out from loop inside switch
, we need to use label
.
m := make(map[string]string)
m["firstName"] = "Oleg"
Loop:
for key, val := range m {
switch {
case key == "firstName" && val == "Oleg":
println("Breaking...")
break Loop
}
println("Won't be outputted")
}
Function
Go has no concept of default parameter values.
In Go (call-by-value language) the called function always receives
only a copy of an argument, not a reference to the original argument.
// Local function
func sum(a int, b int) int {
return a + b
}
// Sum is an exported function and must have a comment
func Sum(a int, b int) int {
return a + b
}
// Short notation (same type)
func sum(a, b int) int {
return a + b
}
// Any number of arguments of the same type
func sum(sl ...int) int { // sl - variadic parameter (== []int)
var res int
for i:= range sl {
res += sl[i]
}
return res
}
// Named return variable => initialized w/ null value (ready to use)
func sum() (res int) {
res = 1
return
}
// Multiple return
func multiReturn() (int, error) {
return 1, nil
}
// Multiple return w/ named variables
func multiReturn() (res int, err error) {
res = 1
err = fmt.Errorf("Error")
return
}
Getter method should be called
Property
(upper case, exported), notGetProperty
. A setter function, if needed, will likely be calledSetProperty
.
Anonymous Function
Function in Go is first class citizen. We can save pointer to a
function or pass it in the moment of creation.
f := func() func() { // return func()
return func() {
println("success")
}
}
pointer = f() // pointer to function
pointer() // "success"
f()() // "success"
Closure
func main() {
closure()()
}
func closure() func() {
a := 1
return func() { // has a in closure
print(a)
}
}
Recursive Functions
Are good for processing data w/ recursive structure. Unefficient on
memory, each new call expands stack. Good w/ last call optimization.
Many programming language implementations use a fixed-size function
call stack; sizes from 64KB to 2MB are typical. Fixed-size stacks
impose a limit on the depth of recursion, so one must be careful to
avoid a stack overflow when traversing large data structures
recursively; fixed-size stacks may even pose a security risk. In
contrast, typical Go implementations use variable-size stacks that
start small and grow as needed up to a limit on the order of a
gigabyte. This lets us use recursion safely and without worrying about
overflow.
defer
Block defer
will be executed on exiting current function. The
canonical examples are unlocking a mutex or closing a file/connection.
Go's gc won't release unused OS resources like open files and
network connections. They must be closed explicitly.
The last defined defer
will be executed first and so on (LIFO).
func main() {
defer println("Done")
println("Last command")
}
func ReadFile(f string) ([]byte, error) {
file, err := os.OpenFile(...)
if err != nil {
return nil, err
}
defer file.Close() // Now we can exit function any time
return nil, nil
}
Right place for a defer statement that releases a resource is
immediately after the resource has been successfully acquired.
In
defer obj.method().close()
-.method()
will be called right
afterdefer
definition! Only.close()
will be deferred. Arguments
to deferred functions are evaluated when thedefer
executes.
panic()
panic()
- creates exception. We can catch it using recover()
. Create
and handle panic - expensive operations!
Use panic when some impossible situation happens.
func main() {
createPanic()
println("Continue working")
}
func createPanic() {
defer func() {
// Handle panic
if r := recover(); r != nil {
println("Error happened")
}
// Capture the stack trace
buf := make([]byte, 10000)
runtime.Stack(buf, false)
println("Stack:", string(buf))
}()
panic("Error") // Will immediately exit current function
}
init()
In every package function init()
will be called at the start of
execution, before main()
.
Doesn't take any arguments or return any values.
package main
var user = os.Getenv("USER")
var a int
func init() {
if user == "" {
panic("no value for $USER")
}
a = 100 // Is set before main() executes
}
func main() {
println(a)
}
It's used to initialize variables w/ needed values or to verify/repair
the program state before real execution begins.
All
init()
functions in all packages will be called simultaneously
=> order is unpredictable.
Pointers
Variable - storage which contains value. Pointer allows access/update
the value indirectly: w/o knowing the name of the variable.
a := 1
aPtr := &a // *int
*aPtr = 10
bPtr := new(int) // return pointer to int
*bPtr = 10
Not every value has an address, but every variable does.
Each component of a variable of aggregate type (field of a struct,
element of an array) - is also a variable and thus han an address.
It is safe for a function to return the address of a local variable.
new
vs. make
new()
- allocates zeroed storage for any typeT
and returns
pointer.new(type)
is just&type{}
. Usenew
when there is no
need of a dummy name.make()
- only for slices, maps and channels, returns an initialized
(not zeroed) value of typeT
(not*T
)
Slice, map or channel represent references to data structures that
must be initialized usingmake
before use. Until these
structures are initialized these types arenil
.new
only allocates
storage!
The new
function is rarely used because the most common unnamed
variables are of struct
types, for which the struct literal syntax
(User{}
) is more flexible.
Heap vs. Stack
A compiler may choose to allocate local variables on the heap or on the
stack based on escape analysis. Each variable that escapes requires an
extra memory allocation.
var global *int
func f() {
var x int
x = 1
global = &x // x escapes from f => stack allocation is not safe => allocate on heap
}
Named basic types
The named type provides a way to separate different uses of the
underlying type so that they can't be mixed unintentionally.
// Distinguishing the types makes it possible to avoid errors
// like combining temperatures in the two different scales.
type Celsius float64
type Fahrenheit float64
// Explicit conversion is required
tc := Celsius(-273.15)
tf := Fahrenheit(180)
// Error: mismatched types
temp := tf + tc
Struct (concrete type)
Type is a pointer to a struct
.
Struct is a value type and describes properties of an object.
If variable is a large
struct
and performance is an issue, it's
preferable to pass variable by pointer*T
. So that to avoid
expensive copying of the whole struct in memory.
We can compare struct
s (only for equality): they are equal if all
their properties are equal.
All fields in a
struct
are initialized w/ defaultnil
values when
usingT{}
type Text string
type Person struct {
Name string
age int
}
// Initialize w/ default values
var article Text // ""
var p1 Person // {"", 0}
p2 := Person{} // {"", 0}
p3 := new(Person) // &{"", 0}
// Initialize w/ custom values using literal
article := Text("Simple Text")
p := Person{
Name: "Oleg", // Do not need to set all fields
age: 26
}
// Short variant
p := Person{"Oleg", 26} // Need to specify all fields
// Get pointer to a struct's field
name := &p.Name
*name = "Jack"
Struct can't contain itself (recursion), but it can contain pointer to
itself.
type tree struct { // binary tree
value int
left, right *tree
}
Type methods
Any type like string
, struct
, func
... can have its own methods. Go
doesn't use this
or self
, it uses receiver name.
type MyType int
// Value method uses copy of a type
func (m MyType) getInfo() { // get m by value (copy)
fmt.Printf("%T %v", m, m)
}
// Pointer method uses address of a type
func (m *MyType) Increment() { // get m by reference (can change value)
*m = *m + 1 // Need to dereference pointer
}
Value methods can be invoked on pointers and values, but
pointer methods can only be invoked on pointers. This rule
arises because pointer methods can modify the receiver, invoking them on
a value would cause the method to receive a copy of the value, so any
modifications would be discarded.
// Person - simple type
type Person struct {
name, surname string
age int
}
// Method for type Person
func (p Person) getAge() int {
return p.age
}
// Method for pointer to type Person
func (p *Person) setAge(age int) {
p.age = age
}
func main() {
oleg := Person{"Oleg", "Legun", 25}
oleg.setAge(26) // implicit &oleg on variable
Person{"Jack", "", 10}.setAge(20) // compile error: can't take address of Person literal
println(oleg.getAge())
}
If any method of a type
T
has a pointer receiver, you should avoid
copying instances ofT
because doing so may violate internal
invariants. For example, copying an instance ofbytes.Buffer
would
cause the original and the copy to alias the same underlying array of
bytes. Subsequent method calls would have unpredictable effects.
Method Values and Expressions
type User struct{ name string }
func (u *User) SetName(n string) { u.name = n }
func (u User) PrintName() { fmt.Println(u.name) }
func main() {
u := User{}
setName := u.SetName // Method value
setName("Jack")
PrintNameExp := User.PrintName // Method expression
PrintNameExp(u) // first arg == instance of User, then any args for method
}
Interface (abstract type)
Interface - describes behaviour = only methods that will be
implemented by some type.
One-method interfaces are named by the method name plus an
-er
suffix:Sender
,Reader
,Closer
...
type Speaker interface {
Speak()
}
type Person struct {
Name string
}
func (p Person) Speak() {
println(p.Name + " is speaking...")
}
func SpeakTwice(s Speaker) { // Receives object w/ methods described in Speaker
s.Speak()
s.Speak()
}
func main() {
oleg := Person{"Oleg"}
oleg.Speak()
SpeakTwice(oleg)
}
We can use Duck Typing in Go - if type struct
implements all methods
from interface
- we can pass it like this interface
.
Use interface
only when it's absolutely needed.
Empty interface as a type
interface{}
- empty interface has no methods -> any type conforms to
it.
func Do(any interface{}) interface{} {
// to use `any` we need to implement type assertion.
}
Interface Value
Interface value has 2 components:
- concrete type (dynamic type)
- value of its type (dynamic value)
Zero value for an interface has both its type and value components set
to nil
.
var w io.Writer
/* w
type = nil
value = nil
*/
w.Write([]byte("text")) // panic: nil pointer dereference
w = os.Stdout
/* w
type = *os.File
value = *---------> os.File(fd int = 1(stdout))
*/
// dynamic value holds a copy of os.Stdout, which is a pointer to the
// os.File variable representing the standard output of the process
w.Write([]byte("text")) // success
We cannot know at compile time what the dynamic type of an
interface
value will be, so a call through aninterface
must use
dynamic dispatch. Instead of a direct call, the compiler must
generate code to obtain the address of the method namedWrite
from
the type descriptor, then make an indirect call to that address. The
receiver argument for the call is a copy of the interface’s dynamic
value,os.Stdout
.
Interface with nil
Pointer
func main() {
var buf *bytes.Buffer // must be io.Writer
f(buf)
}
func f(out io.Writer) {
if out != nil { // dynamic value = nil, type = *bytes.Buffer, thus is not nil!
out.Write([]byte("text")) // panic: nil pointer dereference
}
}
Embed
Similar to inheritance.
By embedding a struct
we get its properties + methods.
type Person struct {
name string
Age int
}
type Student struct {
Person // indicate only type
course int
Age int // shadow property from Person
}
Only interfaces can be embedded within interfaces.
Error
func Do() err {
return errors.New("simple error")
}
Any obj that implements error
interface can be returned as an error.
type error interface {
Error() string
}
Thus we can create custom error types.
// Create a named type for our new error type
type myError string
// Implement the error interface
func (e myError) Error() string {
return string(e) // wrapper above string
}
Message strings should not be capitalized and newlines should be
avoided.
Type Assertion
having created an interface{}
value containing a boolean,
float, string, map, pointer, or any other type, we can
do nothing directly to the value it holds since the interface has
no methods. We need a way to get the value back out again.
func DoubleSpeak(s interface{}) { // Receives object w/ methods described in Speaker
if p, ok := s.(Speaker); ok { // if s implements Speaker interface
p.Speak()
p.Speak()
}
if p, ok := s.(Person); ok { // if s obj is of type Person
println(p.Name) // use property of p
}
}
Switch Type Assertion
Check type:
func showMeTheType(i interface{}) string {
switch i.(type) {
case int:
return "int"
case uint:
return "uint"
case int8:
return "int8"
case float64:
return "float64"
case string:
return "string"
case rune:
return "int32"
case []int:
return "[]int"
case map[string]bool:
return "map[string]bool"
default:
return "undefined"
}
}
err.(type)
works only in switch
.
Check error type:
switch err := err.(type) { // Check error type
case nil:
case *MyError:
default:
}
Goroutines
Overview
Go routine
- "simplified" thread, independent code that executes
asynchronously and has its own stack. To start go routine we need 2-4KB
(depending on the architecture).
Process - in OS, have separate address space (independent). We need to
use pipes or other OS mechanisms to coordinate processes.
Threads are within Process. Have common assets - can conflict (use
mutex). To start thread we need 1MB for its data.
In Go we don't know what thread we are working on. All concurrency is
managed by Go runtime.
Go runtime manages dispatching go routines on threads.
Go routines can lead to memory leaks, if not finished correctly.
Concurrency — structuring a program as independently executing
components and parallelism — executing calculations in parallel for
efficiency on multiple CPUs.
package main
import "fmt"
func main() {
go test() // tells runtime to create "subprocess" w/ its own stack
go print(5) // go routine w/ parameters
go func() { // anonymous
println("anonymous go routine")
}()
fmt.Scanln() // We need to wait go routine completion
}
func test() {
println("routine")
}
func print(i int) {
println(i)
}
go run -race
Used to debug concurrent programs.
Go routines that operate on the same object can cause unexpected changes
(they need to be synced). Check problems w/ go run -race ...
. -race
arbitrary switches between goroutines in runtime.
go run -race main.go
...
Found 5 data race(s)
exit status 66
Indicates that 5 simultaneous read/write operations happened w/o blocks.
go routine
& closure
for i := 0; i < 5; i++ {
go func() {
println(i) // will print '4' every time
}()
}
When async println(i)
is invoked - for
loop has already finished =>
i == 4
We need to explicitly pass needed values to save them in closure:
for i := 0; i < 5; i++ {
go func(i int) {
println(i) // will print 1 2 3 4
}(i)
}
Shadowing variable
It's legal and idiomatic in Go to do this. You get a fresh version of
the variable with the same name, deliberately shadowing the loop
variable locally but unique to each goroutine.
func Serve(queue chan *Request) {
for req := range queue {
req := req // Create new instance of req for the goroutine.
sem <- 1
go func() {
process(req)
<-sem
}()
}
}
Channel
Overview
Channels are pipes for concurrent go routines. We can read from/write to
channel and it works in sync.
Channel is a reference type (pointer to a struct inside).
Buffered & unbuffered channels
Channel can be unbuffered or buffered.
Unbuffered channel works in synchronous mode - we can't write to
channel, unless somebody is ready to read from it.
func main() {
ch := make(chan string) // Create channel for strings
go writeToChannel(ch) // Asynchronously write to channel
fmt.Println(<-ch, ",", <-ch) // Read 2 times from channel
}
func writeToChannel(ch chan<- string) { // receive channel (implicitly indicate write-only channel)
ch <- fmt.Sprintf("Foo") // will pause until smb. is ready to read from ch
ch <- fmt.Sprintf("Bar")
ch <- fmt.Sprintf("Baz") // will not be printed, since we read only 2 times from ch
}
Channel direction (prevents errors in compile time - it's just an
instruction for compiler checks):
<-chan
- read-onlychan<-
- write-only
To use buffered channel, we must implicitly indicate its size.
bufCh := make(chan int, 7) // Create buffered channel for ints
We can write to bufCh
7 times before our program execution blocks (in
case nobody read from bufCh
).
To close channel:
close(bufCh)
// Check if buffer is closed
data, ok := <-bufCh // if opened: ok == true, if closed -> false
We can't write to closed channel (panic
), but can read from it (always
returns nil
)
Read from channel w/ range
func readFromChannel(ch <-chan string) {
for r := range ch {
println(r)
}
}
If chanel is closed - range
will stop iterating.
Channel generator
func main() {
channel := createChannel("Hello")
for {
fmt.Println(<-channel)
}
}
func createChannel(message string) <-chan string {
ch := make(chan string)
go func() {
for i := 0; ; i++ { // Infinite loop
ch <- fmt.Sprintf("%d. %s", i, message)
time.Sleep(time.Second)
}
}()
return ch
}
time.After(interval)
- is a channel generator, that returns channel
that receives bool values true
each time interval passes.
Sync go routines using channels
We can use
chan struct{}
orchan bool
to implementdone
channel.struct{}
has zero size, but syntax is more cumbersomech <- struct{}{}
and space saving is marginal.
func main() {
done := make(chan bool, 1)
go mustPrint(done)
// Blocking receive
<-done // await, read from done, otherwise program will exit earlier
}
func mustPrint(done chan<- bool) {
for i := 0; i < 10; i++ {
println(i)
}
done <- true // write to done on finish
}
Channel multiplexer
Read several channels simultaneously. Order is not guaranteed, i.e.
working w/ unsynced channels.
func main() {
ch1, ch2 := generateNumbers()
ch := readAll(ch1, ch2)
for {
fmt.Println(<-ch)
time.Sleep(time.Second)
}
}
func generateNumbers() (<-chan int, <-chan int) {
ch1, ch2 := make(chan int, 1), make(chan int, 1)
go func() {
for i := 0; i < 10; i++ {
ch1 <- i
}
}()
go func() {
for i := 10; i > 0; i-- {
ch2 <- i
}
}()
return ch1, ch2
}
// readAll reads 2 channels simultaneously and writes result to the returned channel
func readAll(ch1, ch2 <-chan int) <-chan int {
ch := make(chan int)
go func() {
for {
ch <- <-ch1
}
}()
go func() {
for {
ch <- <-ch2
}
}()
return ch
}
Select
Works like switch
but for channels.
Checks all cases in order and checks for available data in channels. If
any - case will be executed, if not - waits and passes control to other
routines.
for {
select {
case msg := <-ch1:
println(msg)
case msg := <-ch2:
println(msg)
case <-time.After(time.Second): // Guarantees non-blocking execution
println("timeout") // excellent for networking requests
}
runtime.Gosched() // Runtime can pass control to other routine
}
If select
doesn't have default
case - program blocks execution
and waits for input on any channel, otherwise it executes default
case.
select {
default: // non-blocking channel operation
println("default")
}
default
case guarantees infinite loop - no code will be blocked if
no channel is ready for read.
Priority select: 1 channel is more important that another.
Go runtime can't take execution control from the subroutine in arbitrary
moment (when it's rendering smth.), but runtime.Gosched()
tells
runtime that there will be waiting, and it can pass control to other
routine during it. Useful in infinite loops!
select
can also act as a channel multiplexer
:
func readAll(ch1, ch2 <-chan int) <-chan int {
ch := make(chan int)
go func() {
for {
select {
case num := <-ch1:
ch <- num
case num := <-ch2:
ch <- num
}
}
}()
}
Packages
io
Working with byte streams.
Reader
type Reader interface {
// Pass buffer p to reuse the same bytes (helps GC)
// p is not guaranteed to be filled: 0 - len(p)
// returns io.EOF on stream done (normal behavior)
Read(p []byte) (n int, err error)
}
ReadFull()
, ReadAtLeast()
// ReadFull ensures that buf is completely filled
// Returns io.EOF or io.ErrUnexpectedEOF on partial read
func ReadFull(r Reader, buf []byte) (n int, err error)
// ReadAtLeast reads at least min bytes
// We can minimize a number of Read() calls buffering additional data
func ReadAtLeast(r Reader, buf []byte, min int) (n int, err error)
MultiReader()
// MultiReader combines n Readers into one concatenated reader
func MultiReader(readers ...Reader) Reader
r := io.MultiReader(
bytes.NewReader([]byte("string")),
file,
)
http.Post("http://...", "application/octet-stream", r)
TeeReader()
// TeeReader duplicates streams: returns new Reader, that wraps r
// Any reads from r will be written to w (ex. bytes.Buffer{}) for further inspection
func TeeReader(r Reader, w Writer) Reader
LimitReader()
// LimitReader wraps r and stops reading with EOF after n bytes or less
func LimitReader(r Reader, n int64) Reader
Set
n = n + 1
to check ifr
has more thann
bytes to read
Writer
type Writer interface {
Write(p []byte) (n int, err error)
}
MultiWriter()
// MultiWriter duplicates writes to multiple streams (useful for logging)
func MultiWriter(writers ...Writer) Writer
WriteString()
// WriteString uses optimized WriteString() method of w if available
// Optimization: converts string to a []byte w/o allocations
func WriteString(w Writer, s string) (n int, err error)
Copy()
, CopyN()
, CopyBuffer()
// Copy copies all bytes from Reader to Writer using 32KB buffer
func Copy(dst Writer, src Reader) (written int64, err error)
// CopyN copies N bytes from Reader to Writer
func CopyN(dst Writer, src Reader, n int64) (written int64, err error)
// CopyBuffer uses passed buf []byte w/o allocation its own 32KB buffer
func CopyBuffer(dst Writer, src Reader, buf []byte) (written int64, err error)
WriterTo
, ReaderFrom
type WriterTo interface {
// Directly write data to w w/o intermediate buffer (optimized)
WriteTo(w Writer) (n int64, err error)
}
type ReaderFrom interface {
ReadFrom(r Reader) (n int64, err error)
}
Pipe()
// Pipe provides new Reader and Writer, any writes to Writer will go to Reader
// Useful if we have a method that only accepts a Reader, but we only have a Writer
func Pipe() (*PipeReader, *PipeWriter)
Closer
type Closer interface {
Close() error
}
Seeker
type Seeker interface {
// Seek is useful for fixed length blocks in a file
// whence can be io.SeekStart, io.SeekCurrent, io.SeekEnd
Seek(offset int64, whence int) (int 64, error)
}
ByteReader
, ByteWriter
, ByteScanner
type ByteReader interface {
// Read single byte
ReadByte() (c byte, err error)
}
type ByteWriter interface {
WriteByte(c byte) error
}
// For working with buffered byte readers
// that push the prev byte back for reading (peek next byte)
type ByteScanner interface {
ByteReader
UnreadByte() error
}
RuneReader
, RuneScanner
type RuneReader interface {
ReadRune() (r rune, size int, err error)
}
type RuneScanner interface {
RuneReader
UnreadRune() error
}
bytes
, strings
Working with bounded, in-memory byte slices. The elements of a byte
slice can be freely modified.
[]byte
- mutable, resizable, contiguous list of bytesstring
- immutable, fixed-size, contiguous list of bytes.
NewReader
Create Reader that wraps an in-memory []byte
or string
. It also
implements io
interfaces: WriterTo
, ByteReader
, ByteScanner
,RuneReader
, RuneScanner
, Seeker
.
func NewReader(b []byte) *Reader
func NewReader(s string) *Reader
// Effective on memory
r := strings.NewReader("oleg") // Can be inside of io.MultiReader() if needed
http.Post("http://...", "text/plain", r)
Buffer
For buffer use bufio
package.
// Buffer of bytes with many Read and Write methods.
// The zero value for Buffer is an empty buffer ready to use.
type Buffer struct { /* filtered */ }
// Useful Type Methods
buf.WriteString(s string) (n int, err error) {} // appends string to the end of the buffer
buf.Reset() // Empties buffer
var buf bytes.Buffer
myApp.Logger = log.New(&buf, "", log.LstdFlags)
myAPp.Start()
if !strings.Contains(buf.String(), "app failed") {
log.Fatal("Message")
}
func Equal(a, b []byte) bool {} // check for equality byte slices, for strings use ==
func EqualFold(s, t string) bool {} // check for equality 2 string ignoring case
func Contains(b, subslice []byte) bool {}// 1+ subslice in b
func Contains(s, substr string) bool {} // 1+ substr in s
func Count(b, subslice []byte) int {} // Exact num of sublices in b
func Count(s, substr string) int {} // Exact num of substr in s, if substr is "" -> returns len+1 of chars
// Find position
func Index(s, sep []byte) int {}
func IndexAny(s []byte, chars string) int {}
func IndexByte(s []byte, c byte) int {}
func IndexFunc(s []byte, f func(r rune) bool {}) int {}
func IndexRune(s []byte, r rune) int {}
// Includes validation for blank slice (no panic)
func HasPrefix(s, prefix []byte) bool {} // "PrefixString"
func HasPrefix(s, prefix string) bool {}
func HasSuffix(s, suffix []byte) bool {} // "StringSuffix"
func HasSuffix(s, suffix string) bool {}
// Remove any runes defines in cutset from the beginning/end
func Trim(s []byte, cutset string) []byte {}
func Trim(s string, cutset string) string {}
func TrimLeft(s []byte, cutset string) []byte {}
func TrimRight(s []byte, cutset string) []byte {}
func TrimFunc(s []byte, f func(r rune) bool {}) []byte {}
// Remove prefix/suffix
func TrimPrefix(s, prefix []byte) []byte {}
func TrimPrefix(s, prefix string) string {}
func TrimSuffix(s, suffix []byte) []byte {}
func TrimSuffix(s, suffix string) string {}
// Remove space from the beginning/end
func TrimSpace(s []byte) []byte {}
func TrimSpace(s string) string {}
// Replace (for templates)
func Replace(s, old, new []byte, n int) []byte {}
func Replace(s, old, new string, n int) string {}
// Change case
func ToUpper(s []byte) []byte {}
func ToUpper(s string) string {}
func ToLower(s []byte) []byte {}
func ToLower(s string) string {}
func Title(s []byte) []byte {}
func Title(s string) string {} // "hello world" -> "Hello World"
// Split slice using separator
func Split(s, sep []byte) [][]byte {}
func Split(s, sep string) []string {} // ("1:2:3", ":") -> ["1", "2", "3"]
func SplitN(s, sep []byte, n int) [][]byte {}
func SplitN(s, sep string, n int) []string {}
func SplitAfter(s, sep []byte) [][]byte {}
func SplitAfter(s, sep string) []string {}
func SplitAfterN(s, sep []byte, n int) [][]byte {}
func SplitAfterN(s, sep string, n int) []string {}
func Fields(s string) []string {} // ("hello world") -> ["hello", "world"]
func Join(s [][]byte, sep []byte) []byte {}
func Join(a []string, sep string) string {}
func Repeat(s string, count int) string {} // ("-", 5) -> "-----"
bufio
// Scanner reads input and breaks it into lines/words.
// Best way to process input that comes in lines
type Scanner struct {}
input := bufio.NewScanner(os.Stdin)
input.Scan() // reads next line + removes `\n`; returns false
input.Text() // retrieve result from scan
encoding
Encoding
/Decoding
applies a logical structure to a stream of
bytes (io.Reader
/io.Writer
)Marshaling
/Unmarshaling
applies a logical structure to bounded,
in-memory bytes ([]byte
)
Go structures are already represented in-memory as bytes. We can use
them directly. gob
(stream encoding) included the schema format when
encoding, but is large. No cross language support.
type BinaryMarshaler interface {
MarshalBinary() (data []byte, err error)
}
type BinaryUnmarshaler interface {
UnmarshalBinary(data []byte) error
}
// Output in UTF-8 format
type TextMarshaler interface {
MarshalText() (text []byte, err error)
}
type TextUnmarshaler interface {
UnmarshalText(text []byte) error
}
encoding/json
JSON encoding/decoding is slow. Encoder checks for a MarshalJSON
method of an type, then for MarshalText
. If no one method is
implemented - it will recursively build an encoder based on the
primitive encoders (intEncoder
, stringEncoder
, mapEncoder
,structEncoder
, sliceEncoder
).
type Marshaler interface {
MarshalJSON() ([]byte, error)
}
type Unmarshaler interface {
UnmarshalJSON([]byte) error
}
Channel, complex types, functions cannot be encoded in JSON.
RawMessage
RawMessage
is a raw encoded JSON value. It implements Marshaler
andUnmarshaler
and can be used to delay JSON decoding or precompute a
JSON encoding.
type T struct {
Type string `json:"type"`
Value json.RawMessage `json:"value"`
}
func (t *T) Val() (interface{}, error) {
switch t.Type {
case "foo":
// parse "t.Value" as Foo
case "bar":
// parse "t.Value" as Bar
default:
return nil, errors.New("invalid type")
}
}
Indent()
, SetIndent()
, Compact()
Pretty print JSON.
func Indent(dst *bytes.Buffer, src []byte, prefix, indent string) error {}
func Compact(dst *bytes.Buffer, src []byte) error {}
// For using with Encoder
func (enc *Encoder) SetIndent(prefix, indent string) {}
JSON package has many error types, there’s not much we can do to handle
them in code other than log an error and have a human operator
intervene.
ffjson
lib - faster JSON processing for Go
NewEncoder()
, NewDecoder()
Useful for efficient encoding/decoding with less heap allocations.
type Person struct {
FirstName string `json:"first_name"`
LastName string `json:"last_name"`
}
func main() {
person := Person{"Oleg", "Legun"}
// Marshal - save to []byte
personEncoded, err := json.Marshal(person)
if err != nil {
log.Fatal(err)
}
personDecoded := Person{}
// Unmarshal
err = json.Unmarshal(personEncoded, &personDecoded)
if err != nil {
log.Fatal(err)
}
fmt.Printf("Unmarshal: %+v\n", personDecoded)
// Encoder - encode & write strait to the Writer
encoder := json.NewEncoder(os.Stdout)
err = encoder.Encode(person)
if err != nil {
log.Fatal(err)
}
// Decoder - read & decode strait from the Reader
reader := bytes.NewReader(personEncoded)
decoder := json.NewDecoder(reader)
err = decoder.Decode(&personDecoded)
if err != nil {
log.Fatal(err)
}
}
Struct Field Attributes
type Person struct {
// Change the field name in JSON output
FirstName string `json:"first_name"`
// Lowercase (unexported) properties are ignored by Marshal()
lastName string `json:"last_name"`
// Omit this field in output
Patronymic string `json"-"`
// Omit this non-struct field if empty
Age int `json:"age, omitempty"`
// Convert field value to a quoted string (very important for 64-bit values
// JS has problems with values above 53 bits start to lose integer precision
Phone int `json:",string"`
}
json.Encoder
uses sync.Pool
to reuse internal buffer (minimize heap
allocations).
encoding/binary
- In text protocol like JSON number 128 is represented with 1 + 2 + 8
chars. - In binary protocol 128 is represented with a single byte
0x80
Endianness
- Little-endian (CPU byte order = resizable) -
[less signif. byte][most signif. byte] = 11 22 33 (00 00)
- Big endian (network byte order = sortable) -
[most signif. byte][less signif. byte] = 33 22 11
v := uint32(65534)
buf := make([]byte, 4)
binary.BigEndian.PutUint32(buf, v) // [00 00 ff fe]
binary.LittleEndian.PutUint32(buf, v) // [fe ff 00 00]
Variable-length encoding
Small numbers take less space (trim empty zero bytes).
// Encode varint x to a []byte slice
func PutUvarint(buf []byte, x uint64) int {}
func PutVarint(buf []byte, x int64) int {}
// Decode varint from a []byte slice
func Uvarint(buf []byte) (uint64, int) {}
func Varint(buf []byte) (int64, int) {}
// Decode varint from a byte stream (one by one)
func ReadUvarint(r io.ByteReader) (uint64, error) {}
func ReadVarint(r io.ByteReader) (int64, error) {}
strconv
Package for efficient conversions of primitive values (bool
, int
,float
, string
). fmt
is slower.
int
// Parse int from a string
// base: 2-36 (0 - auto); bitSize: 8-64 (0 - default system int size 32/64)
func ParseInt(s string, base int, bitSize int) (int64, error) {}
func ParseUint(s string, base int, bitSize int) (uint64, error) {}
func Atoi(s string) (int, error) {} // == ParseInt(s, 10 ,0)
// int -> string (base: 2 - 36 [0-9 + a-z])
func FormatInt(i int64, base int) string {} // new string is allocated on each call
func FormatUint(i uint64, base int) string {}
// int -> string (less allocations, reusable buffer)
func AppendInt(dst []byte, i int64, base int) []byte {}
func AppendUint(dst []byte, i uint64, base int) []byte {}
float
// Parse float from a string, bitSize: 32/64
func ParseFloat(s string, bitSize int) (float64, error) {}
// float -> string
// prec: -1 - auto from bitSize (smallest num possible), bitSize: 32/64
func FormatFloat(f float64, fmt byte, prec, bitSize int) string {}
boolean
// true: "1" "t" "T" "true" "True" "TRUE"
// false: "0" "f" "F" "false" "False" "FALSE"
// error: others
func ParseBool(str string) (bool, error) {}
// bool -> string; returns "true"/"false"
func FormatBool(b bool) string {}
// bool -> string
// Appends "true"/"false" to the end of dst, returns extended buffer
func AppendBool(dst []byte, b bool) []byte {}
string
// Quote (double quotes) string s, converting non-printable chars to `\t`, `\n`, `uXXXX`
// Useful for displaying error messages.
func Quote(s string) string {} // "Hello " -> "\"Hello\t\""
// Converts all non ASCII chars (non-printable, emoji)
func QuoteToASCII(s string) string {}
// Less heap allocations when quoting strings
func AppendQuote(dst []byte, s string) []byte {}
func AppendQuoteToASCII(dst []byte, s string) []byte {}
// Quote individual rune (single quotes)
func QuoteRune(r rune) string {}
func QuoteRuneToASCII(r rune) string {}
// Less heap allocations when quoting runes
func AppendQuoteRune(dst []byte, r rune) []byte {}
func AppendQuoteRuneToASCII(dst []byte, r rune) []byte {}
// Unquote strings (parses "", '' and `` strings)
func Unquote(s string) (string, error) {}
fmt
%v default format
bool: %t
int, int8 etc.: %d
uint, uint8 etc.: %d, %#x if printed with %#v
float32, complex64, etc: %g
string: %s
chan: %p
pointer: %p
%+v add field names
%#v valid Go syntax, can be pasted into the code right away
%T print type (useful for the dynamic type of an interface value)
%d int (base-10)
%x int (base-16)
%f float
%5.2f 1.23 -> " 1.23" (width = 5, precision = 2)
%-5.2f 1.23 -> "1.23 "
%05.2f 1.23 -> "001.23"
%q quoted string (use-case: print with invisible chars)
%p pointer (use-case: check if 2 pointer variables reference the same data)
// Print to os.Stdout
func Print(a ...interface{}) (n int, err error) {}
func Printf(format string, a ...interface{}) (n int, err error) {}
func Println(a ...interface{}) (n int, err error) {}
// Print to io.Writer, F stands for File
func Fprint(w io.Writer, a ...interface{}) (n int, err error) {}
func Fprintf(w io.Writer, format string, a ...interface{}) (n int, err error) {}
func Fprintln(w io.Writer, a ...interface{}) (n int, err error) {}
LogOutput := os.Stdout
fmt.FPrintln(LogOutput, "Error")
// Print to string (slow! - use reusable bytes.Buffer with Fprint())
func Sprint(a ...interface{}) string {}
func Sprintf(format string, a ...interface{}) string {}
func Sprintln(a ...interface{}) string {}
// Just a wrapper for errors.New(Sprintf())
func Errorf(format string, a ...interface{}) error {}
// Use for custom types
type Stringer interface {
String() string
}
Do not use
Scan
functions - use CLI flags, env vars, API calls
instead, config files.
http
ServeMux
HTTP request multiplexer. Matches URLs of incoming requests against a
list of registered patterns and calls the handler for the pattern that
most closely matches the URL. A ServeMux
aggregates a collection ofhttp.Handler
s into a single http.Handler
. Doesn't support wildcards
and RegExp.
Matching rules:
/
matches all unmatched paths + root- long path has more priority over short path (
/src/img/
>/src/
) /src
will be redirected to/src/
if handler for/src/
is
registered (override - register handler for/src
)
Several
ServeMux
may be composed to handle more intricate
dispatching requirements.
Handler
, HandlerFunc
, Handle()
, HandleFunc()
// Any object that implement Handler can used as a router
type Handler interface {
// Return from ServeHTTP == request is finished
ServeHTTP(ResponseWriter, *Request)
}
// HandlerFunc is an Adapter to allow the use of functions as HTTP handlers
// its ServeHTTP() method calls underlying function
type HandlerFunc func(ResponseWriter, *Request)
To add handlers to a ServeMux
use:
// DefaultServeMux
// Requires an object that implements Handler interface
func Handle(pattern string, handler Handler) {}
// HandleFunc registers a handler function for the given pattern
// ServeHTTP() method calls handler() passing (ResponseWriter, *Request)
func HandleFunc(pattern string, handler func(ResponseWriter, *Request)) {}
// Custom ServeMux
mux := http.NewServeMux()
mux.Handle("/", http.HandlerFunc(f))
mux.HandleFunc("/", f) // the same
log.Fatal(http.ListenAndServe("localhost:8080", mux)
func f(w http.ResponseWriter, r *http.Request) {/* ... */}
ListenAndServe()
func ListenAndServe(addr string, handler Handler) error {}
// if handler == nil, use DefaultServeMux (instance of ServeMux)
http.ListenAndServe(":8080", nil)
Handlers
Go web server invokes each handler in a new goroutine, so handlers
must take precautions such as locking when accessing variables the
other goroutines may be accessing.
// FileServer returns Handler for serving static files
func FileServer(root FileSystem) Handler {}
http.Handle("/images", http.FileServer(http.Dir("./images")))
// NotFoundHandler returns Handler that replies to each request with a 404 error
func NotFoundHandler() Handler {}
// RedirectHandler returns Handler that redirects each request to url using status code
// Codes: http.StatusMovedPermanently, http.StatusFound, http.StatusSeeOther
func RedirectHandler(url string, code int) Handler {}
// StripPrefix returns Handler removes given prefix from a request and passes it to h
func StripPrefix(prefix string, h Handler) Handler {}
http.Handle("/files/", http.StripPrefix("/files/", http.FileServer(http.Dir("./"))))
// TimeoutHandler returns Handler that runs h with the given time limit
// On timeout handler responds with a http.StatusServiceUnavailable and msg in body
func TimeoutHandler(h Handler, dt time.Duration, msg string) Handler {}
Chain Handlers (middleware)
Useful for authentication.
type clientRequest struct {
Data string `json:"data"`
}
type validationHandler struct {
next http.Handler
}
func newValidationHandler(next http.Handler) http.Handler {
return validationHandler{next: next}
}
func (v validationHandler) ServeHTTP(rw http.ResponseWriter, r *http.Request) {
var request clientRequest
decoder := json.NewDecoder(r.Body)
err := decoder.Decode(&request)
if err != nil {
http.Error(rw, "Bad Request", http.StatusBadRequest)
return // stop handler chaining on error
}
v.next.ServeHTTP(rw, r)
}
func main() {
handler := newValidationHandler(http.NotFoundHandler())
http.Handle("/", handler)
http.ListenAndServe(":8080", nil)
}
Get()
/Post()
// Get follows the redirect, up to a maximum of 10 redirects.
func Get(url string) (resp *Response, err error)
resp, err := http.Get(url)
b, err := ioutil.ReadAll(resp.Body)
resp.Body.Close() // close to avoid resource leaking
// To set custom headers, use NewRequest and DefaultClient.Do.
func Post(url string, contentType string, body io.Reader) (resp *Response, err error)
context
Useful for passing data from one handler to the next w/o breaking thehttp.Handler
interface.
// Returns an empty context (used as a top-level context by the main function)
func Background() Context {}
// Returns a copy of the parent Context with a cancel function; cancel() releases
// resources associated with the context (call after all operations are complete)
func WithCancel(parent Context) (ctx Context, cancel CancelFunc) {}
// Returns a copy of the parent context that expires after the current time > deadline
// CancelFunc - method for manual context cancellation
func WithDeadline(parent Context, deadline time.Time) (Context, CancelFunc) {}
// Similar to WithDeadline, except we pass it a duration for which Context should exist
func WithTimeout(parent Context, timeout time.Duration) (Context, CancelFunc) {}
// Returns a copy of the parent context in which {key: val} is set
func WithValue(parent Context, key interface{}, val interface{}) Context {}
Context.Value(key interface{}) interface{}
For inbound requests
http.Server
automatically cancels context
when the client connection closes. For outbound requestsContext()
controls cancellation.
// Inbound request
// Context often flows across packages, so we need to create package level type
// for use with context, otherwise collisions can happen with other keys (same name)
type validationContextKey string
func (h validationHandler) ServeHTTP(rw http.ResponseWriter, r *http.Request) {
/* ... */
// Set key in new context
c := context.WithValue(r.Context(), validationContextKey("data"), request.Data)
// Apply new context
r = r.WithContext(c)
// Pass the request with new context down the chain
h.next.ServeHTTP(rw, r)
}
// Inside "next" handler:
// Retrieve value from context using key + type assertion
data := r.Context().Value(validationContextKey("data")).(string)
// Outbound request
r, err := http.NewRequest("GET", "http://site.com", nil)
if err != nil {
log.Fatal(err)
}
// r.Context() - parent context
// timeoutRequest - new context with timeout
timeoutRequest, cancelFunc := context.WithTimeout(r.Context(), 1 * time.Millisecond)
// Cancel the Context() method of the outbound request == cancel the request itself!
defer cancelFunc()
// Apply new context to r (Request.WithContext(ctx context.Context) *Request)
r = r.WithContext(timeoutRequest)
_, err = http.DefaultClient.Do(r)
if err != nil {
log.Fatal(err)
}
net/rpc
RPC Handler's methods must be:
- the method's type is exported.
- the method is exported.
- the method has two arguments, both exported (or builtin) types.
- the method's second argument is a pointer.
- the method has return type error.
// Register (publish) methods of rcvr `Type.Method`
func Register(rcvr interface{}) error {}
// Register (publish) methods of rcvr with custom type name `CustomType.Method`
RegisterName(name string, rcvr interface{}) error {}
RPC over TCP
// contract/entities.go
type RPCHandlerRequest struct {
A, B int
}
type RPCHandlerResponse struct {
Result int
}
// server.go
type RPCHandler struct{}
// We do not need to conform to an interface, all methods of RPCHandler will be registered (published) in RPC server
func (h *RPCHandler) PublicMethodSum(args *contract.RPCHandlerRequest, reply *contract.RPCHandlerResponse) error {
reply.Result = args.A + args.B
return nil
}
func main() {
err := rpc.Register(&RPCHandler{})
FatalOnErr(err)
// Create socket based on protocol binded with ip + port
l, err := net.Listen("tcp", ":8080")
FatalOnErr(err)
for {
conn, err := l.Accept()
FatalOnErr(err)
print("New connection!")
// ServeConn() is a blocking method => using goroutine
// ServeConn uses gob binary format
go rpc.ServeConn(conn)
}
}
// client.go
func main() {
c, err := rpc.Dial("tcp", ":8080")
if err != nil {
log.Fatal(err)
}
args := contract.RPCHandlerRequest{2, 3}
var reply RPCHandlerResponse
err = c.Call("RPCHandler.PublicMethodSum", args, &reply)
FatalOnErr(err)
fmt.Printf("%+v", reply)
}
With
gobs
the source and destination values and types do not need to
correspond exactly. Non-existing field will be ignored (w/o error).
RPC over HTTP
// Sets up 2 endpoints to use HTTP as a transport:
// DefaultRPCPath = "/_goRPC_"
// DefaultDebugPath = "/debug/rpc"
func HandleHTTP() {}
// server.go
err := rpc.Register(&RPCHandler{})
FatalOnErr(err)
rpc.HandleHTTP()
l, err := net.Listen("tcp", ":8080")
FatalOnErr(err)
http.Serve(l, nil)
// client.go
client, err := rpc.DialHTTP("tcp", ":8080")
// sync function invocation
err = client.Call("RPCHandler.PublicMethodSum", args, &reply)
// async function invocation
call := client.Go("RPCHandler.PublicMethodSum", args, reply, nil)
replyCall := <-call.Done // will be equal to call
With RPC over HTTP we can take advantage of HTTP headers and other
metadata
JSON-RPC over HTTP
net/rpc/jsonrpc
provides a build-in codec for
serializing/deserializing to the JSON-RPC standard.
type ClientRequest struct {
A, B int
}
type ServerResponse struct {
Result int
}
type RPCHandler struct{}
func (h *RPCHandler) Sum(args *ClientRequest, reply *ServerResponse) error {
reply.Result = args.A + args.B
return nil
}
type HttpConn struct {
in io.Reader
out io.Writer
}
// Methods to proxy calls to the reader and writer
func (c *HttpConn) Read(p []byte) (n int, err error) { return c.in.Read(p) }
func (c *HttpConn) Write(d []byte) (n int, err error) { return c.out.Write(d) }
func (c *HttpConn) Close() error { return nil }
func main() {
rpc.Register(new(RPCHandler))
l, err := net.Listen("tcp", ":8080")
if err != nil {
log.Fatal(err)
}
http.Serve(l, http.HandlerFunc(httpHandler))
}
func httpHandler(w http.ResponseWriter, r *http.Request) {
// Pass a type ReadWriteCloser
serverCodec := jsonrpc.NewServerCodec(&HttpConn{in: r.Body, out: w})
err := rpc.ServeRequest(serverCodec)
if err != nil {
log.Fatal(err)
}
}
curl -X POST -H "Content-Type: application/json" -d '{"id": 1, "method": "RPCHandler.Sum", "params": [{"A":1,"B":2}]}' http://localhost:8080
net/http/httputil
// DumpRequest dumps HTTP request in ith HTTP/1.x representation into []byte
func DumpRequest(req *http.Request, body bool) ([]byte, error) {}
/* Result:
POST /api/v3/... HTTP/1.1
Host: 10.1.1.2:3000
Accept-Encoding: gzip
Content-Length: 100
Content-Type: application/json
POST_REQUEST_BODY_HERE
*/
html
// Parse returns the parse tree for the HTML from the given Reader
func Parse(r io.Reader) (*Node, error) {}
sort
// To sort any sequence we need to define a type that implements 3 methods
type Interface interface {
Len() int
Less(i, j int) bool // i, j - indices of sequence
Swap(i, j int)
}
Sort(data Interface) {}
// Reverse - decorator, the returns Interface with inversed Less() method
ReverseSort(data Interface) Interface {}
// Makes an object sortable
type StringSlice []string
type IntSlice []int
type Float64Slice []float
func Strings(a []string) {} // Sorts a slice of strings in increasing order
type User struct {
Name string
Age int
}
type byAge []*User
func (u byAge) Len() int { return len(u) }
func (u byAge) Less(i, j int) bool { return u[i].Age < u[j].Age }
func (u byAge) Swap(i, j int) { u[i], u[j] = u[j], u[i] }
type byName []*User
func (u byName) Len() int { return len(u) }
func (u byName) Less(i, j int) bool { return u[i].Name < u[j].Name }
func (u byName) Swap(i, j int) { u[i], u[j] = u[j], u[i] }
type customSort struct {
u []*User
less func(x, y *User) bool
}
func (x customSort) Len() int { return len(x.u) }
func (x customSort) Less(i, j int) bool { return x.less(x.u[i], x.u[j]) }
func (x customSort) Swap(i, j int) { x.u[i], x.u[j] = x.u[j], x.u[i] }
func main() {
users := []*User{{"Jack", 20}, {"Alice", 22}, {"Bob", 10}, {"Nick", 15}}
sort.Sort(byAge(users))
sort.Sort(sort.Reverse(byName(users)))
sort.Sort(customSort{users, func(x, y *User) bool { return x.Name < y.Name }})
}
We use
type byAge []*user
instead oftype byAge []user
for faster
sort (sort
swaps pointers instead of values).
compress/gzip
Use https://github.com/NYTimes/gziphandler
for gzip
.
ioutil
ioutil.Discard // io.Writer on which all Write calls succeed
sync
Low-level synchronization for go routines.
sync/atomic
Used for working w/ single number to guarantee atomic operations on it.
atomic.AddInt64(&num, 5)
atomic.LoadInt64(&num)
WaitGroup
Waits for goroutines to finish.
func main() {
var wg sync.WaitGroup
wg.Add(1) // Increases internal counter (+1 go routine)
go worker(&wg)
wg.Wait() // Will block code execution here, unless counter is set to 0
println("Done!")
}
func worker(wg *sync.WaitGroup) {
wg.Done() // Decrement counter by 1
}
Mutex
Mutex - mutual exclusion lock. Can be created as a part of a struct.
type Account struct { // embed Mutex object to allow to use its Public methods on Account
sync.Mutex
balance int
}
func main() {
a := Account{balance: 0}
var wg sync.WaitGroup
wg.Add(100)
for i := 0; i < 100; i++ {
go worker(&a, &wg)
}
wg.Wait()
println(a.balance) // w/o mutex - balance can be 98, 99, 100 (data race happens)
}
func worker(account *Account, wg *sync.WaitGroup) {
account.Lock()
defer account.Unlock() // convenient if method is long (but w/o defer - faster)
account.balance += 1
wg.Done()
}
RWMutex
Non-blocking read operations on resource.
Many can read data, while no one can edit it. While anyone edits data -
no one can read it.
a.RLock() // lock resource for read
a.RLock() // lock resource for read - works
a.Lock() // lock resource for edit - doesn't work
a.RUnlock()// release resource
a.Lock() // lock resource for edit - works
Map
Concurrent map with constant-time load, save, delete operations. Doesn't
require Mutex. Must not be copied - use pointer.
func main() {
var m sync.Map
for i := 0; i < 3; i++ {
go func(i int) {
for j := 0; ; j++ {
m.Store(i, j)
}
}(i)
}
for i := 0; i < 10; i++ {
m.Range(func(key, value interface{}) bool {
fmt.Printf("%d: %d\t", key, value)
return true
})
fmt.Println()
time.Sleep(time.Second)
}
}
time
start := time.Now()
/* ... */
secs := time.Since(start).Seconds()
func WaitForServer(url string) error {
const timeout = 1 * time.Minute
deadline := time.Now().Add(timeout)
for tries := 0; time.Now().Before(deadline); tries++ {
_, err := http.Head(url)
if err == nil {
return nil // success
}
log.Printf("server not responding (%s); retrying...", err)
time.Sleep(time.Second << uint(tries)) // exponential back-off
}
return fmt.Errorf("server %s failed to respond after &s", url, timeout)
}
runtime
runtime.GOMAXPROCS(runtime.NumCPU()) // activate all CPU cores
runtime.Gosched() // Allow runtime to pass control to other routine
Gosched()
is important in concurrent programs while working in big
loops
flag
// Args returns the non-flag command-line arguments.
func Args() []string {}
Bool defines a bool flag with specified name, default value, and usage string.
func Bool(name string, value bool, usage string) *bool
// Bool creates new flag variable
var v = flag.Bool("v", false, "enable verbose mode")
var sep = flag.String("s", " ", "separator")
func main() {
flag.Parse() // update the flag variables from their default values
if *v {
fmt.Println("verbose mode enabled")
}
fmt.Print(strings.Join(flag.Args(), *sep))
}
flag.Duration()
creates a flag variable of typetime.Duration
and
allows the user to specify the duration in a variety of user-friendly
formats:./cmd -flag 2m30s
.
os
os.Args // []string containing CL arguments starting from CMD name itself
os.Getwd // Working directory
// Open returns *File that is used in subsequent reads by the bufio.NewScanner(*os.File)
func Open(name string) (*File, error) {}
// RemoveAll removes path and any children it contains.
func RemoveAll(path string) error {}
Failure to check the result of the close
operation could cause serious
data loss to go unnoticed.
f, err := os.Create("file.txt")
if err != nil {
return err
}
io.Copy(f, buf)
// Close file
err := f.Close()
if err != nil {
return err
}
Network
TCP echo server (Async)
func main() {
listener, err := net.Listen("tcp", ":3000") // Bind (connect + listen on OS Port)
if err != nil {
fmt.Println("Problem w/ creating server")
}
for {
conn, err := listener.Accept()
if err != nil {
fmt.Println("Problem w/ connection")
conn.Close()
continue
}
fmt.Println("Connected")
bufReader := bufio.NewReader(conn)
go func() {
defer conn.Close()
for {
// Reading from socket by bytes
rbyte, err := bufReader.ReadByte()
if err != nil {
fmt.Println("Problem w/ reading from socket")
break
}
fmt.Println(string(rbyte))
conn.Write([]byte{rbyte})
}
}()
}
}
HTTP Server
func main() {
// Map request to a handler function
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
path := r.URL.Path
c := &http.Client{} // create HTTP Client w/ default transport
// Make request
response, err := c.Get("http://setgetgo.com/get.php?Id=" + path[1:])
if err != nil {
log.Println(err)
}
defer response.Body.Close() // Connection is opened => Close Body after read, otherwise leaks may appear
body, _ := ioutil.ReadAll(response.Body) // Read until EOF, return []byte
w.WriteHeader(http.StatusOK)
w.Write(body)
})
http.ListenAndServe(":3000", nil)
}
Server port can be set from
CLI
or fromenv
.
JSON API Server
type apiRequest struct {
Name string `json:"name"`
}
type apiResponse struct {
Greeting string `json:"greeting"`
}
func main() {
http.HandleFunc("/json", jsonHandler)
log.Fatal(http.ListenAndServe(":8080", nil))
}
func jsonHandler(w http.ResponseWriter, r *http.Request) {
var request apiRequest
decoder := json.NewDecoder(r.Body)
err := decoder.Decode(&request)
if err != nil {
log.Fatal(err)
}
w.Header().Set("Content-Type", "application/json")
encoder := json.NewEncoder(w)
err = encoder.Encode(apiResponse{"Hello " + request.Name})
if err != nil {
log.Fatal(err)
}
}
Templates
text/template
Template - string or file containing one or more portions enclosed
in double braces {{...}}
called actions.
.
- current value{{range .Items}}... {{end}}
- loop.Name | func
- pipe field as parameter to a function
Producing output with a template:
- Parse the template into internal representation (done only once)
- Execute it on specific inputs
package main
import (
"text/template"
"time"
"os"
"log"
)
const templ = `Car List ({{len . }}):
{{range .}}----------------------------------------
Model: {{.Model | printf "%.30s"}}
Age: {{.Year | countAge}} years
Condition: {{.Condition}}
Owner: {{.Owner.Name}} ({{.Owner.Phone}})
{{end}}`
type Owner struct {
Name string
Phone string
}
type Car struct {
Model string
Year int
Condition string
Owner *Owner
}
func countAge(year int) int {
return time.Now().Year() - year
}
// template.Must accepts a template and an error, checks that the error is nil and returns a template
var report = template.Must(template.New("carlist").
Funcs(template.FuncMap{"countAge": countAge}).
Parse(templ))
func main() {
cars := []Car{
{"Volvo", 2000, "Used", &Owner{"Jack", "0123456"}},
{"Lexus", 2010, "New", &Owner{"Helen", "0123456"}},
}
// cars will be `.` in template
if err := report.Execute(os.Stdout, cars); err != nil {
log.Fatal(err)
}
}
Car List (2):
----------------------------------------
Model: Volvo
Age: 17 years
Condition: Used
Owner: Jack (0123456)
----------------------------------------
Model: Lexus
Age: 7 years
Condition: New
Owner: Helen (0123456)
html/template
Uses the same API as text/template
but adds features for automatic
escaping of strings for HTML, JS, CSS or URL.
Inline
func main() {
t1 := template.New("my-template") // Allocate new HTML template
t1, _ = t1.Parse(`<div>{{.}}</div>`)
t1.Execute(os.Stdout, "Inserted text")
t2 := template.New("condition")
t2, _ = t2.Parse(`{{if ne . "Hello"}}Not {{end}}Hello`)
t2.Execute(os.Stdout, "Hello")
}
Separate file
Template in separate file template.html
:
<html>
<head>
<title>Shopping List</title>
</head>
<body>
<div class="root">
{{if CheckAllInactive .}}
<div>All items were bought.</div>
{{else}}
<div class="title">Items to buy:</div>
{{range $index, $value := .}}
<div class="item">
<div class="item-name">{{.Name}}</div>
<form action="/" method="post">
<input type="hidden" name="id" value="{{$index}}"/>
<input type="submit" {{if eq .Active false}}disabled{{end}} value="Bought!"/>
</form>
</div>
{{end}}
{{end}}
</div>
</body>
</html>
type Item struct {
Name string
Active bool
}
func CheckAllInactive(items []Item) bool {
for index := range items {
if items[index].Active {
return false
}
}
return true
}
func main() {
items := []Item{
{"Bread", true},
{"Eggs", true},
{"Milk", true},
{"Butter", true},
}
t1, err := template.New("template.html").Funcs(template.FuncMap{"CheckAllInactive": CheckAllInactive}).ParseFiles("template.html")
if err != nil {
log.Fatal("Can't expand template", err)
return
}
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodPost {
index, _ := strconv.ParseInt(r.FormValue("id"), 10, 0) // urlencoded -> string -> int
items[index].Active = false
}
err = t1.Execute(w, items)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
})
http.ListenAndServe(":3000", nil)
}
Cookies
Set-Cookie: session_id=ow42234e88vhg0f83hfs93r32' Expires=Mon, 4 Mar
2018 12:00:00 GMT
func main() {
sessions := make(map[string]string)
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
sessionID, err := r.Cookie("session_id")
if err == http.ErrNoCookie {
w.Write([]byte(formTemplate))
return
} else if err != nil {
log.Panic(err)
}
username, ok := sessions[sessionID.Value]
if !ok {
fmt.Fprint(w, "Session not found.")
} else {
fmt.Fprintf(w, "Welcome, "+username)
}
})
http.HandleFunc("/get-cookie", func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodPost {
r.ParseForm()
login := r.Form["login"][0]
sessionId := strconv.Itoa(rand.Int())
expiration := time.Now().Add(365 * 24 * time.Hour)
cookie := http.Cookie{
Name: "session_id",
Value: sessionId,
Expires: expiration,
}
sessions[sessionId] = login
http.SetCookie(w, &cookie) // Cookie is set in headers, all content must be below
http.Redirect(w, r, "/", http.StatusFound)
}
})
http.ListenAndServe(":3000", nil)
}
DataBase
MySQL
MySQL Driver: import _ "github.com/go-sql-driver/myslq"
Connect
// Create structure of the DB
// No connections to the DB here
// Can use `interpolateParams=true` to change PREPARE+EXECUTE => QUERY
db, err := sql.Open("mysql", "root:password@tcp(localhost:3306)/testdb?charset=utf8&interpolateParams=true")
if err != nil {
panic(err)
}
db.SetMaxOpenConns(10) // Optional; 10 connections can easily process 8000+ fast QPS
// Connect to the DB
if err = db.Ping(); err != nil {
panic(err)
}
fmt.Println("Open connections to the DB:", db.Stats().OpenConnections)
Query()
for SELECT
many
// Query() returns multiple results
// Requires closing connection to DB
rows, err := db.Query("SELECT name, info FROM users")
// Iterate over rows
for rows.Next() {
var name string
var info sql.NullString // Now we can differentiate NULL and "" value returned from DB
if err := rows.Scan(&name, &info); err != nil { // Populates `name` and `info` variables
panic(err)
}
if info.Valid {
fmt.Println("User:", name, info.String)
} else {
fmt.Println("User:", name, "No info") // `info` value is NULL
}
}
rows.Close() // close connection, otherwise - leaks
QueryRow()
for SELECT
one
// QueryRow() returns single result
// Auto closes connection to DB
row := db.QueryRow("SELECT name, info FROM users WHERE id=?", 1)
var name string
var info string
if err := row.Scan(&name, &info); err != nil {
panic(err)
}
fmt.Println("User:", name, info)
Exec()
for INSERT
/UPDATE
// Exec() executes query and returns num of changed rows
result, err := db.Exec("INSERT INTO users (name, info) VALUES (?, ?)", "Oleg Legun", "Some information")
if err != nil {
panic(err)
}
affected, err := result.RowsAffected()
if err != nil {
panic(err)
}
lastId, err := result.LastInsertId()
if err != nil {
panic(err)
}
fmt.Println("Inserted values:", affected, "Last ID:", lastId)
Prepared statements: Prepare()
+ Exec()
// Prepared statements
// Send request to DB (PREPARE)
stmt, err := db.Prepare("UPDATE users SET info = ? WHERE id = ?")
if err != nil {
panic(err)
}
// Now DB is waiting for values: info, id (EXECUTE)
result, err = stmt.Exec("Updated information", 5)
if err != nil {
panic(err)
}
affected, err = result.RowsAffected()
if err != nil {
panic(err)
}
fmt.Println("Updated values:", affected)
PostgreSQL
Postgres driver import _ "github.com/lib/pq"
Connect
// Create structure of the DB
// No connections to the DB here
db, err := sql.Open("postgres", "user=postgres password=password dbname=testdb sslmode=disable")
if err != nil {
panic(err)
}
db.SetMaxOpenConns(10) // Optional; 10 connections can easily process 8000+ fast QPS
// Connect to the DB
if err = db.Ping(); err != nil {
panic(err)
}
fmt.Println("Open connections to the DB:", db.Stats().OpenConnections)
Query()
for SELECT
many
// Query() returns multiple results
// Requires closing connection to DB
rows, err := db.Query("SELECT name, info FROM users")
// Iterate over rows
for rows.Next() {
var name string
var info sql.NullString // Now we can differentiate NULL and "" value returned from DB
if err := rows.Scan(&name, &info); err != nil { // Populates `name` and `info` variables
panic(err)
}
if info.Valid {
fmt.Println("User:", name, info.String)
} else {
fmt.Println("User:", name, "No info") // `info` value is NULL
}
}
rows.Close() // close connection, otherwise - leaks
QueryRow()
for SELECT
one
// QueryRow() returns single result
// Auto closes connection to DB
row := db.QueryRow("SELECT name, info FROM users WHERE id=$1", 1)
var name string
var info string
if err := row.Scan(&name, &info); err != nil {
panic(err)
}
fmt.Println("User:", name, info)
QueryRow()
for INSERT
+ LastInsertId()
// QueryRow() is used for INSERT queries, when we need to return lastId (pq doesn't support LastInsertId())
// Auto closes connection to DB
var lastId int64
err = db.QueryRow(
"INSERT INTO users (name, info) VALUES ($1, $2) RETURNING id",
"John",
"Some Information",
).Scan(&lastId)
if err != nil {
panic(err)
}
fmt.Println("Last insert Id:", lastId)
Exec()
for INSERT
/UPDATE
// Exec() executes query and returns num of changed rows
result, err := db.Exec(
"UPDATE users SET name = $1, info = $2 WHERE id = $3",
"Oleg Legun",
"Updated information",
4)
if err != nil {
panic(err)
}
affected, err := result.RowsAffected()
if err != nil {
panic(err)
}
fmt.Println("Updated values:", affected)
Prepared statements: Prepare()
+ Exec()
// Prepared statements
// Send request to DB (PREPARE)
stmt, err := db.Prepare("UPDATE users SET info = $1 WHERE id = $2")
if err != nil {
panic(err)
}
// Now DB is waiting for values: info, id (EXECUTE)
result, err = stmt.Exec("Updated information", 5)
if err != nil {
panic(err)
}
affected, err = result.RowsAffected()
if err != nil {
panic(err)
}
fmt.Println("Updated values:", affected)
Memcached
Very efficient noSQL DB.
import "github.com/bradfitz/gomemcache/memcache"
var (
memcacheClient *memcache.Client // Make it global - useful for functions
)
func getRecord(mkey string) *memcache.Item {
println("get", mkey)
item, err := memcacheClient.Get(mkey)
if err == memcache.ErrCacheMiss {
fmt.Println("Record not found.")
return nil
} else if err != nil {
panic(err)
}
return item
}
func main() {
MemcachedAddresses := []string{"127.0.0.1:11211"}
memcacheClient = memcache.New(MemcachedAddresses...)
// Get record
mkey := "user_123"
item := getRecord(mkey)
fmt.Printf("Item:%+v\n", item)
// Add record
ttl := 5 // Seconds
//ttl := 86400 // very useful for User sessions 24H
// Set() - add record anyway
err := memcacheClient.Set(&memcache.Item{
Key: mkey,
Value: []byte("Oleg"),
Expiration: int32(ttl),
})
if err != nil {
panic(err)
}
// Add() - add record if not exist
err = memcacheClient.Add(&memcache.Item{
Key: "counter",
Value: []byte("1"),
Expiration: int32(ttl),
})
if err == memcache.ErrNotStored {
fmt.Println("Record not stored.") // Record was added before
} else if err != nil {
panic(err)
}
// Increment()
newValue, err := memcacheClient.Increment("counter", 10)
fmt.Println(newValue)
newValue, err = memcacheClient.Decrement("counter", 5)
fmt.Println(newValue)
// GetMulti(): Get multiple records
mkeys := []string{mkey, "counter"}
items, err := memcacheClient.GetMulti(mkeys)
if err != nil {
panic(err)
}
fmt.Println("Map with multiple items:", items)
}
Redis
Heroku recommends import "github.com/garyburd/redigo/redis"
Cache methodology:
- Check for key in Redis
- If key does not exist
- Rebuild cache: Make SQL query and save result to Redis (ttl n sec)
- Rebuild cache: Make SQL query and save result to Redis (ttl n sec)
var (
c redis.Conn
)
func getRecord(mkey string) string {
fmt.Println("get", mkey)
// Parse result (command reply) to a string
item, err := redis.String(c.Do("GET", mkey))
if err == redis.ErrNil {
fmt.Println("Record not found (nil).")
return ""
} else if err != nil {
panic(err)
}
return item
}
func main() {
var err error
// Connect
c, err = redis.DialURL("redis://user:@localhost:6379/0")
if err != nil {
panic(err)
}
defer c.Close()
// Get Record
mkey := "name"
item := getRecord(mkey)
fmt.Println("Record:", item)
// Add Record
ttl := 60
result, err := redis.String(c.Do("SET", "counter", 1, "EX", ttl))
if err != nil {
panic(err)
}
if result != "OK" {
panic("result not OK: " + result)
}
// Increment (INCR/INCRBY/DECRBY)
// If counter key is not set - Redis will create it with value: 0
counter, err := redis.Int(c.Do("INCR", "counter"))
if err != nil {
panic(err)
}
fmt.Println("Counter:", counter)
// MGET - Get multiple records
keys := []interface{}{mkey, "counter", "nonexisting"}
reply, err := redis.Strings(c.Do("MGET", keys...))
if err != nil {
panic(err)
}
fmt.Println(reply) // [Oleg Legun 1 nil]
}
Cache Invalidation
If cache is expired (key doesn't exist), and several processes are going
to rebuild it, we need to use locks, otherwise - DB will be overloaded!
Set new record like mkey+"lock"
while rebuilding cache.
// If key was not found, lock it and try to rebuild it
result, err := redis.String(c.Do("SET", mkey+"_lock", name, "PX", 50, "NX"))
if result != "OK" {
fmt.Println("Other user is rebuilding it!")
}
// Rebuild cache
// Delete _lock if operation was quicker than expected
n, err := redis.Int(c.Do("DEL", mkey+"lock"))
MongoDB
MongoDB driver: import "gopkg.in/mgo.v2"
import (
"fmt"
"gopkg.in/mgo.v2"
"gopkg.in/mgo.v2/bson"
)
// MongoDB can have objects with any structure inside.
// In Go we need to explicitly describe each structure to successfully deserialize it.
// json is used when we serialize to JSON, bson - to Binary JSON (MongoDB type)
// Any extra keys that are present in MongoDB, but not in the `user` struct will be omitted
type user struct {
ID bson.ObjectId `json:"id" bson:"_id"` // _id - primary key (_) in MongoDB
Name string `json:"name" bson:"name"`
Info string `json:"info" bson:"info"`
}
var (
sess *mgo.Session
)
func main() {
var err error
sess, err = mgo.Dial("mongodb://localhost")
if err != nil {
panic(err)
}
// Collection is like a table in MySQL
collection := sess.DB("testdb").C("users")
// Create index
index := mgo.Index{
Key: []string{"name"},
}
// Save index
err = collection.EnsureIndex(index)
if err != nil {
panic(err)
}
// Add document
if n, _ := collection.Count(); n == 0 {
firstUser := &user{bson.NewObjectId(), "Oleg", "New information"}
err = collection.Insert(firstUser)
if err != nil {
panic(err)
}
}
// Get values
var allUsers []user
// Find all documents in collection (bson.M{empty} == ALL) and save them to allUsers
err = collection.Find(bson.M{}).All(&allUsers)
if err != nil {
panic(err)
}
for i, v := range allUsers {
fmt.Printf("user[%d]: %+v\n", i, v)
}
id := bson.NewObjectId()
// bson.M{"_id": id} - set find condition
var notExistentUser user
// Find() returns *Query, .One() will Unmarshall values to &newUser
err = collection.Find(bson.M{"_id": id}).One(¬ExistentUser)
if err != nil {
//panic(err)
}
// Insert new document
user1 := &user{id, "Jack", "Incorrect Information"}
err = collection.Insert(user1)
if err != nil {
panic(err)
}
// Edit retrieved document
user1.Info = "Correct Information"
collection.Update(bson.M{"_id": user1.ID}, &user1)
// Edit documents inside MongoDB
collection.UpdateAll(
bson.M{"name": "Jack"}, // Filter condition WHERE (for all Jacks) or bson.M{"_id": id} - 1 document
bson.M{
"$set": bson.M{"info": "Correct Information"},
},
)
}
ORM
Not as effective as the traditional SQL queries. Can be slow for
high-loaded services.
ORM can generate suboptimal queries: we need 1 field, orm pulls all
fields.
Complex queries with several JOIN
- in ORM can be very slow or even
not possible.
Values from SQL tables are automatically mapped to Go structures.
ORM is excellent for UPDATE'ing single user record (not for SELECT!).
import (
"fmt"
"github.com/jinzhu/gorm"
_ "github.com/jinzhu/gorm/dialects/mysql"
)
var (
db *gorm.DB
)
type User struct {
ID uint `sql:"AUTO_INCREMENT" gorm:"primary_key"`
Name string
Info string
}
func (u *User) TableName() string { // For GORM
return "users"
}
func (u *User) BeforeSave() (err error) { // For GORM
fmt.Println("Trigger on before save")
return
}
func PrintByID(id uint) {
u := User{}
err := db.Find(&u, id).Error // SELECT * FROM users WHERE id = ?
if err == gorm.ErrRecordNotFound {
fmt.Println("Record no found", id)
} else if err != nil {
panic(err)
}
fmt.Printf("PrintByID: %v, data: %+v\n", id, u)
}
func main() {
var err error
// Connect
db, err = gorm.Open("mysql", "root:password@tcp(localhost:20000)/testdb?charset=utf8")
if err != nil {
panic(err)
}
defer db.Close()
db.DB() // ORM requires explicit connection to DB
db.DB().Ping()
PrintByID(1)
PrintByID(100)
// Select all records
all := []User{} // SELECT * FROM users
db.Find(&all)
for i, v := range all {
fmt.Printf("User %d: %+v\n", i, v)
}
// Create new record
newUser := User{
Name: "Jack",
Info: "Some information",
}
db.NewRecord(newUser) // => returns `true` as primary key is blank
db.Create(&newUser)
PrintByID(newUser.ID)
// Update record
newUser.Info = "New information"
db.Save(&newUser)
PrintByID(newUser.ID)
}
Tests
Unit tests - most valuable tests.
Command | Description |
---|---|
go test | execute *_test.go files |
go test ./handlers | Test package handlers |
go test -v | verbose mode (prints t.Log() ) |
go test -run TestDivide | execute only *TestDivide* test case (supports RegExp) |
go test -cover | Show coverage percentage |
go test -coverprofile=coverage.out | Export coverage report to cover.out |
go tool cover -func=coverage.out | Coverage by functions based on cover.out |
go tool cover -html=coverage.out | Open in Browser |
go tool cover -html=coverage.out -o coverage.html | Convert .out -> .html |
Simple
package main
import "github.com/pkg/errors"
type User struct {
Name string
Surname string
}
func Sum(a int, b int) int {
return a + b
}
func AddUser(usersPtr *[]User, name string, surname string) {
*usersPtr = append(*usersPtr, User{name, surname})
}
func Divide(a int, b int) (int, error) {
if b == 0 {
return 0, errors.New("Zero division error.")
}
return a / b, nil
}
package main
import (
"github.com/pkg/errors"
"reflect"
"testing"
)
func TestSum(t *testing.T) {
if Sum(1, 2) != 3 {
// Errorf() generates an error and continues to execute current test case
t.Errorf("Expected %d, got %d", 3, Sum(1, 2))
}
}
func TestAddUser(t *testing.T) {
users := []User{}
AddUser(&users, "Oleg", "Legun")
if len(users) != 1 {
// Fatalf() generates an error and returns from current function
t.Fatalf("Expected len = %d, got %d", 1, len(users))
}
expected := []User{
{
Name: "Oleg",
Surname: "Legun",
},
}
if !reflect.DeepEqual(users, expected) {
t.Errorf("Expected %+v, got %+v", expected, users)
}
}
// Table testing
func TestTable(t *testing.T) {
t.Skip()
testCases := []struct {
A int
B int
Result int
Err error
}{
{10, 5, 2, nil},
{-10, 2, -5, nil},
{10, 0, 0, errors.New("Zero division error.")},
}
for _, testCase := range testCases {
res, err := Divide(testCase.A, testCase.B)
if res != testCase.Result {
t.Errorf("Divide(%d, %d) => %d, want %d", testCase.A, testCase.B, res, testCase.Result)
}
if err != nil && err.Error() != testCase.Err.Error() {
t.Errorf("Divide(%d, %d) => Error: %v, want Error: %v", testCase.A, testCase.B, err.Error(), testCase.Err.Error())
}
}
}
t.Parallel()
- makes tests execution concurrentt.Log("Message")
displays message in verbose mode-v
.t.Skip()
skips current test.
Test HTTP handlers
To make code testable:
- break the code into pure functions
keep all external data in separate files, that can be mocked
PROJECT ├── main/ │ └── main.go └── handlers/ ├── adduser.go ├── adduser_test.go ├── handlers.go └── index.go
// main.go
package main
import (
"../handlers"
"net/http"
)
func main() {
handler := handlers.Handler{
Users: &[]handlers.User{},
}
http.HandleFunc("/", handler.HandleIndex)
http.HandleFunc("/adduser", handler.HandleAddUser)
http.ListenAndServe(":3000", nil)
}
// handlers/handlers.go
package handlers
type User struct {
Name string `json:"name"`
Surname string `json:"surname"`
Age uint8 `json:"age"`
}
type Handler struct {
Users *[]User
}
// handlers/index.go
package handlers
import (
"encoding/json"
"log"
"net/http"
)
func (h Handler) HandleIndex(w http.ResponseWriter, r *http.Request) {
switch r.Method {
case http.MethodGet:
json, err := json.Marshal(*h.Users)
if err != nil {
log.Panic("JSON marshalling error")
}
w.Header().Set("Content-Type", "application/json")
w.Write([]byte(json))
case http.MethodPost:
w.WriteHeader(http.StatusMethodNotAllowed)
}
}
// handlers/adduser.go
package handlers
import (
"log"
"net/http"
"strconv"
)
func (h Handler) HandleAddUser(w http.ResponseWriter, r *http.Request) {
switch r.Method {
case http.MethodGet:
q := r.URL.Query()
name := q["name"][0]
surname := q["surname"][0]
age, err := strconv.Atoi(q["age"][0])
if err != nil {
log.Panic("Invalid age.")
}
*h.Users = append(*h.Users, User{name, surname, uint8(age)})
case http.MethodPost:
w.Write([]byte("Not implemented yet."))
default:
w.WriteHeader(http.StatusMethodNotAllowed)
}
}
// handlers/adduser_test.go
package handlers
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
)
func TestHandler_HandleAddUser(t *testing.T) {
// Create handler w/ test data
h := Handler{
Users: &[]User{},
}
// Create handler from a function
handler := http.HandlerFunc(h.HandleAddUser)
r, err := http.NewRequest("GET", "/adduser?name=Oleg&surname=Legun&age=26", strings.NewReader(""))
if err != nil {
t.Errorf("Error %v", err)
}
// recorder keeps all http requests
w := httptest.NewRecorder()
handler.ServeHTTP(w, r)
if w.Code != http.StatusOK {
t.Fatalf("Expected %d, got %d", http.StatusOK, w.Code)
}
}
func TestHandler_HandleIndex(t *testing.T) {
// Create handler w/ test data
handler := Handler{
Users: &[]User{
{"Oleg", "Legun", 26},
},
}
// Create handler from a function
h := http.HandlerFunc(handler.HandleIndex)
r, err := http.NewRequest("GET", "/", strings.NewReader(""))
if err != nil {
t.Errorf("Error %v", err)
}
// recorder keeps all http requests
w := httptest.NewRecorder()
h.ServeHTTP(w, r)
if w.Code != http.StatusOK {
t.Fatalf("Expected %d, got %d", http.StatusOK, w.Code)
}
expected := `[{"name":"Oleg","surname":"Legun","age":26}]`
// Can also use DeepEqual() instead of a string
if w.Body.String() != expected {
t.Errorf("Expected %s, got %s", expected, w.Body.String())
}
}
Mocks
https://github.com/golang/mock
We can test how our program works w/ external dependency by using a
generated mock based on its interface.
Package must be available on GOPATH.
Command | Description |
---|---|
mockgen -destination=./mocks/users_mock.go users UserInterface | Generate mocks based on UserInterface from users package |
// External Dependency `/users` on GOPATH
package users
type User struct {
ID int
Name string
}
func (u User) GetName() string {
return u.Name
}
func (u *User) SetName(name string) {
u.Name = name
}
type UserInterface interface {
GetName() string
SetName(string)
}
// users_test.go
package users
import (
"./mocks"
"./users"
"fmt"
"github.com/golang/mock/gomock"
"testing"
)
func SetAndGetName(u users.UserInterface) {
u.SetName("Oleg") // 1st operation
name := u.GetName()
fmt.Println(name) // 2nd operation
}
func TestSetAndGetName(t *testing.T) {
mockCtrl := gomock.NewController(t) // pass `t` to enable logging to the console if gomock fails
defer mockCtrl.Finish() // Check all methods calls after SetAndGetName() finishes and log PASS/FAIL
testUser := mocks.NewMockUserInterface(mockCtrl)
// Record a particular sequence of methods calls, that must be executed
testUser.EXPECT().SetName("Oleg") // record 1st operation
testUser.EXPECT().GetName().Return("Oleg") // record 2nd operation
SetAndGetName(testUser) // Pass `testUser` mock to the function to test it
}