2022.09.27

Go

Rob Pike you sick son of a bitch

Too close for missles, switching to guns.

Syntax

Conditionals

if num := 9; num < 0 {
    fmt.Println(num, "is negative")
} else if num < 10 {
    fmt.Println(num, "has 1 digit")
} else {
    fmt.Println(num, "has multiple digits")
}

Iterations

for i := 0; i < 10; i++ {
    sum += i
}

full for syntax

// range returns index and value of a slice
for i, v := range pow {
    fmt.Printf("2**%d = %d\n", i, v)
}

// can use on a string to iterate runes
for i, c := range "abc" {
    fmt.Println(i, " => ", string(c))
}

range syntax

for sum < 1000 {
    sum += sum
}

for is go’s while

for i := range unfilteredSpan {
    count := 0
    for _, n := range unfilteredSpan[i].neighbors {
        if nodes[n].distance > request.MinNeighborDistance {
            count++
        }
    }

    unfilteredSpan[i].distantNeigbors = count
}

must use index to update the struct of an array of structs

Slices and Arrays

// Array literal
b := [2]string{"Penn", "Teller"}

// Have the compiler do the counting
b := [...]string{"Penn", "Teller"}

// Slice literal
letters := []string{"a", "b", "c", "d"}

// make function
s := make([]byte, 5)

// append slice to slice
s3 := append(s2, s0...)

// slice a slice
// 0 <= low <= high <= cap(a)
s[inclusive:exclusive]

// pass pointer for recursive append
backtrack(candidates, target, current, &combinations)
...
func backtrack(candidates []int, target int, current []int, combinations *[][]int)
...
*combinations = append(*combinations, tmp)

// copy slice
tmp := make([]int, len(current))
copy(tmp, current)

// delete index
a = append(a[:i], a[i+1:]...)

[:]

Maps

// with make
m = make(map[string]int)
// literal
m = map[string]int{}

// check existence
i, ok := m["route"]
// more compressed
if val, ok := dict["foo"]; ok {
    //do something here
}

// maps can be used for sets
m := make(map[rune]bool)

// iterate over key values
for k, v := range m { 
    fmt.Printf("key[%s] value[%s]\n", k, v)
}
// or just keys
for k := range m { 
    fmt.Printf("key[%s]\n", k)
}

This causes non-obvious issues when trying to assign a value to a struct in a map of structs. Easiest work around seems to be to use a map of pointers to structs.

Types

// unnamed type
var x struct{ I int }

// named type
type Foo struct{ I int }
var y Foo
type person struct {
    name string
    age  int
}

p := person{name: name}

structs

const and var

const Pi = 3.14

doesn’t use :=

var (
    home   = os.Getenv("HOME")
    user   = os.Getenv("USER")
    gopath = os.Getenv("GOPATH")
)

no type needed

alias declarations and type definitions

type (
	Name = string
	Age  = int
)

alias declaration binds an identifier to the given type

type definition has no =

Type definitions may be used to define different boolean, numeric, or string types and associate methods with them

Generics

Go lets you express behavioral polymorphism with interfaces, but until generics land, you essentially can’t express state polymorphism. So don’t try. Seriously. Whatever hacks you have to do with interface{} or reflect or whatever are worse than the disease. The empty interface{} and package reflect are tools of last resort, for when there’s literally no other way to solve your problem. In practice, that means in library code that needs to operate on types that are unknown at compile time. In application code, by definition, you know the complete set of types you’ll ever have to deal with at compile time. So there’s almost never a reason to use either of these hacks. You can always write type-specific versions of everything you need. Do that.

In the above quote, behavioral ≈ subtyping and state ≈ parametric. I think the heap library is a good example of library code that doesn’t know the type it will be operating on.

make vs. new

slice, map and chan are data structures. They need to be initialized, otherwise they won’t be usable.

p := new(chan int)   // p has type: *chan int
c := make(chan int)  // c has type: chan int

// make only inits the top level data structure
cache := make([]map[int]int, len(nums))
for i,_ := range cache {
	cache[i] = make(map[int]int, 0)
}

type assertion

t := i.(T)

panic if i is not T, change to T if it is

Pointers

i := 42
p = &i
fmt.Println(*p) // read i through the pointer p
*p = 21         // set i through the pointer p

// pointers to builtins (e.g. slices) need to be de-referenced before using syntax (unlike normal structs)
(*traversal)[level] = append((*traversal)[level], root.Val)

pointers in action

Precedence and Association

Functions

func hello(a int, b ...int) {  
}

// can take 0 or more args

variadic function

return value or pointer?

Methods (Receivers)

type Vertex struct {
	X, Y float64
}

func Abs(v Vertex) float64 {
	return math.Sqrt(v.X*v.X + v.Y*v.Y)
}

can access values of v, but can’t change them cause passed by value

func (v *Vertex) Scale(f float64) {
	v.X = v.X * f
	v.Y = v.Y * f
}

can access values of v and change them

with interfaces
type Foo interface {
    foo()
}

type Bar struct {}

func (b Bar) foo() {}

func main() {
	// type pointer of Bar (*Bar)
	var foo Foo = &Bar{}
	// works, go can pass the value receiver from by dereferencing the pointer's type
	foo.foo()

	// type Bar
	var foo Foo = Bar{}
	// works
	foo.foo()
}

value receiver

type Foo interface {
    foo()
}

type Bar struct {}

func (b *Bar) foo() {}

func main() {
	// type pointer of Bar (*Bar)
	var foo Foo = &Bar{}
	// works
	foo.foo()

	// type Bar
	var foo Foo = Bar{}
	// DOES NOT WORK! Foo interface's value is not addressable, go can't pass a pointer to the method receiver
	foo.foo()
}

pointer receiver

nil receiver
16:00:42 <yonson> whats the reason for checking if `m != nil` in this case? https://play.golang.org/p/zGoRwTc3g77
16:03:26 <fizzie> It's not entirely uncommon to make methods callable on a nil receiver.
16:03:32 <fizzie> All proto accessors do that, for example.
16:04:02 <fizzie> And of course if you want to do that, you need to have a `m != nil` check in order for the `m.getUserFn` access not to panic.
16:05:28 <b0nn> hmm, you shouldn't be able to call that function if m is nil
16:05:44 <fizzie> No, you're perfectly able to call that method even if m is nil.
16:06:10 <fizzie> As long as you have a value of type `*userTeamMock`, you can call the method on it.
16:07:30 <fizzie> https://play.golang.org/p/CNRLk7yEEt3 and so on.
16:08:25 <fizzie> I mean, you can certainly argue making methods not panic on a nil pointer receiver is a bad idea and you shouldn't do it, if that's what you meant. But the language doesn't have any rule against it.

some real life IRC on https://play.golang.org/p/zGoRwTc3g77

package main

import (
	"fmt"
)

type example struct {
	i int
}

func (e *example) foo() int {
	if e == nil {
		return 123
	}
	return e.i
}

func main() {
	var e *example
	fmt.Println(e.foo())
}

// returns 123
embedding

it is a union of the embedded interfaces. Only interfaces can be embedded within interfaces.

Embedding types introduces the problem of name conflicts but the rules to resolve them are simple. First, a field or method X hides any other item X in a more deeply nested part of the type. If log.Logger contained a field or method called Command, the Command field of Job would dominate it.

Interfaces

Strings and Runes

Rob Pike’s blog post on strings

a string holds arbitrary bytes. It is not required to hold Unicode text, UTF-8 text, or any other predefined format. As far as the content of a string is concerned, it is exactly equivalent to a slice of bytes.

Characters in golang are called runes, but they are not the characters (aka 2 bytes) of old. They try to cover up some of the ambiguity of utf8 (more bytes).

In Go rune type is not a character type, it is just another name for int32

indexing a string yields its bytes, not its characters: a string is just a bunch of bytes

Most deterministic way to iterate per “character” is to use range to get the runes (plus beginning byte index). ASCII is single byte encoding, so if it can be assumed, can use more general slicing.

// use of rune type
m := make(map[rune]int)

can treat strings like slices to get substrings

s = s[:len(s)-1]

// have to convert byte to string in order to append
s + string(c)

// type byte
'1'
// type string
"1"

Errors

// the error interface
type error interface {
    Error() string
}

// errorString is a trivial implementation of error.
type errorString struct {
    s string
}

func (e *errorString) Error() string {
    return e.s
}

// New returns an error that formats as the given text.
func New(text string) error {
    return &errorString{text}
}

this type is what errors.New() builds

fmt.Errorf("error parsing something: %w", err)

wrap (annotate) errors with %w to see where they come from, easier to debug

The pattern in Go is for functions to return an error interface, not a struct, which goes against the “accept interfaces, return structs” idiom. Why is this? The key is that an interface type holds a concrete value and concrete type, both of which have to be nil for the interface to be nil. If a function returns a type instead of an interface, it will always be a non-nil interface. This “breaks the chain” of bubbling up errors and checking if err != nil.

type MyError string

func (e MyError) Error() string { return string(e) }

func f() *MyError {
	return nil
}

// returns an interface with type *MyError and value nil
func g() error {
	return f()
}

func main() {
	x := g()
	if x == nil {
		fmt.Println("nil")
	}
}

this won’t print anything, the concrete type isn’t nil

Error Type Assertions

Inspect error type when need to pull more info out of the error.

if e, ok := err.(net.Error); ok && e.Timeout() {
	// it's a timeout, sleep and retry
}

type assertion to check if error of certain type

type case for multiple type assertions

Sentinel Errors

Sentinel errors are error variables or constants exposed by a package. Be careful, expand a package’s interface.

var ErrNotFound = errors.New("not found")

if err == ErrNotFound {
    // something wasn't found
}

no type assertion!

As and Is

As and Is were introduced in go 1.13 to help examine wrapped errors.

err := f()
if errors.Is(err, ErrFoo) {
	// you know you got an ErrFoo
	// respond appropriately
}

var bar *BarError
if errors.As(err, &bar) {
	// you know you got a BarError
	// bar's fields are populated
	// respond appropriately
}

Bitwise Operators

 &   bitwise AND
 |   bitwise OR
 ^   bitwise XOR
&^   AND NOT
<<   left shift
>>   right shift
// hammingWeight is the number of `1` bits in an integer
func hammingWeight(num uint32) int {
    bits := 0
    mask := uint32(1)
    
    for num != 0 {
        if num & mask == 1 {
            bits = bits + 1
        }
        num = num >> 1
    }
    
    return bits
}

Concurrency

The world is made up of concurrent agents, but our minds are sequential. Language concurrency features try to bridge this gap, not performance (e.g. parallelism).

c := make(chan int)
go sum(s[:len(s)/2], c)
go sum(s[len(s)/2:], c)
x, y := <-c, <-c // receive from c

// buffered channels provide backpressure
ch := make(chan int, 100)

// blocks until a channel can receive or send (kinda like an event loop)
select {
case i := <-c:
    // use i
default:
    // receiving from c would block
}

// can create a channel which fires an event for a timeout

// range pulls from channel until builtin close is sent

//  nil channels in select cases are ignored

Building

Comments

// Package sort provides primitives for sorting slices and user-defined
// collections.
package sort

package level

Packages and Modules

A package is a collection of source files in the same directory that are compiled together. Functions, types, variables, and constants defined in one source file are visible to all other source files within the same package.

A repository contains one or more modules. A module is a collection of related Go packages that are released together. A Go repository typically contains only one module, located at the root of the repository. A file named go.mod there declares the module path: the import path prefix for all packages within the module. The module contains the packages in the directory containing its go.mod file as well as subdirectories of that directory, up to the next subdirectory containing another go.mod file (if any).

Modules provide Dependency Management. The import syntax imports packages not modules.

If your module depends on A that itself has a require D v1.0.0 and your module also depends on B that has a require D v1.1.1, then Go Modules would select v1.1.1 of dependency D. This selection of v1.1.1 remains consistent even if some time later a v1.2.0 of D becomes available. (Like Gradle’s default)

If later an indirect dependency is removed, Go modules will still keep track of the latest not-greatest version. In other words, if we were to remove from our module the dependency B containing a require D v1.1.1 but keep dependency A with a require D v1.0.0, then Go modules would not fallback to v1.0.0 but instead keep v1.1.1 of D.

The -m flag causes go list to list modules instead of packages

A replace directive replaces the contents of a specific version of a module, or all versions of a module, with contents found elsewhere. The replacement may be specified with either another module path and version, or a platform-specific file path.

exclude and replace directives only operate on the current (\u201cmain\u201d) module. exclude and replace directives in modules other than the main module are ignored when building the main module. The replace and exclude statements, therefore, allow the main module complete control over its own build, without also being subject to complete control by dependencies.

// indirect dependencies are added to go.mod to provide 100% reproducible builds and tests by recording precise dependency information.

multi-module repos

A multi-module repository is a repository that contains multiple modules, each with its own go.mod file. Each module starts at the directory containing its go.mod file, and contains all packages from that directory and its subdirectories recursively, excluding any subtree that contains another go.mod file.

Each module has its own version information. Version tags for modules below the root of the repository must include the relative directory as a prefix.

make use of replace directives. Replace directives allow neighboring modules to import each others code without first publishing and remotely fetching it. However, they arent the most robust solution because they can be tricky to maintain (e.g. certain commands such as go list won work as expected) and require a decent amount of boilerplate code (e.g. replace directives are ignored outside of the current module being run/developed in, so youll need to repeat a replace directive in every go.mod in the repo where its relevant).

github issue on multi module one repo

A go.mod is like its own little GOPATH. There is no implicit reference to other nearby modules. In particular being in one repo does not mean that they all move in lock step and always refer to the code from the same commit. That’s not what users will get either.

The layout in the filesystem does not imply any kind of technical relation between packages (e.g. net/http/cookiejar is as much related to net/http as it is related crypto/md5: not at all).

Project layout

A Go package has both a name and a path. The package name is specified in the package statement of its source files; client code uses it as the prefix for the package’s exported names. Client code uses the package path when importing the package. By convention, the last element of the package path is the package name.

I think there are 2 choices:

  1. the top level package is main
    • can bury the library package in a nested directory (e.g. mango/mango)
  2. the top level package is the library’s name (e.g. mango)
    • can bury the main package in the command directory (e.g. cmd/mango) following convention; the app name as the leaf directory is used by the install command to name the executable

Option (1) makes sense for an application, the user will interact with the executable and having it at the top level as interface keeps things clean. Option (2) is much better for a library, since (1) would introduce needless/confusing nesting (mango/mango).

Might try to always use option (2) just for simplicity.

some conventions in the wild

bare minimum

go mod init git.sr.ht/~yonson/sandbox

initialize empty mod file

// main.go
package main

import "fmt"

func main() {
    fmt.Println("Hello, world.")
}

executable commands must always use package main and have main() func

go run main.go

run

get, build, and install

install

go generate

Testing

go test ./...

run test on all packages in the directory, recursive

coverage

go test -cover ./...

test tool has coverage output

go test -coverprofile cover.out ./...

test can output a profile for further examination

go tool cover -func cover.out

built in cover tool examining output

interfaces

tables

tests := []struct {
    name    string
    graph   *lnrpc.ChannelGraph
    request NodesByDistanceRequest
    want    []Node
}{
    {
        name: "identity",
        graph: &lnrpc.ChannelGraph{
            Nodes: []*lnrpc.LightningNode{
                {
                    PubKey:     rootPubkey,
                    Alias:      rootAlias,
                    LastUpdate: uint32(rootUpdatedTime(t).Unix()),
                },
            },
        },
        request: NodesByDistanceRequest{
            MinUpdated: rootUpdatedTime(t).Add(-time.Hour * 24),
            Limit:      1,
        },
        want: []Node{
            {
                pubkey:  rootPubkey,
                alias:   rootAlias,
                updated: rootUpdatedTime(t),
            },
        },
    },
}

for _, tc := range tests {
    app := App{
        Infoer:  fakeInfoer{info: &lnrpc.GetInfoResponse{IdentityPubkey: rootPubkey}},
        Grapher: fakeGrapher{graph: tc.graph},
        Log:     log.New(ioutil.Discard, "", 0),
        Verbose: false,
    }

    nodes, err := NodesByDistance(app, tc.request)

    if err != nil {
        t.Fatal("error calculating nodes by distance")
    }

    if !reflect.DeepEqual(tc.want, nodes) {
        t.Fatalf("%s nodes by distance are incorrect\nwant: %v\ngot: %v", tc.name, tc.want, nodes)
    }
}

Few things of note in this code block

Advanced patterns

gotests and impl

go install github.com/cweill/gotests/...@latest

to install with go 1.18+

go install github.com/josharian/impl/...@latest
$ impl -dir /home/njohnson/grd/mash/platform/fiat mock stripeProcessor

time

The time.RFC3339 format is a case where the format string itself isn’t a valid time. You can’t have a Z and an offset in the time string, but the format string has both because the spec can contain either type of timezone specification.

Linting

go fmt is built-in

Patterns

Sorting

// sort the characters of a string

// create a type which implements the sort interface of Less, Swap, and Len
type sortRunes []rune

func (s sortRunes) Less(i, j int) bool {
    return s[i] < s[j]
}

func (s sortRunes) Swap(i, j int) {
    s[i], s[j] = s[j], s[i]
}

func (s sortRunes) Len() int {
    return len(s)
}

func SortString(s string) string {
    r := []rune(s)
    // passing the sort type to Sort
    sort.Sort(sortRunes(r))
    return string(r)
}

func main() {
    w1 := "bcad"
    w2 := SortString(w1)

    fmt.Println(w1)
    fmt.Println(w2)
}

Enums

// AccessLevel is the level of access for a Dataset
type AccessLevel string

// Access levels
const (
        SharedAccessLevel     AccessLevel = "shared"
        RestrictedAccessLevel AccessLevel = "restricted"
)

iota vs. string?

CLI

ff