Overview In today’s high-performance and concurrent computing environments, effectively processing a stream of messages using a mix of purely computational functions and remote procedure calls has become a significant challenge. The Go programming language is designed to handle concurrency well, but when it comes to managing a hybrid load, even Go can struggle to achieve optimal CPU utilization. In this article, we will discuss the Hybrid Handler pattern, an efficient and unified approach to address this challenge. ...

A directed acyclic graph or DAG is a directed graph with no directed cycles. An arbitrary directed graph may also be transformed into a DAG, called its condensation, by contracting each of its strongly connected components into a single super vertex. Go’s import declaration declares a dependency relation between the importing and imported package. It is illegal for a package to import itself, directly or indirectly, or to directly import a package without referring to any of its exported identifiers. ...

Again and again, a concurrent pattern emerges from the need to control goroutine lifecycles and handle their errors, and I call it the “Runner Pattern”. The runner interface and its contract The pattern is as simple as a single-method interface: // Runner defines the Run method to be executed within a goroutine type Runner interface { Run(ctx context.Context) error } The contract of the interface covers two aspects. On the goroutine lifecycle, the Run method will block until one of the following occurs: ...

Imagine you need to wrap multiple objects which implements io.Closer, e.g. three clients to fetch and combine data from different endpoints. type Parent struct { child1 Child1 child2 Child2 child3 Child3 } Parent closer Let’s see how we can create and destroy a parent object. func NewParent() (*Parent, error) { child1, err := NewChild1() if err != nil { return nil, err } child2, err := NewChild1() if err != nil { // oops, child1 needs to be closed here child1.Close() return nil, err } child3, err := NewChild1() if err != nil { // oops again, both child1, and child2 needs to be closed here child1.Close() child2.Close() return nil, err } return &Parent{ child1: child1, child2: child2, child3: child3, }, nil } func (p *Parent) Close() error { var errs []error if err := p.child1.Close(); err != nil { errs = append(errs, err) } if err := p.child2.Close(); err != nil { errs = append(errs, err) } if err := p.child3.Close(); err != nil { errs = append(errs, err) } return multierr.Combine(errs...) } Note the boilerplate code of closing the children. Because the parent creates its children, it must be responsible for calling their Close method whenever needed. If there are any errors during the initialisation, the children already created have to be properly closed, and before the parent exits its scope, it has to close its children too. ...

A summary of Boolean algebra Given the following notations: a proposition is denoted as a lowercase letter, e.g. $p$, $q$ the truth value of a proposition $p$ is denoted as $ B(p) \in \set{0, 1} $, where $B(p)=1$ if $p$ is true or $B(p)=0$ if $p$ is false Negation (not, $¬$), conjunction (and, $∧$) and disjunction (or, $∨$) are defined by the truth tables below: $B(p)$ $B(¬p)$ 0 1 1 0 ...

We often use Mutex or RWMutex as locks in Go, but sometimes we need a lock that can be cancelled by a context during the lock attempt. The pattern is simple - we use a channel with length 1: lockChan := make(chan struct{}, 1) lockChan <- struct{}{} // lock <- lockChan // unlock When multiple goroutines try to obtain the lock, only one of them is able to fill into the only slot, and the rest are blocked until the slot is empty again after a readout. ...

A buffered writer is so ubiquitous that we do not usually consider it as a pattern, but sometimes we reinvent it or even do it in an inferior way. Let us look at a real use case first. Batch processor What would you do to improve the throughput of a service? The answer is short: batching. By processing and sending in a batch of multiple items instead of a single item at a time, you are amortizing the network overhead from the request-response round trip among all the items in the batch. ...

Should I use value receivers or pointer receivers? Value receivers have some benefits include immutability, concurrent safety and clean logic (not always, often true). But to what extend can I use value receivers without an issue or performance penalty? In the Go FAQ, there are 3 rules: most important, does the method need to modify the receiver? If it does, the receiver must be a pointer if the receiver is large, a big struct for instance, it will be much cheaper to use a pointer receiver if some of the methods of the type must have pointer receivers, the rest should too, so the method set is consistent regardless of how the type is used Let’s look at rule 1. In many cases, an object needs to be modified by a method after its initialization, but it doesn’t mean the modification has to be done in place, an alternative and often a better way is to get a modified copy of the object, which is usually called a Fluent Interface. Basically it means a method with a value receiver and a return value of the same type. It’s just pure (no side effect) functional programming style under another name. ...

Slices Go slice and Python slice have very similar syntax, but Python slice is a shallow copy of part of the original list, while Go slice is just a new range within the same underlying array of the original slice. Let’s try: a = [1, 2, 3] b = a[:2] b[0] = 9 print(a) print(b) # output: # [1, 2, 3] # [9, 2] See a[0] remains the same. package main import ( "fmt" ) func main() { a := []int{1, 2, 3} b := a[:2] b[0] = 9 fmt.Println(a) fmt.Println(b) # output: # [9 2 3] # [9 2] } See a[0] changes because slice a and b shares the same underlying array. ...