Go Secret - Slice: Be Careful When Resizing

So you’re adding a new item to a slice in Go. What’s the deal with the original array underneath? Does it stretch out to make room for more stuff?

Photo by charlesdeluvio on Unsplash

---

Yes, this is the second installment in a series of articles on Go secrets. In the first article, we discussed about Go Secret — Defer: What you know about DEFER in Go is not enough!

This time, we’re looking at slices in Go.

We’ll see how they work behind the scenes, and how understanding that can help make your code run better.

This is important if you want your code to run fast and smooth.

---

1. Structure of Array

To really understand what a slice is, we should check out its foundation: the array, here’s a simple example:

func main() {
  a := [5]int{}
  fmt.Printf("%p, %p\n", &a, &a[0])
}

// 0x14000018240, 0x14000018240

You might have noticed that the first item in the array shares its memory address with the array itself. What does this mean? Well, if you send an array to a function or save it in a variable, you’re actually dealing with the memory spot of that first item.

So, an array’s memory setup is like a line of blocks, each one right next to the last.

But the size of the array is fixed, creating somewhat of a predicament when you’re dealing with dynamic data sizes and that’s where slices sashay into the scene.

2. Structure of Slice

A slice has three main components

• Underlying array pointer: This points to the first item in the slice’s memory space.

• Length: This tells you how many items in the slice you can actually use or see..

• Capacity: This is about how many items the underlying array, starting from the array pointer.

To get this slice info, we looked at Go’s official runtime code. Here’s how a slice is laid out in the Go language:

type slice struct {
  array unsafe.Pointer
  len   int
  cap   int
}

For those who like examples, let’s dig into length and capacity a bit more (If you’re already good on these points, feel free to skip this part).

func main() {
  original := []int{0, 1, 2, 3, 4}
  s := original[1:2] 
  fmt.Println(len(s), cap(s)) // 1 4
}

In this example, the slice ‘s’ is basically one number: []int{1}. And its capacity is worked out from the first to the fourth spot in the original list, like this:

3. Understanding Slices and Array Changes in Go

When you work with slices in Go, it's important to know that changing values in a slice can sometimes also change the original array.

However, this doesn't always happen, so it's risky to expect it in your code.

func main() {
  original := []int{0, 1, 2, 3, 4}
  s := original[:]
  
  fmt.Println("Same array:")
  s[0] = 100
  fmt.Println(original, s)
  
  fmt.Println("Different array:")
  s = append(s, 5)
  s[0] = 200
  fmt.Println(original, s)
}

// Same array:
// [100 1 2 3 4] [100 1 2 3 4]
// Different array:
// [100 1 2 3 4] [200 1 2 3 4 5]

“So what’s going on here?”

In this example, let's figure out why the outputs are different. The append() function is doing more than adding elements to the slice:

1. First, it checks if there is enough space in the slice to add new elements.

2. If not, append() creates a new slice (which means new underlying array), moves everything over, and the slice s now points to this new array.

3. After that, it adds the new elements.

And, for those interested, here’s a simpler version of the append() function that I’ve rewritten, making use of generics

func append[T any](s []T, x ...T) []T {
  n := len(s)
  maxN := len(s) + len(x)
  
  // If there is not enough capacity, create a new slice with larger capacity
  if n+len(x) > cap(s) {
    newSlice := make([]T, maxN, maxN*2)
    copy(newSlice, s) // Copy the elements from the original slice to the new slice
    s = newSlice
  }
  
  s = s[:maxN]
  copy(s[n:], x)
  return s
}

Pre-allocating Technique

Resizing a slice takes up memory and slows down your program because it needs to create a new array and move all elements over. So, if you already know how many items your slice will hold, it’s smart to set that size in advance.

---

”Can you use append() and pre-allocate at the same time? Filling in data by index feels like a hassle”

You bet you can.

Use the make() function to create a slice with preset size and capacity. This way, you can still use append() and sidestep the fuss of setting each item by its index.

func main() {
  s := make([]int, 0, 3)
  s = append2(s, 1, 2, 3, 4)
  fmt.Println(s)
}

---

So remember, pre-allocating is a big deal.

Don’t let your app waste time resizing slices and imagine having a slice that starts with zero length and suddenly needs to hold 1,000 items. Now picture 100 slices doing the same thing.

Yeah, that’s a recipe for trouble.