slice

Slice is Go's solution to flexible/undetermined length of array. It is something similar to "linked list" in C, only with a lot of linked list hassles being solved by Go internals. Therefore, instead of storing values directly into the memory like any other data types, slice is actually a list of data pointers pointing to the respective data memory.

Slice Internal

This is how a slice looks like internally (not a code but a diagram based on https://blog.golang.org/go-slices-usage-and-internals):

slice:
        ptr (pointer to element)
        len (length of slice)
        cap (capacity of the slice)

Hence, if we make a new slice, say:

b := make([]byte, 5)

We are essentially create the b object as follows, where the pointer is pointing to the first element address in the memory:

memory:
        addr1 = 0x00        # b
        addr2 = 0x00
        addr3 = 0x00
        addr4 = 0x00
        addr5 = 0x00

b:
        ptr = addr1
        len = 5
        cap = 5


Assigning Values

When we assign values, it will automatically seek the memory location and fills the data in, example say:

b[0] = 0xF1
b[1] = 0xF2
b[2] = 0xF3

This alters the slice, continue form the example above:

memory:
        addr1 = 0xF1        # b
        addr2 = 0xF2
        addr3 = 0xF3
        addr4 = 0x00
        addr5 = 0x00

b:
        ptr = addr1
        len = 5
        cap = 5

This is also the reason why when you grows the slice, Go needs to perform "copy" and "append" functions internally. Say:

  • if we grow b to b[6] = 0xE0
    1. Go first perform a memory allocation similar to make([]byte, 6)
    2. It then copies the values from old memory to new memory.
    3. It then adds the 7th elements into the slice.

Hence, we got a new memory location, shown as below (notice the address turned into theoretical addr11):

memory:
        addr11 = 0xF1        # b
        addr12 = 0xF2
        addr13 = 0xF3
        addr14 = 0x00
        addr15 = 0x00
        addr16 = 0xE0

b:
        ptr = addr11
        len = 5
        cap = 5



Duplicate the Slice

The above is also the reason why you can't simply duplicate a slice by just equating it into a new variable. As a matter of fact, you need to use copy function to duplicate it manually. Example:

var s []byte
s = copy(b, s)

We got:

memory:
        addr11 = 0xF1        # b
        addr12 = 0xF2
        addr13 = 0xF3
        addr14 = 0x00
        addr15 = 0x00
        addr16 = 0xE0
        ...
        addr21 = 0xF1        # s
        addr22 = 0xF2
        addr23 = 0xF3
        addr24 = 0x00
        addr25 = 0x00
        addr26 = 0xE0

s:
        ptr = addr21
        len = 5
        cap = 5

b:
        ptr = addr11
        len = 5
        cap = 5


Deleting Slice

Similar to any other data types in Go, once there is no reference pointing to the slice, it will be garbage collected.

Slicing the Slice

Unlike array, slicing a slice is very tricky. It follows the following conventions:

variable[start:stop:capacity]
  • start = the starting index. You get this element
  • stop = the stopping index. It signals for stopping and itself would not be included
  • capacity = the defined length of your new slice or array.

You must create the slice first before you can slice it. In another words, save it into a variable, and you can't perform something as such:

newS := []byte{1, 2, 3, 4, 5, 6}[0:2:5]    # bad practice


s := []byte{1, 2, 3, 4, 5, 6}
newS  := s[0:2:5]                          # correct practice
newS2 := s[0:2]                            # common practice

This produces new S with the following data in memory:

memory:
        addr11 = 0x01        # s
        addr12 = 0x02
        addr13 = 0x03
        addr14 = 0x04
        addr15 = 0x05
        addr16 = 0x06
        ...
        addr21 = 0x01        # newS
        addr22 = 0x02
        addr23 = 0x00
        addr24 = 0x00
        addr25 = 0x00
        ...
        addr51 = 0x01        # newS2
        addr52 = 0x02

newS2:
        ptr = addr51
        len = 2
        cap = 2

newS:
        ptr = addr21
        len = 2
        cap = 5

s:
        ptr = addr11
        len = 6
        cap = 6

Keep in mind that if you slice the slice with the wrong parameter (e.g. wrong capacity, out of range etc.), you will get a panic instead of error.

Passing into Function

Slice, a lot of Gopher recommends to pass it as it is instead of its pointer form into a function parameter since itself is a list of pointers. That is true for beginner and those without a good grasp of what pointer means. Just like any other data structure, passing the slice as it is will trigger a copy of itself. Consider this source codes:

package main

import (
        "fmt"
)

func runWithPointer(b *[]byte) {
        l := *b
        l[0] = 255
        (*b)[1] = 254
        fmt.Printf("pointer for b: %p\n", b)
        fmt.Printf("pointer for l: %p\n", l)
}

func runWithoutPointer(k []byte) {
        k[2] = 253
        fmt.Printf("pointer for k: %p\n", k)
}

func main() {
        s := "abcde"
        b := []byte(s)
        c := b

        fmt.Printf("pointer for b: %p\n", &b)
        runWithPointer(&b)
        runWithoutPointer(b)
        d := b

        fmt.Printf(`
ori = %v
b   = %v
c   = %v
d   = %v
`, []byte(s), b, c, d)
}

This produces the following outputs:

b original pointer: 0xc00000c060
pointer for b     : 0xc00000c060
pointer for l     : 0xc0000180f8
pointer for k     : 0xc0000180f8

ori = [97 98 99 100 101]
b   = [255 254 253 100 101]
c   = [255 254 253 100 101]
d   = [255 254 253 100 101]

We all see that the b values got altered, regardless being passed in as a value or a pointer. That is:

  • b = [255 254 253 100 101] // which is set by [l *b k 100 101]

We will explain one by one in the sub headings with the "magical" part.



Passing as Value

The conventional way is to pass in as slice value instead of pointer. As usual, every function will duplicates the value for its local stack operations. Notice the pointer for k is different from the b pointer:

b original pointer: 0xc00000c060
pointer for k     : 0xc0000180f8

This indicates that the slice is duplicated the moment the system enters func runWithoutPointer(k []byte) function. However, the function is able to alter the slice's value even-though k is a totally new variable:

k[2] = 253
...
ori = [97 98 99 100 101]
b   = [255 254 253 100 101]

This is because the slice got duplicated is not the entire slice but the slice heading itself. The value pointers inside the slice is still same, which is pointing to original memory location. This is why you can alter the slice values.



Passing as Pointer

WARNING: not for pointer faint heart folks.

Normally if you pass any variables or data pointer into the function, there are 2 motives:

  1. You want to alter/observe the original data directly.
  2. You want to avoid the "copying" mechanism happening at stack level memory.

Notice the output of pointer to be is the same as pointer b? You can operate original data directly, without copying if you pass the slice into the function.

b original pointer: 0xc00000c060
pointer for b     : 0xc00000c060

That being said, a typical operation for such function is usually obscuring since you need to wrap the variable with the parenthesis. Example (*b):

func runWithPointer(b *[]byte) {
        (*b)[1] = 254
        c := (*b)[0:2]
        ...
}

Keep in mind though, if you set a variable equals to the value of the slice, you are essentially copying the slice header itself, not dereferencing it. Example:

        l := *b
        l[0] = 255
...
b original pointer: 0xc00000c060
pointer for l     : 0xc0000180f8
b   = [255 254 253 100 101]

Notice the address for l has changed to a new one, similar to k when passing in as value.



Premature Optimization?

Now, if you notice the differences, the only gain for you to pass in as pointer is that you avoided the function copying mechanism at the expense of the weird parenthesis notation. Consider the clean version:

func runWithPointer(b *[]byte) {
        (*b)[1] = 254
        c := (*b)[0:2]
        ...
}

func runWithLocalVariablePointer(b *[]byte) {
        l := *b
        l[1] = 254
        c := l[0:2]
        ...
}

func runWithoutPointer(k []byte) {
        k[2] = 253
        c := k[0:2]
        ...
}

Of course runWithoutPointer is recommended since it is clean and tidy. This is why the Go community recommends not to pass slice as pointer.

However, it does not means you cannot use runWithPointer. It is just the Go community will frown upon it since the gain not worth sacrificing the readability. Specify a local variable copy like runWithLocalVariablePointer does is essentially the same as runWithPointer.

Besides, a lot of folks are scared of pointer (no idea why) so maintaining the pointer convention can be a pain at some point.

Conversion

Array to Slice

You can easily convert an array to a slice by using the array boundary selector and select all elements in the slice. Example:

array := [5]byte{0x02, 0x03, 0x04, 0x05, 0x06}
slice = array[:]

This is a direct array to slice conversion when both array and slice shares the same length. DO NOT delete the array since slice relies on it to point towards the memory.

Otherwise, you need to perform manual copy using append function. Example:

slice = []byte{}
slice = append(slice, a[:]...)


Slice to Array

Slice to array requires you to copy the slice components into the a newly built slice. Example:

slice := []byte{0x02, 0x03, 0x04, 0x05, 0x06}
var array [5]byte
copy(array[:], slice)

That's all about slices in Go.