Is a slice a dynamically sized array?

A slice has 3 fields:

  • The pointer to the array data.
  • The length.
  • The capacity.
pointer length capacity

The capacity field is needed for the exponential growth scheme. This is the same as any dynamic array implementation such as C++ std::vector or Python list.

Is a slice a fat pointer?

Out-of-bound errors are a common problem in C. Golang improved this by mandating bounds checking for slices. The slice index is checked against the slice length. Unlike C, Golang doesn’t allow arbitrary pointer arithmetic (including using a pointer as an array), so many plain pointer usages are replaced by slices.

A pointer with a length field for bounds checking is called a “fat pointer”. In this use case, the capacity field seems irrelevant, and most importantly, append() makes no sense.

pointer length capacity pointer length capacity

A plain pointer references a single item, a fat pointer references a range of items.

A slice cannot be both!

So a slice is either a dynamic array or a fat pointer, depending on the use case. This is a great source of confusion, as people will assume what a slice is based on what they learned first, and then be surprised when the actual use case differs.

Why does this matter? Because the 2 types of slices are incompatible — you cannot append to a fat pointer.

Muddling data mutability and ownership

Go slices are not only confusing, they are a source of bugs. This is because there is a huge difference between a dynamic array and a fat pointer:

  • A dynamic array owns the data; it’s the only reference to the data.
  • A fat pointer doesn’t own the data; the data may be referenced elsewhere.

When a function returns a slice, what can you do with it?

s := foobar();     // a function returns a slice
s[idx] = x;        // Q1: can I modify it? (mutability)
s = append(s, y);  // Q2: can I append it? (ownership)

It’s easy to ignore the differences and make these mistakes:

  1. Violating mutability by modifying via a slice when you shouldn’t.
  2. Violating ownership by appending to a fat pointer (overwrites the data after it).

Conclusion: Always clarify what a slice is!

If a function returns a fat pointer, you certainly cannot append to it, and you probably cannot modify it because no ownership often implies no mutability.

If a function returns a dynamic array, then the ownership has passed to the caller. You can do anything with the slice.

Merging incompatible concepts was a mistake

So a dynamic array and a fat pointer are very incompatible concepts. Merging them into a single datatype is a mistake. The merged datatype has invalid operations that are context-specific, which causes bugs.

I’ve seen this kind of mistake in other languages as well. For example, Lua merges array and hashtable into a single datatype, but in practice, the intended datatype is either hashtable or array, never both.

More surprises: Reslice beyond length

Did you know that you can reslice a slice beyond its length?

arr := make([]int, 4, 4)
s1 := arr[0:2]  // len=2, cap=4
v1 := s1[2]     // Out of bounds error!
s2 := s1[2:4]   // But this is OK?
s3 := s1[0:4]   // Larger slice from a smaller one.

When reslicing a slice, the result is bounded by the capacity field, not the length field!

Why is this surprising? Because you would expect slicing to have the same bounds checking as indexing a single item. But Go uses the seemingly irrelevant capacity field instead.

A fat pointer with a capacity field is a Go novelty. It allows you to make a larger slice from a smaller slice. However, this feature doesn’t even have many use cases, because you can just keep the original slice around. Maybe this feature exists only to give the capacity field a purpose?

More surprises: Modify the capacity field

Did you know that there is a syntax for modifying the capacity field?

s[low:high:max]
  • high - low is the new length.
  • max - low is the new capacity.

Of course, you can only reduce the capacity. The purpose of this feature is to remove the extra capacity.

s = s[:len(s):len(s)]   // cap == len

This is a guard against the mistake of accidentally appending to a slice you don’t own, since there is no capacity left, append() causes a reallocation rather than overwriting the data after the slice.

Why is this surprising? Because Go generally sticks to extreme minimalism, I didn’t expect them to add special syntax for a niche feature.