Go EP16: How Go Handles Key-Value Pair Storage in Maps

Have you ever set a ‘hint’ for a map and wondered why it’s called a ‘hint’ and not something simple like length, like we do with slices?

Go Maps Explained: How Key-Value Pairs Are Actually Stored

When you start working with maps in Go, you’ll quickly notice they’re a really flexible way to store key-value pairs. Unlike arrays, which are stuck with sequential indices, maps let you use almost anything as a key, as long as it’s a comparable type. That opens up a bunch of possibilities.

But with that flexibility comes a few things going on under the hood that are worth knowing.

First off, Go maps are built around something called an hmap.

type hmap struct {
	count     int 
	flags     uint8
	B         uint8  
	noverflow uint16 
	hash0     uint32 

	buckets    unsafe.Pointer 
	oldbuckets unsafe.Pointer 
	nevacuate  uintptr        

	extra *mapextra 
}

It’s basically a structure that holds a pointer to an array of buckets. Each of those buckets can store up to 8 key-value pairs.

When you throw a new key-value pair into a map, Go will hash the key and use that hash to figure out which bucket to place it in. This hashing keeps things pretty evenly spread out, but it also means you’re never going to get a predictable order when you loop through the map.

As you keep adding more items, eventually, one of the buckets will fill up.

Now, instead of doubling the whole array right away, Go uses what are called overflow buckets. These kick in when there are too many collisions and help keep memory usage in check. But once the map’s load factor (which is just a measure of how full it is) hits a certain point - currently set at 6.5, Go will go ahead and double the bucket array size and redistribute everything.

One of the trickier parts of Go maps is how they grow.

When a map needs to get bigger, Go doesn’t just move everything all at once. Instead, it uses something called incremental growth, so the resizing happens bit by bit.

The growth process gets triggered when you add or remove items from the map, and it’s basically just shuffling key-value pairs from the old buckets to the new ones based on their hashes.

If you're interested in diving deeper into the content, you can follow the links https://victoriametrics.com/blog/go-map/ to read the full posts.

---