剖析 Golang Go语言中 Bigcache 的极致性能优化

Hi, I will write in English but I hope many people will find my opinion and review of cache libraries in Go useful.

So, cache libraries in Go are of two types
1. Cache libraries that use a good eviction policy but put extra pressure on gc
2. Cache libraries that don’t use eviction policies (just delete the first inserted element) but don’t put extra pressure on gc

Let’s talk about the second category first. The main representatives are: fastcache( https://github.com/VictoriaMetrics/fastcache), bigcache( https://github.com/allegro/bigcache) and freecache( https://github.com/coocood/freecache). Ok, when should we use them? It seems that approximately never because the only advantage they give is the absence of pressure on gc, even when storing tens of millions of key-value pairs libraries of the first type will win. Even the article about the creation of bigcache makes me smile https://blog.allegro.tech/2016/03/writing-fast-cache-service-in-go.html . They just wrote a worse version of redis or memcached and got no benefits. The only case when I think that using such libraries is justified is when you write a release-cycle storage where data will be stored in ram, like VictoriaMetrics( https://github.com/VictoriaMetrics), for which fastcache was written (it is faster than bigcache by the way). But for the rest there is no point in such libraries, as a good eviction policy will give much more cache hits than trying to store gigabytes of additional data. Also, these libraries do not really reduce the load on gc because they work only with strings/slices of bytes, which forces to convert other types into them, which firstly takes a lot of time and secondly greatly increase the pressure on gc, which has to do all this, and also leads to a strong fragmentation of memory (external because of string allocation and internal, because after deleting items in memory allocated by these caches huge holes are formed).

Now let’s talk about libraries of the first type.

There are two types of libraries: slow but simple libraries with global locking (the vast majority of them) like https://github.com/hashicorp/golang-lru and faster libraries that try to avoid global locking ristretto( https://github.com/dgraph-io/ristretto), theine( https://github.com/Yiling-J/theine-go) and I’m writing a much faster alternative to them otter( https://github.com/maypok86/otter). RTO (ristretto theine otter) is already suitable for many more users, as it gives a good eviction policy and is more user friendly. Ok, which one should I choose then? Let’s take a look at them in order
1. Ristretto. DON’T USE IT AT ALL. It has a terrible hit ratio and the authors don’t answer questions about it (and basically don’t answer anything). You can see it here https://github.com/dgraph-io/ristretto/issues/346 and here https://github.com/dgraph-io/ristretto/issues/336. It also allows eviction policy updates to be lost after being added to the map, which is actually a memory leak. It also replaces keys with their hash, which can cause you to run into zombies. And other problems
2. Theine. A good library that I don’t really have any complaints about, except that its speed degrades already 100000 items to cache level with LRU and global locking but in return it provides a great hit ratio
3. Otter. Do not use it yet, as it is not production-ready yet. Although the intermediate results are very impressive: otter is more than 5 times faster than ristretto and theine, and on most traces from ristretto’s benchmarks outperforms all of them by hit ratio

Somehow, I hope it was useful because I meet a lot of misunderstandings on this topic