The Bloom Filter Optimization Saga: The "Benign" Read That Wasn't
An atomic-first Go Bloom filter still tripped the race detector. A war story about the read that looked benign, and what the Go memory model has to say about it.
An atomic-first Go Bloom filter still tripped the race detector. A war story about the read that looked benign, and what the Go memory model has to say about it.
A line-by-line reading of avx2.s: runtime AVX2 detection, and hand-written Go assembly for population count, bitwise OR and AND, and clearing 32 bytes at a time.
Making the Bloom filter thread-safe cost 8–12x in throughput. The culprit was a heap allocation on every Add and Contains; the fix was a stack array for small hash counts.
Concurrent reads were panicking in a Go Bloom filter because Contains() quietly mutated shared scratchpad state. How the race was found, fixed, and what the fix cost.
A benchmark that measured fmt.Sprintf instead of the filter, several pprof sessions, one optimisation that made things worse, and the data-structure change that finally paid.
A comprehensive exploration of a cache-line optimized Bloom filter implementation in Go with SIMD acceleration and assembly integration.