Replace []Result cache with bitmap cache for reduced memory usage

Replace the per-chunk query cache from []Result slices to fixed-size
bitmaps (ChunkBitmap: [16]uint64 = 128 bytes per entry). Each bit
indicates whether the corresponding item in the chunk matched.

This reduces cache memory by 86 times in testing:
- Old []Result cache: ~22KB per chunk per query (for 500 matches)
- New bitmap cache:   ~262 bytes per chunk per query (fixed)

With the reduced per-entry cost, queryCacheMax is raised from
chunkSize/5 to chunkSize/2, allowing broader queries (up to 50% match
rate) to be cached while still using far less memory.
This commit is contained in:
Junegunn Choi
2026-03-08 11:40:37 +09:00
parent 9249ea1739
commit 2f27a3ede2
5 changed files with 191 additions and 87 deletions

View File

@@ -37,14 +37,15 @@ const (
progressMinDuration = 200 * time.Millisecond
// Capacity of each chunk
chunkSize int = 1000
chunkSize int = 1024
chunkBitWords = (chunkSize + 63) / 64
// Pre-allocated memory slices to minimize GC
slab16Size int = 100 * 1024 // 200KB * 32 = 12.8MB
slab32Size int = 2048 // 8KB * 32 = 256KB
// Do not cache results of low selectivity queries
queryCacheMax int = chunkSize / 5
queryCacheMax int = chunkSize / 2
// Not to cache mergers with large lists
mergerCacheMax int = 100000