Use a shared work queue instead of static partitioning in matcher

Replace static chunk partitioning (sliceChunks) with a shared atomic
counter that workers pull from. This gives natural load balancing;
workers that finish chunks quickly grab more work instead of idling.

With this change, NumCPU workers suffice (no need for 8x oversubscription),
reducing goroutine overhead while improving throughput by 5-22%.

Now the performance scales linearly to the number of threads:

=== query: 'linux' ===
  [all]   baseline:    17.12ms  current:    14.28ms  (1.20x)  matches: 179966 (12.79%)
  [1T]    baseline:   136.49ms  current:   137.25ms  (0.99x)  matches: 179966 (12.79%)
  [2T]    baseline:    75.74ms  current:    68.75ms  (1.10x)  matches: 179966 (12.79%)
  [4T]    baseline:    41.16ms  current:    34.97ms  (1.18x)  matches: 179966 (12.79%)
  [8T]    baseline:    32.82ms  current:    17.79ms  (1.84x)  matches: 179966 (12.79%)
This commit is contained in:
Junegunn Choi
2026-03-07 18:26:42 +09:00
parent 92dc40ea82
commit 92bfe68c74
2 changed files with 23 additions and 47 deletions
+1 -3
View File
@@ -34,9 +34,7 @@ const (
maxBgProcessesPerAction = 3
// Matcher
numPartitionsMultiplier = 8
maxPartitions = 32
progressMinDuration = 200 * time.Millisecond
progressMinDuration = 200 * time.Millisecond
// Capacity of each chunk
chunkSize int = 1000