mirror of
https://github.com/junegunn/fzf.git
synced 2026-08-20 06:38:04 +08:00
Prefilter rune-mode input
asciiFuzzyIndex gave up on non-ASCII lines, so every item ran the full score matrix. A []rune is a fixed 4-byte stride, so the SIMD byte scanners can run over it directly: find the low byte, then confirm 4-byte alignment and three zero bytes. Case folding and normalization can turn a non-ASCII rune into the ASCII char being searched, which the scan cannot see. ToChars now flags lines holding such a rune and those keep the old path. Normalization is Latin-only, so Hangul, CJK, Cyrillic, Greek, Hebrew, Arabic, Thai, kana and emoji never set the flag. Chars had no spare padding, so inBytes moves into a flags byte. Measured on 1.4M-line corpora: - Mostly-ASCII paths behind a Hangul prefix: 'conf' 1.8x, 'binutils' 2.9x, 'ltversion' 4.0x, no-match 8.4x - Every line CJK: 17x on both matching and non-matching queries - ASCII input unchanged, non-ASCII patterns not covered yet
This commit is contained in:
@@ -102,7 +102,7 @@ itest:
|
||||
# FUZZTIME (e.g. make fuzz FUZZTIME=5m).
|
||||
FUZZTIME ?= 30s
|
||||
fuzz:
|
||||
@for t in FuzzFuzzyMatchV2Single FuzzFuzzyMatchV2Two; do \
|
||||
@for t in FuzzFuzzyMatchV2Single FuzzFuzzyMatchV2Two FuzzRunePrefilter; do \
|
||||
echo "== $$t =="; \
|
||||
$(GO) test -run '^$$' -fuzz "^$$t$$" -fuzztime $(FUZZTIME) ./src/algo || exit 1; \
|
||||
done
|
||||
|
||||
Reference in New Issue
Block a user