mirror of
https://github.com/junegunn/fzf.git
synced 2026-08-19 06:08:05 +08:00
Junegunn Choi
388ce31527
The scan only ran for ASCII patterns, so searching CJK text with a CJK query still built the full score matrix. Scan for one byte of the pattern rune and verify all four. Which byte matters. Every ASCII rune contributes three zero bytes, so U+AE00 scanned by its zero low byte hits on nearly every character of an ASCII-heavy line. Pick a byte that cannot occur in an ASCII rune, else any non-zero one. A non-ASCII pattern rune is safe only when no other rune lowercases onto it. Uncased is not sufficient: U+00DF has no simple uppercase, yet U+1E9E lowercases to it, so the foldable set is excluded too. Measured on 1.4M-line corpora, with a non-ASCII query: - Every line CJK: 5.1x to 10.2x - Mostly-ASCII paths behind a Hangul prefix: 5.6x to 6.0x, and 1.2x where every line matches so nothing can be rejected - ASCII queries unchanged, kept off the non-inlinable guard
24 lines
721 B
Go
24 lines
721 B
Go
//go:build !386 && !amd64 && !arm64
|
|
|
|
package algo
|
|
|
|
// The byte-view scanners in runeindex_x86.go reinterpret a []rune as
|
|
// little-endian 4-byte lanes, which is not valid everywhere. Elsewhere the
|
|
// reference scanners are the implementation.
|
|
|
|
func indexAsciiRune(runes []rune, caseSensitive bool, b byte, from int) int {
|
|
return indexAsciiRuneRef(runes, caseSensitive, b, from)
|
|
}
|
|
|
|
func lastIndexAsciiRune(runes []rune, caseSensitive bool, b byte, from int) int {
|
|
return lastIndexAsciiRuneRef(runes, caseSensitive, b, from)
|
|
}
|
|
|
|
func indexRune(runes []rune, r rune, from int) int {
|
|
return indexRuneRef(runes, r, from)
|
|
}
|
|
|
|
func lastIndexRune(runes []rune, r rune, from int) int {
|
|
return lastIndexRuneRef(runes, r, from)
|
|
}
|