ToRunes aliases the rune array, and the editing actions append into
t.input in place when the cursor is not at the end, so the keystrokes
edit the item. Non-ASCII items only, ASCII gets a fresh slice.
printf '한글abcde\n' | fzf --bind 'ctrl-y:replace-query'
ctrl-y, Left, BSpace, ctrl-u -> 한글abcee
Runes and ToRunes are now documented read-only. A stale fold bit was the
other symptom, letting the prefilter reject an item the general path
matches.
asciiFuzzyIndex gave up on non-ASCII lines, so every item ran the full
score matrix. A []rune is a fixed 4-byte stride, so the SIMD byte
scanners can run over it directly: find the low byte, then confirm
4-byte alignment and three zero bytes.
Case folding and normalization can turn a non-ASCII rune into the ASCII
char being searched, which the scan cannot see. ToChars now flags lines
holding such a rune and those keep the old path. Normalization is
Latin-only, so Hangul, CJK, Cyrillic, Greek, Hebrew, Arabic, Thai, kana
and emoji never set the flag.
Chars had no spare padding, so inBytes moves into a flags byte.
Measured on 1.4M-line corpora:
- Mostly-ASCII paths behind a Hangul prefix: 'conf' 1.8x, 'binutils'
2.9x, 'ltversion' 4.0x, no-match 8.4x
- Every line CJK: 17x on both matching and non-matching queries
- ASCII input unchanged, non-ASCII patterns not covered yet
utf8.DecodeRune already fast-paths ASCII, but it is too complex to inline
(cost 201 against a budget of 80), so a mostly-ASCII line pays one call
per byte just to be told the byte is ASCII.
Only the run after the first non-ASCII byte reaches the decode loop, so
the gain depends on where that byte falls.
- 70-rune ASCII line: 145ns -> 42ns in the decode loop
- Ingestion of 1.4M mostly-ASCII paths behind a Hangul prefix, where the
loop covers the whole line: 377ms -> 233ms
- The same paths with the Hangul at the end, where it covers six bytes:
207ms -> 196ms
- Break-even sits at ~100% non-ASCII runes: still 1.01x at 95%. Only a
line holding no ASCII byte at all loses, by ~0.14ns per rune, ~5% of
the loop
- Capacity was byte length, over-allocating by bytes-per-rune (2-4x)
- Count non-continuation bytes with SWAR before allocating
- Invalid bytes undercount, never overcount, so append covers the gap
- Query performance unchanged, this is a memory fix
- The gain tracks bytes-per-rune, the cost tracks how much of the line
follows the first non-ASCII byte, so the two move independently
Measured on 1.4M-line corpora:
- Every line CJK: RSS 362MB -> 255MB, ingestion -5%
- Mostly-ASCII paths behind a Hangul prefix: RSS 556MB -> 533MB,
ingestion +3.5%, the counting pass covering the whole line
- The same paths with the Hangul at the end: RSS 563MB -> 535MB,
ingestion +0.9%, the counting pass covering six bytes
This option can be used to replace a sed or awk in the post-processing step.
ps -ef | fzf --multi --header-lines 1 | awk '{print $2}'
ps -ef | fzf --multi --header-lines 1 --accept-nth 2
This may not be a very "Unix-y" thing to do, so I've always felt that fzf
shouldn't have such an option, but I've finally changed my mind because:
* fzf can be configured with a custom delimiter that is a fixed string
or a regular expression.
* In such cases, you'd need to repeat the delimiter again in the
post-processing step.
* Also, tools like awk or sed may interpret a regular expression
differently, causing mismatches.
You can still use sed, cut, or awk if you prefer.
Close#3987Close#1323
Find the last occurrence of the last character in the pattern and
perform the search algorithm only up to that point.
The effectiveness of this mechanism depends a lot on the shape of the
input and the pattern.
- Prefix matcher will trim leading whitespaces only when the pattern
doesn't start with a whitespace
- Suffix matcher will trim trailing whitespaces only when the pattern
doesn't end with a whitespace
- Equal matcher will trim leading whitespaces only when the pattern
doesn't start with a whitespace, and trim trailing whitespaces only
when the pattern doesn't end with a whitespace
Previously, only suffix matcher would trim whitespaces unconditionally.
Fix#1894
- Replaced time.Now().Sub() with time.Since()
- Replaced unnecessary string/byte slice conversions
- Removed obsolete return and value assignment in range loop
Add String() methods to types, so they can be printed with %s. Change
some %s format specifiers to %v, when the default string representation
is good enough. In Go 1.10, `go test` triggers a parallel `go vet`. So
this also makes fzf pass `go test`.
Close#1236Close#1219
When --with-nth is used, fzf used to preprocess each line and store the
result as rune array, which was wasteful if the line only contains ascii
characters.
In the best case (all ascii), this reduces the memory footprint by 60%
and the response time by 15% to 20%. In the worst case (every line has
non-ascii characters), 3 to 4% overhead is observed.
Junegunn Choi