4656 Commits

Author SHA1 Message Date
Johannes Altmanninger
5868b3c380 read_unquoted_escape: remove dead loop condition
This was recently converted to a while-loop. However, we only
loop in a specific case when (by hitting "continue") so a
loop condition is not necessary.

No functional change.
2022-10-05 22:27:00 -05:00
Fabian Boehm
e7a7a58030 Remove use of maybe_t that makes gcc grumpy
We have a state machine here already, we can just use the state where
the variable is valid.
2022-10-05 22:34:19 +02:00
Fabian Boehm
460f56f95a Revert "Silence gcc warning"
This reverts commit 8ab437a98944aaa9e1c373f0bf03ca6c4a589f06.

It introduced a warning for clang - because that read the GCC pragma and didn't understand it.
2022-10-05 22:29:04 +02:00
Fabian Boehm
8ab437a989 Silence gcc warning
This complained that the variable might be uninitialized *right* after
the check that it wasn't, because it doesn't understand maybe_t.
2022-10-05 19:07:41 +02:00
Fabian Boehm
396e276286 Decode multibyte escapes immediately
We forgot to decode (i.e. turn into nice wchar_t codepoints)
"byte_literal" escape sequences. This meant that e.g.

```fish
string match ö \Xc3\Xb6

math 5 \X2b 5
```

didn't work, but `math 5 \x2b 5` did, and would print the wonderful
error:

```
math: Error: Missing operator
'5 + 5'
   ^
```

So, instead, we decode eagerly.
2022-10-05 18:55:01 +02:00
Sergei Shilovsky
e274ef6c0d
commandline --selection-start and --selection-end implementation
Fixes #9197
2022-10-05 18:51:00 +02:00
Fabian Boehm
dcf52dbba5 fix path --null-out
Regression from 7bc4c9674bb24f09e0b7af89442f700864217327.

Appending `"\0"` to an std::string does nothing.

I blame C++.
2022-10-05 17:25:00 +02:00
Fabian Boehm
cb28b39b24 string shorten: Make max of 0 mean no shortening
This makes it easier to just slot in `string shorten` wherever,
without having to do a weird "if test $max -gt 0" check.
2022-10-04 18:44:21 +02:00
Fabian Boehm
cdf1a94e29 ifdef DT_WHT 2022-10-04 17:00:04 +02:00
ridiculousfish
757c117591 Handle symlink loops in descend_unique_hierarchy
descend_unique_hierarchy is used for the cd autosuggestion: if a directory
contains exactly one subdirectory and no other entries, then propose that
as part of the cd autosuggestion.

This had a bug: if the subdirectory is a symlink to the parent, we would
chase that, going around the loop suggesting a longer path until we hit
PATH_MAX.

Fix this by using the new API which provides the inode "for free," and
track whether we've seen this inode before. This is technically too
conservative since the inode may be for a directory on a different device,
but devices are not available for free so this would incur a cost. In
practice encountering the same inode twice with different devices in a
unique hierarchy is unlikely, and should it happen the consequences are
merely cosmetic: we fail to suggest a longer path.
2022-10-02 18:56:46 -07:00
ridiculousfish
0b47ba0642 Remove wreaddir and wreaddir_resolving
dir_iter_t has replaced these functions; we can remove them.
2022-10-02 18:48:16 -07:00
ridiculousfish
a2d816710f Adopt dir_iter_t in wildcard.cpp
Migrate wildcard's directory iteration to the new dir_iter_t.
Remove a now-unused function.
2022-10-02 18:48:16 -07:00
ridiculousfish
749d71288d Adopt dir_iter_t in descend_unique_hierarchy
Migrate this function from wreaddir_resolving to dir_iter_t
2022-10-02 18:48:16 -07:00
ridiculousfish
2a9366f938 Migrate highlight.cpp usage of wreaddir to dir_iter_t
Switch to the new API instead of using opendir directly.
2022-10-02 18:48:16 -07:00
ridiculousfish
36fbfef74c Switch uses of dir_t to dir_iter_t
dir_t was a thin wrapper around readdir; switch to the new dir_iter_t API
and remove dir_t.
2022-10-02 18:48:16 -07:00
ridiculousfish
b684f7b076 Introduce dir_iter_t
This introduces dir_iter_t, a new class for iterating the contents of a
directory. dir_iter_t encapsulates the logic that tries to avoid using
stat() to determine the type of a file, when possible.
2022-10-02 18:48:16 -07:00
Fabian Boehm
942308bf72 highlight: Unicode above 0x10FFFF is an error
This should really just be using read_unquoted_escape, where this was
changed in #1107
2022-09-29 17:16:42 +02:00
Fabian Boehm
5ada59996f Reduce write() calls for explicitly separated buffers
This can improve performance for `string split ""` for up to 1.8x.
2022-09-27 16:33:47 +02:00
ridiculousfish
9a3a67ba31 Migrate PUA constants out of wutil.h
These defines are only used inside the .cpp file. Place them in there
and switch to an enum.
2022-09-26 10:21:45 -07:00
Fabian Boehm
e726627993 Upgrade widechar_width to Unicode 15 2022-09-26 17:17:17 +02:00
Mahmoud Al-Qudsi
5d64b56127 Remove needless usage of maybe_t
builtin_function() never returns `none()`; this must have been leftover from a
previous version of the code.
2022-09-25 14:40:49 -05:00
Mahmoud Al-Qudsi
ff00d3ca08 fixup! Fix stomping of last_option_requires_param
Fix accidental misuse of maybe_t boolean operator instead of maybe_t payload.
2022-09-25 13:33:33 -05:00
Mahmoud Al-Qudsi
1811a2d725 Prevent undefined behavior by intercepting return -1
While we hardcode the return values for the rest of our builtins, the `return`
builtin bubbles up whatever the user returned in their fish script, allowing
invalid return values such as negative numbers to make it into our C++ side of
things.

In creating a `proc_status_t` from the return code of a builtin, we invoke
W_EXITCODE() which is a macro that shifts left the return code by some amount,
and left-shifting a negative integer is undefined behavior.

Aside from causing us to land in UB territory, it also can cause some negative
return values to map to a "successful" exit code of 0, which was probably not
the fish script author's intention.

This patch also adds error logging to help catch any inadvertent additions of
cases where a builtin returns a negative value (should one forget that unix
return codes are always positive) and an assertion protecting against UB.
2022-09-25 12:33:40 -05:00
Fabian Boehm
ccca5b553f Disable VQUIT for shell modes
This allows binding ctrl+\ by default.

Fixes #9234
2022-09-25 13:27:01 +02:00
ridiculousfish
bc4e7c3fea 'C_' function to use g_empty_string
Use the global empty string instead of having its own.
2022-09-23 14:32:20 -07:00
Mahmoud Al-Qudsi
1f41ce9446 Change localized_desc() to return a reference
Bubble up the reference returned by `C_()`.

This is a prerequisite for a bigger change I'm working on.
2022-09-23 14:01:02 -05:00
Mahmoud Al-Qudsi
1f91056539 Always return a const wcstring reference from _C()
This was always the case if HAVE_TEXT wasn't defined, but if it was then we were
coercing the result of `_C()` to a `const wchar_t *` pointer, because we were
returning the address of a constant zero-length wchar_t pointer. This reserves a
local static `wcstring` variable that we can return as the "no text" sentinel
and bubbles back the `wcstring` reference rather than decomposing it into a
pointer.

This is a prerequisite for a bigger change I'm working on.
2022-09-23 14:00:42 -05:00
Mahmoud Al-Qudsi
67c0a1db85 Reduce size of complete_entry_opt_t
It's gone from 136 bytes to a 128 bytes by rearranging the items in order of
decreasing alignment requirements. While this reduces the memory consumption
slightly (by around 6%) for each completion we have in-memory, that translates
to only around ~8KiB of savings for a command with 1000 possible completions,
which is nice but ultimately not that big of a deal.

The bigger benefit is that a single `complete_entry_t` might now fit in a cache
line, hopefully making the process of testing completions for matches more
cache friendly (and maybe even faster).
2022-09-23 12:09:26 -05:00
Mahmoud Al-Qudsi
0e9371cf24 complete_entry_opt_t: Rename list member condition to conditions
We used both a singular "condition" and a plural "condition" with the latter
referring to a list of the former. Clean that up.
2022-09-23 12:03:02 -05:00
Fabian Boehm
e69be38235 string: Reduce write() calls
The impact here depends on the command and how much output it
produces.

It's possible to get up to 1.5x - `string upper` being a good example,
or a no-op `string match '*'`.

But the more the command actually needs to do, the less of an effect
this has.
2022-09-22 22:41:35 +02:00
Fabian Boehm
7bc4c9674b builtins: Reduce streams.out.append/push_back calls
This basically immediately issues a "write()" if it's to a pipe or the
terminal.

That means we can reduce syscalls and improve performance, even by
doing something like

```c++
streams.out.append(somewcstring + L"\n");
```

instead of

```c++
streams.out.append(somewcstring);
streams.out.push_back(L'\n');
```

Some benchmarks of the

```fish
for i in (string repeat -n 2000 \n)
    $thing
end
```

variety:

1. `set` (printing variables) sped up 1.75x
2. `builtin -n` 1.60x
3. `jobs` 1.25x (with 3 jobs)
4. `functions` 1.20x
5. `math 1 + 1` 1.1x
6. `pwd` 1.1x

Piping yields similar results, there is no real difference when
outputting to a command substitution.
2022-09-22 22:41:35 +02:00
Fabian Boehm
c5b5dd7563 printf: Buffer output
This writes the output once per argument instead of once per format or
escaped char.

An egregious case:

```fish
printf (string repeat -n 200 \\x7f)%s\n (string repeat -n 2000 aaa\n)
```

Has been sped up by ~20x by reducing write() calls from 40000 to 200.

Even a simple

```fish
printf %s\n (string repeat -n 2000 aaa\n)
```

should now be ~1.2x faster by issuing 2000 instead of 4000 write
calls (the `\n` was written separately!).
2022-09-22 22:41:35 +02:00
Fabian Boehm
64927677c8 complete: Write each completion at once for --do-complete
This at least halves the number of "write()" calls we do if it goes to
a pipe or the terminal, or reduces them by 75% if there is a
description.

This makes

```fish
complete -c foo -xa "(seq 50000)"
complete -C"foo "
```

faster by 1.33x.
2022-09-22 22:41:35 +02:00
Mahmoud Al-Qudsi
42e177dc1b Fix build on macOS 10.10 Yosemite 2022-09-22 14:00:58 -05:00
Fabian Boehm
6a93d58797 wildcard: Use wreaddir_resolving if directories are needed
This uses wreaddir_resolving, which tries to use the dirent d_type
field if it exists. In that way, it can skip the `stat` to determine
if the given file is a directory.

This allows `cd` completions to skip stat in most cases:

```fish
strace -Ce newfstatat fish --no-config -c 'complete -C"cd /tmp/completion_test/"' >/dev/null
```

prints before:
```
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
100,00    0,002627           2      1033         4 newfstatat
```

after:

```
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
100,00    0,000054           1        31         3 newfstatat
```

for a directory with 1000 subdirectories.

(just `fish --no-config -c exit` does 26 newfstatat)

This should improve the situation with slow filesystems like fuse or
network fsen.

In case we have no d_type, we use `stat`, which would yield about the
same results.

The worst case is that we need directories *and* descriptions or the
"executable" flag (which we don't currently check for cd, if I read
this right?).
2022-09-21 19:49:17 +02:00
Fabian Boehm
a277f9aa93 WSL: Only skip ".dll" files for *executable* completions
This was overzealous and didn't allow anything named ".dll" in any
file completions.

This allows us to now add the cd completion fast path for WSL
2022-09-21 19:49:17 +02:00
Fabian Boehm
8b9a051b93 wreaddir_resolving: Don't add "/" for empty paths
This could end up trying to `stat()` a file in /, like "/glassdoor",
if the dir_path was empty.
2022-09-21 19:49:17 +02:00
Mahmoud Al-Qudsi
429534496a fixup! Fix stomping of last_option_requires_param 2022-09-20 22:37:17 -05:00
Mahmoud Al-Qudsi
663919228b Fix stomping of last_option_requires_param
This flag determines whether or not more shortopt switches will be offered up as
potential completions (vs only the payload for the last-parsed shortopt switch).

Previously, it was being stomped before it was determined whether or not two
`complete` rules with different `result_mode.requires_param` values were
actually resolved against the current command line or not, and the last
evaluated completion rule would win out.

There are two changes here:
* `last_option_requires_param` is only assigned if all associated conditions for
  a potential completion are also met, and
* If already assigned by a conflicting rule (which can only be user/developer
  error), `last_option_requires_param` is allowed to change from true to false
  but not the other way around (i.e. in case of a conflict, generate both
  payloads and other shortopt completions)

The first change is immediately noticeable and affects many of our own
completions, see the discussion in #9221 for an example regarding `git` where
`-c` has any of about a million different possible meanings depending on which
completion preconditions have been met. The second change should only happen if
a dev/user mistakenly enters a `complete -c ...` rule for the same shortopt more
than once, both with conditions matching, sometimes requiring an argument and
not sometimes not. It should be a rare occurence.
2022-09-20 21:49:30 -05:00
ridiculousfish
e7de342259 Remove a variable name in a defaulted function
This fixes a g++ 4.8 warning.
2022-09-20 14:41:22 -07:00
ridiculousfish
81c29d8891 clang-format and minor cleanup of tinyexpr.cpp
Clarifies some code and fixes some g++ 4.8 warnings.
2022-09-20 14:41:22 -07:00
ridiculousfish
5f4583b52d Revert "Re-implement macro to constexpr transition"
This reverts commit 3d8f98c395237982b12f6dbefa70925def637567.

In addition to the issues mentioned on the GitHub page for this commit,
it also broke the CentOS 7 build.

Note one can locally test the CentOS 7 build via:

    ./docker/docker_run_tests.sh ./docker/centos7.Dockerfile
2022-09-20 11:58:37 -07:00
Fabian Boehm
8b1da4b63d path: Actually use mtime instead of ctime
Fixes #9222
2022-09-20 16:10:17 +02:00
Mahmoud Al-Qudsi
3d8f98c395 Re-implement macro to constexpr transition
Be more careful with sign extension issues stemming from the differences in how
an untyped literal is promoted to an integer vs how a typed (and signed) `char`
is promoted to an integer.
2022-09-19 18:10:41 -05:00
Mahmoud Al-Qudsi
7c3e4a7ccb Revert "Convert constant macros to constexpr expressions"
This reverts commit e1626818f7c5706954db7c8a95a4a500e9243cf5.
2022-09-19 17:42:11 -05:00
Mahmoud Al-Qudsi
e1626818f7 Convert constant macros to constexpr expressions
Also convert some `const[expr] static xxx` to `const[expr] xxx` where it makes
sense to let the compiler deduce on its own whether or not to allocate storage
for a constant variable rather than imposing our view that it should have STATIC
storage set aside for it.

A few call sites were not making use of the `XXX_LEN` definitions and were
calling `strlen(XXX)` - these have been updated to use `const_strlen(XXX)`
instead.

I'm not sure if any toolchains will have raise any issues with these changes...
CI will tell!
2022-09-19 17:17:09 -05:00
ridiculousfish
9ec2e42e0e Revert "Reduce memory allocations for deduping completions"
The optimization takes references to strings which are stored in a vector,
and stores those references in a set; but the strings are simultaneously
being moved within the vector, which may invalidate those references.

It's  probably safe if you work through which particular strings are being
moved,  but as a matter of principle we shouldn't take references to elements
of a vector while the vector is being rearranged, absenet a clear improvement
on a benchmark.

This reverts commit d5561623aab5e6547f258e8165951beae11e94e3.
2022-09-17 11:57:44 -07:00
Mahmoud Al-Qudsi
d5561623aa Reduce memory allocations for deduping completions
Instead of adding the completions themselves to an `unordered_set` to
see if any are duplicates, just add a reference to the item instead.
2022-09-16 21:36:50 -05:00
Mahmoud Al-Qudsi
3ef047f242 Remove needless rank comparison
We've already removed any ranks that aren't equal to `best_rank` at this
point, so why are we comparing them again?
2022-09-16 21:34:10 -05:00
Johannes Altmanninger
31f7be3c8d fixup! reader: when updating commandline, also update rendered highlighting 2022-09-16 19:36:58 -05:00