IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
We check lazyfree_lazy_server_del in sunionDiffGenericCommand
to see if we need to lazyfree the temp set. Now do the same in
zunionInterDiffGenericCommand to lazyfree the temp zset.
This is a minor change, follow #5903. Also improved the comments.
Additionally, avoid creating unused zset object in ZINTERCARD,
results in some 10% performance improvement.
So far clients being blocked and unblocked by a module command would
update the c->woff variable and so WAIT was ineffective and got released
without waiting for the command actions to propagate.
This seems to have existed since forever, but not for RM_BlockClientOnKeys.
It is problematic though to know if the module did or didn't propagate
anything in that command, so for now, instead of adding an API, we'll
just update the woff to the latest offset when unblocking, this will
cause the client to possibly wait excessively, but that's not that bad.
XREAD only supports a special ID of $ and XREADGROUP only supports ^.
make sure not to suggest the wrong one when rerunning an error about unbalanced ID arguments
Co-authored-by: Oran Agra <oran@redislabs.com>
Rather than a fixed iovcnt for connWritev, support maxiov per connection type instead.
A minor change to reduce memory for struct connection.
Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
It was missing in #12223, and the reply-schemas daily
was failing:
```
jsonschema.exceptions.ValidationError: 'nothing' is not valid under any of the given schemas
Failed validating 'oneOf' in schema[0]['properties']['loglevel']:
{'oneOf': [{'const': 'debug'},
{'const': 'verbose'},
{'const': 'notice'},
{'const': 'warning'},
{'const': 'unknown'}]}
On instance['loglevel']:
'nothing'
```
The light version only shows the table sizes, while the pre-existing
version that shows chain length stats is reachable with the `full` argument.
This should allow looking into rehashing state, even on huge dicts, on
which we're afraid to run the command for fear of causing a server freeze.
Also, fix a possible overflow in dictGetStats.
This pr can get two performance benefits:
1. Stop redundant initialization when most robj objects are created
2. LRU_CLOCK will no longer be called in io threads, so we can avoid the `atomicGet`
Another code optimization:
deleted the redundant judgment in dbSetValue, no matter in LFU or LRU, the lru field inold
robj is always the freshest (it is always updated in lookupkey), so we don't need to judge if in LFU
We add a new loglevel 'nothing' to disable logging in #12133.
This PR syncs that config change to sentinel. Because in #11214
we support modifying loglevel in runtime.
Although I think sentinel doesn't need this nothing config,
it's better to be consistent.
In #12166, we removed a call to CLUSTER SLAVES, which
then caused reply-schemas ci to fail:
```
WARNING! The following commands were not hit at all:
cluster|slaves
ERROR! at least one command was not hit by the tests
```
Because we already have command output that cover CLUSTER REPLICAS
elsewhere, here we simply add some dummy tests to fix the ci.
This commit excludes aux fields from the output of the `cluster nodes` and `cluster replicas` command.
We may decide to re-introduce them in some form or another in the future, but not in v7.2.
Users can record logs of different levels by setting the `loglevel`.
However, sometimes there are many logs even at the warning level,
which can affect the performance of Redis.
For example, when a user accesses the tls-port using a non-encrypted link,
Redis will log lots of "# Error accepting a client connection: ...".
We can provide the ability to disable logging so that users can temporarily turn
off logging and turn it back on after the problem is resolved.
previously the argv wasn't freed so would leak. not a common case, but should be handled.
Solution: move RUN_AS_USER setup and error exit to the right place.
this way, when we do `goto cleanup` (instead of return) it'll automatically do the right thing (including autoMemoryAdd)
Removed the user argument from moduleAllocTempClient (reverted to the state before 6e993a5)
Co-authored-by: Oran Agra <oran@redislabs.com>
Optimized HRANDFIELD and ZRANDMEMBER commands as in #8444,
CASE 3 under listpack encoding. Boost optimization to CASE 2.5.
CASE 2.5 listpack only. Sampling unique elements, in non-random order.
Listpack encoded hashes / zsets are meant to be relatively small, so
HRANDFIELD_SUB_STRATEGY_MUL / ZRANDMEMBER_SUB_STRATEGY_MUL
isn't necessary and we rather not make copies of the entries. Instead, we
emit them directly to the output buffer.
Simple benchmarks shows it provides some 400% improvement in HRANDFIELD
and ZRANGESTORE both in CASE 3.
Unrelated changes: remove useless setTypeRandomElements and fix a typo.
A single SPOP with command with count argument resulted in many SPOP
commands being propagated to the replica.
This is inefficient because the key name is repeated many times, and is also
being looked-up many times.
also it results in high QPS metrics on the replica.
To solve that, we flush batches of 1024 fields per SPOP command.
Co-authored-by: zhaozhao.zz <zhaozhao.zz@alibaba-inc.com>
For zsets that will eventually be stored as the skiplist encoding (has a dict),
we can convert it to skiplist ahead of time. This change checks the number
of arguments in the ZADD command, and converts the data-structure
if the number of new entries exceeds the listpack-max-entries configuration.
This can cause us to over-allocate memory if there are duplicate entries in the
input, which is unexpected.
For ZRANGESTORE, we know the size of the zset, so we can expand
the dict in advance, to avoid the temporary dict from being rehashed
while it grows.
Simple benchmarks shows it provides some 4% improvement in ZADD and 20% in ZRANGESTORE
Current tests for BITFIELD_RO command are skipped in the external mode,
and therefore reply-schemas-validator reports a coverage error.
This PR adds basic tests to increase coverage.
When `replicationFeedSlaves()` serializes a command, it repeatedly calls
`feedReplicationBuffer()` to feed it to the replication backlog piece by piece.
It is unnecessary to call `incrementalTrimReplicationBacklog()` for every small
amount of data added with `feedReplicationBuffer()` as the chance of the conditions
being met for trimming are very low and these frequent calls add up to a notable
performance cost. Instead, we will only attempt trimming when a new block is added
to the replication backlog.
Using redis-benchmark to saturate a local redis server indicated a performance
improvement of around 3-3.5% for 100 byte SET commands with this change.
Extend SENTINEL CONFIG SET and SENTINEL CONFIG GET to be
compatible with variadic CONFIG SET and CONFIG GET and allow multiple
parameters to be modified in a single call atomically.
Co-authored-by: Oran Agra <oran@redislabs.com>
The measured latency(duration) includes the list below, which can be shown by `INFO STATS`.
```
eventloop_cycles // ever increasing counter
eventloop_duration_sum // cumulative duration of eventloop in microseconds
eventloop_duration_cmd_sum // cumulative duration of executing commands in microseconds
instantaneous_eventloop_cycles_per_sec // average eventloop count per second in recent 1.6s
instantaneous_eventloop_duration_usec // average single eventloop duration in recent 1.6s
```
Also added some experimental metrics, which are shown only when `INFO DEBUG` is called.
This section isn't included in the default INFO, or even in `INFO ALL` and the fields in this section
can change in the future without considering backwards compatibility.
```
eventloop_duration_aof_sum // cumulative duration of writing AOF
eventloop_duration_cron_sum // cumulative duration cron jobs (serverCron, beforeSleep excluding IO and AOF)
eventloop_cmd_per_cycle_max // max number of commands executed in one eventloop
eventloop_duration_max // max duration of one eventloop
```
All of these are being reset by CONFIG RESETSTAT
When using scan in redis-cli, the SCAN COUNT is fixed, which means the
full scan can take a long time if there are a lot of keys, this will let users specify
a bigger COUNT option.
This pattern is from COMMAND INFO:
Returns information about one, multiple or all commands.
Also re-generate commands.def, the GEO change was missing in #12151.
The test failed on MacOS:
```
*** [err]: EXPIRE precision is now the millisecond in tests/unit/expire.tcl
Expected 'somevalue {}' to equal or match '{} {}'
```
`set a [r get x]`, even though we tried 10 times, sometimes we
still get {}, this is a time-sensitive test.
In this PR, we add the following changes:
1. More attempts, change it from 10 to 30.
2. More tolerant, change the `after 900` to `after 800`.
In addition, we judging $a in advance and changing `after 1100`
to `after 300`, this will save us some times.
For sets and hashes that will eventually be stored as the hash encoding, it's much faster to immediately convert them to their hash encoding and then perform the insertions since it avoids the O(N) search and frequent reallocations. This change checks the number of arguments in the incoming command, and converts the data-structure if the number of new entries exceeds the listpack-max-entries configuration. This can cause us to over-allocate memory if their are duplicate entries in the input, which is unexpected.
unstable
Summary:
throughput summary: 805.54 requests per second
latency summary (msec):
avg min p50 p95 p99 max
61.908 25.680 68.351 73.279 75.967 79.295
hset-improvement
Summary:
throughput summary: 4701.46 requests per second
latency summary (msec):
avg min p50 p95 p99 max
10.546 0.832 11.959 12.471 13.119 14.967
Upgrade to jemalloc 5.3.0 hoping to resolve a potential for deadlock in a fork child
see https://github.com/jemalloc/jemalloc/issues/2402
steps:
* Upgrade subtree according to the instructions in deps/README
* update iget_defrag_hint by following changes to arena_dalloc_no_tcache
When `RM_ZsetAdd()`/`RM_ZsetIncrby()`/`RM_StreamAdd()` fails, if a new key happens to
be created using `moduleCreateEmptyKey()`, we should clean up the empty key.
## Test
1) Add new module commands(`zset.add` and `zset.incrby`) to cover `RM_ZsetAdd()`/`RM_ZsetIncrby()`.
2) Add a large-memory test to cover `RM_StreamAdd()`.
When master reboot from RDB, if rsi in RDB is valid we should not free replication backlog, even if master_repl_offset or repl-offset is 0.
Since if master doesn't send any data to replicas master_repl_offset is 0, it's a valid number.
A clear example:
1. start a master and apply some write commands, the master's master_repl_offset is 0 since it has no replicas.
2. stop write commands on master, and start another instance and replicaof the master, trigger an FULLRESYNC
3. the master's master_repl_offset is still 0 (set a large number for repl-ping-replica-period), do BGSAVE and restart the master
4. master load master_repl_offset from RDB's rsi and it's still 0, and we should make sure replica can partially resync with master.
Tests occasionally fail since #12000:
```
*** [err]: query buffer resized correctly when not idle in tests/unit/querybuf.tcl
Expected 0 > 32768 (context: type eval line 11 cmd {assert {$orig_test_client_qbuf > 32768}} proc ::test)
*** [err]: query buffer resized correctly with fat argv in tests/unit/querybuf.tcl
query buffer should not be resized when client idle time smaller than 2s
```
The reason may be because we set hz to 100, querybuf shrinks before we count
client_query_buffer. We avoid this problem by setting pause-cron to 1.
1. reset the readraw mode after a test that uses it. undetected since the
only test after that on the same server didn't read any replies.
2. fix a cross slot issue that was undetected in cluster mode because
readraw doesn't throw exceptions on errors.
Technically declaring a prototype with an empty declaration has been deprecated since the early days of C, but we never got a warning for it. C2x will apparently be introducing a breaking change if you are using this type of declarator, so Clang 15 has started issuing a warning with -pedantic. Although not apparently a problem for any of the compiler we build on, if feels like the right thing is to properly adhere to the C standard and use (void).
