1
1
mirror of https://github.com/systemd/systemd-stable.git synced 2025-01-12 09:17:44 +03:00
Commit Graph

23899 Commits

Author SHA1 Message Date
Vito Caputo
7a24f3bf2f journal: coalesce ftruncate()s in 250ms windows
Prior to this change every journal append causes an ftruncate() for the
sake of inotify propagation of the mmap-based writes.

With this change the notification is deferred up to ~250ms, coalescing
any repeated journal writes during the deferred period into a single
ftruncate().  The ftruncate() call isn't free and doing it on every
append adds unnecessary overhead and latency in the journald event loop.

Introduces journal_file_enable_post_change_timer() which manages a
timer on the provided sd-event instance for scheduling coalesced
ftruncates.  The ftruncate() behavior is unchanged unless
journal_file_enable_post_change_timer() is called on the JournalFile.

While not a tremendous improvement, profiling systemd-journald event loop
latencies using instrumentation as introduced by 34b8751 it was observed that
coalescing the ftruncates was low-hanging fruit worth pursuing.

Note orders 12 and 13 shifting left into order 11 and order 6 dipping into
order 5:

Unmodified:
     log2(us)   1 2 3  4 5  6   7   8  9   10 11   12   13 14 15 16 17 18 19
                -----------------------------------------------------------
[10685.414572]  0 0 0  0 38 602 61  2  290 60 1643 2554 13 1  4  1  0  0  1
[10690.415114]  0 0 0  0 0  646 54  7  309 44 2073 2148 17 1  3  0  0  0  1
[10695.415509]  0 0 0  0 1  650 73  3  324 37 2071 2270 9  0  0  1  0  1  0
[10700.416297]  0 0 0  0 0  659 50  4  318 38 2111 2152 6  0  1  0  0  1  1
[10705.417136]  0 0 0  0 2  660 48  4  320 38 2129 2146 12 1  1  0  0  1  1
[10710.489114]  0 0 0  0 0  673 38  3  321 37 1925 2339 7  0  0  0  0  1  1
[10715.489613]  0 0 0  0 3  656 64  8  317 48 2365 2007 7  0  0  0  0  0  1

Coalesced:
     log2(us)   1 2 3  4 5  6   7   8  9   10 11   12   13 14 15 16 17 18 19
                -----------------------------------------------------------
[ 6169.161360]  0 0 0  1 24 786 54  11 389 24 4192 771  6  4  0  0  1  0  1
[ 6174.161705]  0 0 0  1 18 800 35  6  380 27 3977 893  3  1  0  0  1  0  1
[ 6179.162741]  0 0 0  1 28 768 51  4  391 16 3998 831  5  3  0  0  0  0  2
[ 6184.162856]  0 0 0  0 19 770 60  2  376 26 3795 1004 9  5  1  0  1  0  1
[ 6189.163279]  0 0 0  0 28 761 49  7  372 27 3729 1056 3  2  0  0  1  0  1
[ 6194.164255]  0 0 0  0 25 785 49  7  394 19 3996 908  6  3  2  0  0  0  1
[ 6199.164658]  0 0 0  0 29 797 35  5  389 18 3995 898  3  4  1  1  1  0  1

The remaining high-order delays are a result of the synchronous fsyncs in
systemd-journald, beyond the scope of this commit.
2016-01-14 16:36:07 -08:00
Daniel Mack
c57d67f718 Merge pull request #2096 from teg/resolved-cache
Misc resolved cache fixes
2015-12-10 20:48:42 +01:00
Tom Gundersen
a257f9d4a5 resolved: cache - only stringify RR keys when in debug mode
This is in the fast path, so let's not do all this work unneccessarily.
2015-12-10 20:17:49 +01:00
Tom Gundersen
02c2857b8d resolved: cache - don't flush the cache of mDNS records unneccesarily
When the DNS_RESOURCE_KEY_CACHE_FLUSH flag is not set for an mDNS packet, we should not flush
the cache for RRs with matching keys. However, we were unconditionally flushing the cache
also for these packets.

Now mark all packets as cache_flush by default, except for these mDNS packets, and respect
that flag in the cache handling.

This fixes 90325e8c2e.
2015-12-10 20:15:58 +01:00
Tom Gundersen
2250592422 TODO 2015-12-10 19:47:47 +01:00
Daniel Mack
7b56301a88 Merge pull request #2133 from poettering/import-drop-dkr
importd: drop dkr support
2015-12-10 17:40:39 +01:00
Tom Gundersen
d7ce6c9464 resolved: cache - rework which RR types we apply redirection to
The logic of dns_cache_get() is now:
 - look up the precise key;
 - look up NXDOMAIN item;
 - if an RR type that may be redirected
   (i.e., not CNAME, DNAME, RRSIG, NSEC, NSEC3, SIG, KEY, or
   NXT) look up a correpsonding CNAME or DNAME record;
 - look up a corresponding NSEC record;

Before this change we would give up before potentially finding
negative cache entries for DNAME, CNAME and NSEC records, we
would return NSEC records for aliases where we had DNAME or CNAME
records available and we would incorrectly try to redirect DNSSEC RRs.
2015-12-10 17:06:26 +01:00
Tom Gundersen
c3cb6dc20b resolved: cache - improve logging
Some DNS servers will hand out negative answers without SOA records,
these can not be cached, so log about that fact.
2015-12-10 17:06:26 +01:00
Tom Gundersen
71e136699c resolved: cache - don't cache NXDOMAIN by TYPE
An NXDOMAIN entry means there are no RRs of any type for a name,
so only cache by CLASS + NAME, rather than CLASS + NAME + TYPE.
2015-12-10 17:06:26 +01:00
Tom Gundersen
5d27351f85 resolved: cache - do negative caching only on the canonical name
Apart from dropping redundant information, this fixes an issue
where, due to broken DNS servers, we can only be certain of whether
an apparent NODATA response is in fact an NXDOMAIN response after
explicitly resolving the canonical name. This issue is outlined in
RFC2308. Moreover, by caching NXDOMAIN for an existing name, we
would mistakenly return NXDOMAIN for types which should not be
redirected. I.e., a query for AAAA on test-nx-1.jklm.no correctly
returns NXDOMAIN, but a query for CNAME should return the record
and a query for DNAME should return NODATA.

Note that this means we will not cache an NXDOMAIN response in the
presence of redirection, meaning one redundant roundtrip in case the
name is queried again.
2015-12-10 17:04:42 +01:00
Daniel Mack
34130ea7aa Merge pull request #2134 from jorgenschaefer/detect-ipv6-with-sockstat6
Use /proc/net/sockstat6 to detect IPv6 support
2015-12-10 16:54:57 +01:00
Lennart Poettering
b43d75c378 importd: drop dkr support
The current code is not compatible with current dkr protocols anyway,
and dkr has a different focus ("microservices") than nspawn anyway
("whole machine containers"), hence drop support for it, we cannot
reasonably keep this up to date, and it creates the impression we'd
actually care for the microservices usecase.
2015-12-10 16:54:41 +01:00
Lennart Poettering
4d3f0f66e6 Merge pull request #2135 from zonque/resolved-mdns-3
resolved: more mDNS specific bits (3)
2015-12-10 16:49:31 +01:00
Daniel Mack
fe2dfc8b49 resolved: make sure the packet's transaction ID is always 0 for mDNS
RFC6762, 18.1:

   In multicast query messages, the Query Identifier SHOULD be set to
   zero on transmission.
2015-12-10 16:32:03 +01:00
Daniel Mack
40fa4728eb resolved: discard any reply packet that contains a bogus name
Only .in-addr.arpa and .local are considered local in mDNS, so discard the
packet if anything else is thrown at us.
2015-12-10 16:32:02 +01:00
Tom Gundersen
553947b77c Merge pull request #2129 from poettering/dnssec3
Third DNSSEC patch series
2015-12-10 15:22:18 +01:00
Jorgen Schaefer
7377964d4c Use /proc/net/sockstat6 to detect IPv6 support
The file /sys/module/ipv6 does not exist in all container
implementations (e.g. Virtuozzo). Using /proc/net/sockstat6
detects IPv6 support reliably in these environments, too.

This file does not exist when the kernel is not compiled with
IPv6 support, or if IPv6 support is disabled, so simply checking
for existence should be a suitable check.

Fixes #2059
2015-12-10 13:47:19 +01:00
Lennart Poettering
312501458d Merge pull request #2086 from evverx/fix-journal-upload-installation
build: fix systemd-journal-upload installation
2015-12-10 12:05:26 +01:00
Lennart Poettering
ada64a0cff README: Recommend kinvolk regarding engineering services
They are our friends, do systemd development, hence add them.
2015-12-10 11:57:08 +01:00
Lennart Poettering
e93af635c7 Merge pull request #2076 from keszybz/downgrade-masked-unit-message
core: do not warn about Wants depencencies on masked units
2015-12-10 11:40:23 +01:00
Lennart Poettering
c842ff2488 resolved: rename dns_transaction_prepare_next_attempt()
Let's simply call it dns_transaction_prepare(), so that we have the nice
cycle for prepare() → go() → emit() → process().

After all it's pretty clear that what we prepare there, and we dont call
the others go_next_attempt(), emit_next_attempt() or
process_next_attempt().
2015-12-10 11:35:52 +01:00
Lennart Poettering
f649045c10 journal: make mmap_cache_unref() a NOP when NULL is passed, like all other destructors 2015-12-10 11:35:52 +01:00
Lennart Poettering
9eae2bf318 resolved: don't accept doing queries for invalid RR types 2015-12-10 11:35:52 +01:00
Lennart Poettering
547973dea7 resolved: chase DNSKEY/DS RRs when doing look-ups with DNSSEC enabled
This adds initial support for validating RRSIG/DNSKEY/DS chains when
doing lookups. Proof-of-non-existance, or proof-of-unsigned-zones is not
implemented yet.

With this change DnsTransaction objects will generate additional
DnsTransaction objects when looking for DNSKEY or DS RRs to validate an
RRSIG on a response. DnsTransaction objects are thus created for three
reasons now:

1) Because a user asked for something to be resolved, i.e. requested by
   a DnsQuery/DnsQueryCandidate object.
2) As result of LLMNR RR probing, requested by a DnsZoneItem.
3) Because another DnsTransaction requires the requested RRs for
   validation of its own response.

DnsTransactions are shared between all these users, and are GC
automatically as soon as all of these users don't need a specific
transaction anymore.

To unify the handling of these three reasons for existance for a
DnsTransaction, a new common naming is introduced: each DnsTransaction
now tracks its "owners" via a Set* object named "notify_xyz", containing
all owners to notify on completion.

A new DnsTransaction state is introduced called "VALIDATING" that is
entered after a response has been receieved which needs to be validated,
as long as we are still waiting for the DNSKEY/DS RRs from other
DnsTransactions.

This patch will request the DNSKEY/DS RRs bottom-up, and then validate
them top-down.

Caching of RRs is now only done after verification, so that the cache is
not poisoned with known invalid data.

The "DnsAnswer" object gained a substantial number of new calls, since
we need to add/remove RRs to it dynamically now.
2015-12-10 11:35:52 +01:00
Lennart Poettering
aa89931749 resolved: when matching up DNSKEY and DS RRs, it's fine if we don't support the DNSKEY's algorithm
As long as we support the digest we are good.
2015-12-10 11:35:52 +01:00
Lennart Poettering
15accc2765 resolved: when matching up RRSIG and DNSKEY RRs, use the RRSIG's signer name, not the owner name
When the DNSKEY is in higher zone, then that's OK, and we need to check
the RRSIG's signer name against the DNSKEY hence.
2015-12-10 11:35:52 +01:00
Lennart Poettering
6c5e8fbf4e resolved: fix sorting of RRsets
We actually maintain an array of pointers to RRs, not of RRs themselves,
fix the qsort() invocation accordingly.
2015-12-10 11:35:52 +01:00
Lennart Poettering
2f763887b8 resolved: grow DnsAnswer exponentially
When increasing the DnsAnswer array, don't operate piecemeal, grow the
array exponentially.

This way, the default logic for DnsAnswer allocations matches the
behaviour for GREEDY_REALLOC and suchlike, and we can reduce the number
of necessary allocations.
2015-12-10 11:35:52 +01:00
Lennart Poettering
48d5616b92 resolved: log when we chase a CNAME RR 2015-12-10 11:35:52 +01:00
Lennart Poettering
b5efcf29d2 resolved: reenable caching for LLMNR
This got borked in 547493c5ad.
2015-12-10 11:35:52 +01:00
Lennart Poettering
8af5b88322 resolved: split out check whether reply matches our question
It's complicated enough, it deserves its own call.

(Also contains some unrelated whitespace, comment and assertion changes)
2015-12-10 11:35:52 +01:00
Lennart Poettering
8b5b564947 resolved: IXFR and AXFR cannot be the type of RRs, only of RR keys
Enforce this while parsing RRs.
2015-12-10 11:35:52 +01:00
Lennart Poettering
d75acfb059 resolved: when parsing DNS packets, handle OPT RR specially
As soon as we encounter the OPT RR while parsing, store it in a special
field in the DnsPacket structure. That way, we won't be confused if we
iterate through RRs, and can check that there's really only one of these
RRs around.
2015-12-10 11:35:52 +01:00
Lennart Poettering
c296dd2eea resolved: refuse modifying DnsAnswer objects that have more than one reference
DnsAnswer objects should be considered immutable after having passed to
more than one user, i.e. with a reference counter > 1. Enforce that in
code, so that we can track down misuses easier.
2015-12-10 11:28:02 +01:00
Lennart Poettering
d12bf2bdff resolved: fix libgcrypt error checking
libgcrypt encodes the error source in the error code, we need to mask
that away before comparing error codes.
2015-12-10 11:28:02 +01:00
Lennart Poettering
d28ac939c1 build-sys: libgcrypt error messages make no sense without libgpg-error
Hence, pull in this library too, if we need libgcrypt.
2015-12-10 11:28:02 +01:00
Lennart Poettering
d42800f18e resolved: split out logic to flush DnsAnswer objects
Let's simplify things, by making this a function call of its own.
2015-12-10 11:28:02 +01:00
Lennart Poettering
7bcffc2efa resolved: honour RFC6761's ban on the invalid TLD 2015-12-10 11:28:02 +01:00
Lennart Poettering
aa44ee274c resolved: fix DNS_ANSWER_FOREACH_IFINDEX() to not collide with user defined ifindex variable 2015-12-10 11:28:02 +01:00
Lennart Poettering
0bb4749d1f resolved: partially revert 5eefe54
Quoting @teg:

"Contrary to what the comment said, we always verify redirect chains in
full, and cache all the CNAME records. There is therefore no need to
do extra negative caching along a CNAME chain."

This simply steals @teg's commit since we'll touch the SOA matching case
in a later patch, and rather want this bit gone, so that we don't have
to "fix" it, only to remove it later on.
2015-12-10 11:28:01 +01:00
Lennart Poettering
c52a97b896 resolved: when outputting RRs in text form, append a trailing dot to owner names
After all, that's how this is done in DNS, and is particularly important
if we look a DS/DNSKEY RRs for the root zone itself, where the owner
name would otherwise be shown as completely empty (i.e. missing).
2015-12-10 11:28:01 +01:00
Lennart Poettering
4d247a6cd3 resolved: shortcut RR comparisons if pointers match
When iterating through RR lists we frequently end up comparing RRs and
RR keys with themselves, hence att a minor optimization to check ptr
values first, before doing a deep comparison.
2015-12-10 11:28:01 +01:00
Lennart Poettering
c84e853934 resolved: fix parameter type of dns_type_is_pseudo()
DNS RR types are uint16_t after all, treat them as such.
2015-12-10 11:28:01 +01:00
Lennart Poettering
e4b0e86225 Merge pull request #2056 from evverx/expose-soft-limits-on-the-bus
Expose soft limits on the bus
2015-12-10 11:20:03 +01:00
Lennart Poettering
59c74f2175 Merge pull request #2128 from zonque/resolved-mdns-2
resolved: more mDNS specific bits (2)
2015-12-10 11:10:30 +01:00
Daniel Mack
261f3673c1 resolved: add more linked packets for overlong known answers
For mDNS, if we're unable to stuff all known answers into the given packet,
allocate a new one, push the RR into that one and link it to the current
one.
2015-12-10 10:21:50 +01:00
Daniel Mack
80a62095dc resolved: handle linked packet in dns_scope_emit()
In dns_scope_emit(), walk the list of additional packets and emit all of
them. Set the TC bit in all but the last of them.

This is specific to mDNS, so an assertion is triggered if used with other
protocols.
2015-12-10 10:20:55 +01:00
Daniel Mack
9c49156383 resolved: add support for linked packets
For mDNS, we need to support the TC bit in case the list of known answers
exceed the maximum packet size.

For this, add a 'more' pointer to DnsPacket for an additional packet.

When a packet is unref'ed, the ->more packet is also unrefed, so it
sufficient to only keep track of the 1st packet in a chain.
2015-12-10 10:20:54 +01:00
Daniel Mack
dbfbb6e776 resolved: add dns_packet_set_flags()
We need to support the TC bit in queries in case known answers exceed the
maximum packet size. Factor out the flags compilation to
dns_packet_set_flags() and make it externally available.
2015-12-10 10:20:23 +01:00
Lennart Poettering
2b48c44bc4 Merge pull request #2108 from evverx/fix-distcheck-for-disable-resolved
build-sys: move "dist" parts out of conditionals
2015-12-09 20:48:40 +01:00