1
0
mirror of https://github.com/systemd/systemd.git synced 2024-11-07 01:27:11 +03:00
Commit Graph

30988 Commits

Author SHA1 Message Date
Vito Caputo
647c7b7440 test-hashmap: test IteratedCache
Add some rudimentary testing of the new IteratedCache
2018-01-27 13:11:58 -08:00
Vito Caputo
5d4ba7f2b3 journal: use IteratedCache in sd-journal
This changes real_journal_next() to leverage the IteratedCache for
accelerating iteration across the open journal files.

journalctl timing comparisons with 100 journal files of 8MiB size
party to this boot:

Pre (~v235):
  # time ./journalctl -b --no-pager > /dev/null
  real    0m9.613s
  user    0m9.560s
  sys     0m0.053s

  # time ./journalctl -b --no-pager > /dev/null
  real    0m9.548s
  user    0m9.525s
  sys     0m0.023s

  # time ./journalctl -b --no-pager > /dev/null
  real    0m9.612s
  user    0m9.582s
  sys     0m0.030s

Post-IteratedCache:

  # time ./journalctl -b --no-pager > /dev/null
  real    0m8.449s
  user    0m8.425s
  sys     0m0.024s

  # time ./journalctl -b --no-pager > /dev/null
  real    0m8.409s
  user    0m8.382s
  sys     0m0.027s

  # time ./journalctl -b --no-pager > /dev/null
  real    0m8.410s
  user    0m8.350s
  sys     0m0.061s

~12.5% improvement, the benefit increases the more log files there are.
2018-01-27 13:11:58 -08:00
Vito Caputo
45ea84d8ed basic: implement the IteratedCache
Adds the basics of the IteratedCache and constructor support for the
Hashmap and OrderedHashmap types.

iterated_cache_get() is responsible for synchronizing the cache with
the associated Hashmap and making it available to the caller at the
supplied result pointers.  Since iterated_cache_get() may need to
allocate memory, it may fail, so callers must check the return value.

On success, pointer arrays containing pointers to the associated
Hashmap's keys and values, in as-iterated order, are returned in
res_keys and res_values, respectively.  Either may be supplied as NULL
to inhibit caching of the keys or values, respectively.

Note that if the cached Hashmap hasn't changed since the previous call
to iterated_cache_get(), and it's not a call activating caching of the
values or keys, the cost is effectively zero as the resulting pointers
will simply refer to the previously returned arrays as-is.

A cleanup function has also been added, iterated_cache_free().

This only frees the IteratedCache container and related arrays.  The
associated Hashmap, its keys, and values are not affected.  Also note
that the associated Hashmap does not automatically free its associated
IteratedCache when freed.

One could, in theory, safely access the arrays returned by a
successful iterated_cache_get() call after its associated Hashmap has
been freed, including the referenced values and keys.  Provided the
iterated_cache_get() was performed prior to the hashmap free, and that
the type of hashmap free performed didn't free keys and/or values as
well.
2018-01-27 13:11:50 -08:00
Vito Caputo
84dcca75b4 basic: track dirty state in HashmapBase
This only adds marking the HashmapBase as dirty, no clearing of
the dirty state happens yet.

No functional changes.
2018-01-26 16:04:35 -08:00
Yu Watanabe
9d4f242a53 core/dbus-execute: actually set PassEnvironment= (#7510)
Follow-up for #7444.
2017-11-30 12:34:58 +01:00
Yu Watanabe
606df9a5a5 man: fix typo (#7511) 2017-11-30 12:02:20 +01:00
Krzysztof Nowicki
8739f23e3c Fix SELinux labels in cgroup filesystem root directory (#7496)
When using SELinux with legacy cgroups the tmpfs on /sys/fs/cgroup is by
default labelled as tmpfs_t. This label is also inherited by the "cpu"
and "cpuacct" symbolic links. Unfortunately the policy expects them to
be labelled as cgroup_t, which is used for all the actual cgroup
filesystems. Failure to do so results in a stream of denials.

This state cannot be fixed reliably when the cgroup filesystem structure
is set-up as the SELinux policy is not yet loaded at this
moment. It also cannot be fixed later as the root of the cgroup
filesystem is remounted read-only. In order to fix it the root of the
cgroup filesystem needs to be temporary remounted read-write, relabelled
and remounted back read-only.
2017-11-30 11:59:29 +01:00
Lennart Poettering
949befd3f0
core: support upgrading from DynamicUser=0 to DynamicUser=1 for unit directories (#7507)
This makes sure we migrate /var/lib/<foo> if it exists to
/var/lib/private/<foo> if DynamicUser=1 is set. This is useful to allow
turning on DynamicUser= on services that previously didn't use it, and
we can deal with this, and migrate the relevant directories as
necessary.

Note that "downgrading" from DynamicUser=1 backto DynamicUser=0 works
too. However in that case we simply continue to use
/var/lib/private/<foo>, which works because /var/lib/<foo> is a symlink
there after all.
2017-11-30 11:52:39 +01:00
Yu Watanabe
191e9ef87d
Merge pull request #7444 from poettering/dbus-no-spec
unit writing escaping fixes + related fixes and additions
2017-11-30 14:50:36 +09:00
Vito Caputo
f9daa63a15
Merge pull request #7508 from poettering/journal-n-drop
journal: fix log message when dropping messages
2017-11-29 14:05:21 -08:00
Lennart Poettering
5908ff1c4b journal: fix log message when dropping messages
Fixes: #7506
2017-11-29 22:11:59 +01:00
Lennart Poettering
c6009ff029 update TODO 2017-11-29 20:12:26 +01:00
Lennart Poettering
c2288ade1e bus-unit-util: slightly tweak log output for unknown props
Pretty:

        Unknown assignment Foo=bar.

Prettier:

        Unknown assignment: Foo=bar
2017-11-29 20:12:26 +01:00
Lennart Poettering
f53bddf308 run: rearrange error handling when adding unit properties
Let's always leave logging to the call that actually added the fields to
the bus message. This way we don't get duplicate logging whenver
bus_append_unit_property_assignment() ends up being called, which does
all its logging on its own (and probably should do, as it can output
much more precise errors).
2017-11-29 20:12:26 +01:00
Lennart Poettering
147fe484c1 bus-unit-util: add proper MemorySwapMax= serialization
Fixes: #7505
2017-11-29 20:12:26 +01:00
Lennart Poettering
66a892ae3d core: accept MemorySwapMax= properties that are scaled, too
Let's do what we already do for MemoryMax= and friends for
MemorySwapMax= too.
2017-11-29 20:12:26 +01:00
Lennart Poettering
421eaea0d1 tty-ask-password-agent: drop NULL sentinel
found by coccinelle
2017-11-29 20:12:26 +01:00
Lennart Poettering
2d0bc68450 coccinelle: add a run-coccinelle.sh script that runs all scripts
One day we should start running something like this as part of CI so
that non-well-formed commits are not even accepted...
2017-11-29 20:12:26 +01:00
Lennart Poettering
a2d61f0746 coccinelle: fix IN_SET/!IN_SET scripts, and apply some changes it found
IN_SET only works for constant values, hence clarify that. Moreover, we
declared a statement "s" we never made use of. Drop it.

Also, for both scripts, let's support 10 items. More causes spatch to
die with "Stack overflow" for me.
2017-11-29 20:12:26 +01:00
Lennart Poettering
8dc3f4842b
Merge pull request #7493 from keszybz/revert-revert
Revert "meson: drop rootprefix option"
2017-11-29 18:56:26 +01:00
Lennart Poettering
a7af7362a3
Merge pull request #7504 from keszybz/two-mkosi-ideas
Two mkosi tweaks
2017-11-29 16:24:14 +01:00
Lennart Poettering
abc861e865
Merge pull request #7500 from poettering/journal-field-count
fix journald field counts
2017-11-29 16:21:48 +01:00
Zbigniew Jędrzejewski-Szmek
22ecadbee2 Add mkosi.output/ to .gitignore
It's not something that would ever be checked in, and mkosi along
with systemd, so this directory is likely to appear.
2017-11-29 14:33:56 +01:00
Zbigniew Jędrzejewski-Szmek
c7e501475a mkosi: do not build man pages 2017-11-29 14:33:55 +01:00
Zbigniew Jędrzejewski-Szmek
9a8e64b0e9 meson: emit a warning if rootprefix is set
It's printed at the end because it's the easiest to spot there.

When meson is upgraded to 0.43.1, we'll be able to use warning() instead.
2017-11-29 13:29:59 +01:00
Lennart Poettering
7b943bb7e3 tree-wide: use strv_isempty() instead of strv_length() == 0
It's a lot faster in many cases, since it's O(1) rather than O(n).
2017-11-29 12:41:08 +01:00
Lennart Poettering
5716965132 bus-unit-util: properly serialize EnvironmentFile= empty assignment
In that case we need to generate an empty array.
2017-11-29 12:34:12 +01:00
Lennart Poettering
7ae8162f29 tests: fix description of TEST-14-MACHINE-ID
It used the same description as TEST-01, and that's hardly helpful.
2017-11-29 12:34:12 +01:00
Lennart Poettering
50fb00b707 core: use safe_fclose() where we can 2017-11-29 12:34:12 +01:00
Lennart Poettering
7d7a99ac9f fileio: document why fileio-label.c and fileio.c are two different modules 2017-11-29 12:34:12 +01:00
Lennart Poettering
91d2f19594 documentation: add document listing all unit file settings also supported for transient units
Unfortunately, there are far too many properties currently not
accessible to transient units. The first step to opening them all up is
to figure out which ones they are.
2017-11-29 12:34:12 +01:00
Lennart Poettering
f6c66be1dc core: open up all ExecXYZ= fields of service units to transient units
Fixes: #7400
2017-11-29 12:34:12 +01:00
Lennart Poettering
45639f1be5 core: never remove "transient" and "control" directories from unit search path
This changes the unit search path logic to never drop the transient and
control directories from the unit search path. This is necessary as we
add new entries to both during runtime, due to the "systemctl
set-property" and transient unit logic.

Previously, the "transient" directory was created during early boot to
deal with this, but the "control" directories were not covered like
that. Creating the control directories early at boot is not possible
however, as /etc might be read-only then, and we do define a persistent
control directory. Hence, let's create these dirs on-demand when we need
them, and make sure the search path clean-up logic never drops them from
the search path even if they are initially missing.

(Also, always create these paths properly labelled)
2017-11-29 12:34:12 +01:00
Lennart Poettering
45a7b16bae core: don't reference rescue/emergency targets in --user mode
They are only defined for system mode, hence let's not check for them in
--user mode.

Follow-up for #7433
2017-11-29 12:34:12 +01:00
Lennart Poettering
0126c8f3f6 core: minor simplification 2017-11-29 12:34:12 +01:00
Lennart Poettering
fbc5883523 path-lookup: LookupPathsFlags are a flags type, hence define it like one
We shouldn't rely on C's incremental assignment of values of enums for
bit fields. That'll work only between the first two flags, but for
everything following will break horrible. Hence, let's avoid any
ambiguity here, and let's clearly define the flags as shifts of 1.
2017-11-29 12:34:12 +01:00
Lennart Poettering
e9f4aabd80 systemctl: reword "needs-reload" message a bit
This message is displayed either when the unit file itself is newer than
what is loaded, but also when any of the drop-ins is newer. Say so in
the message, in order not to confuse the user unnecessarily.
2017-11-29 12:34:12 +01:00
Lennart Poettering
721060d446 core: allow changing the description for units dynamically at runtime
This makes "systemctl set-property foobar.service Description=waldo"
work.

See: #7245
2017-11-29 12:34:12 +01:00
Lennart Poettering
e74f76ca86 tree-wide: generate SD_BUS_ERROR_INVALID_ARGS when we get invalid arguments on bus calls
Let's make sure that when we return a D-Bus error, we return a native
one, if we generate it ourselves, and use errno-based error
synthetization only if we received an errno ourselves. Yes, this makes
things slightly longer, but is highly misleading as we propagate D-Bus
errors, and not errnos to the client.
2017-11-29 12:34:12 +01:00
Lennart Poettering
2e59b241ca core: add proper escaping to writing of drop-ins/transient unit files
This majorly refactors the transient unit file and drop-in writing
logic, so that we properly C-escape and specifier-escape (% → %%)
everything we write out, so that when we read it back again, specifiers
are parsed that aren't supposed to be parsed.

This renames unit_write_drop_in() and friends by unit_write_setting().
The name change is supposed to clarify that the functions are not only
used to write drop-in files, but also transient unit files.

The previous "mode" parameter to this function is replaced by a more
generic "flags", which knows additional flags for implicit C-style and
specifier escaping before writing things out. This can cover most
properties where either form of escaping is defined. For the cases where
this isn't sufficient, we add helpers unit_escape_setting() and
unit_concat_strv() for escaping individual strings or strvs properly.

While we are at it, we also prettify generation of transient unit files:
we try to reduce the number of section headers written out: previously
we'd write the right section header our for each setting. With this
change we do so only if the setting lives in a different section than
the one before.

(This should also be considered preparation for when we add proper APIs
to systemd to write normal, persistant unit files through the bus API)
2017-11-29 12:34:12 +01:00
Lennart Poettering
ddd59d0c7f logind: fix sysfs change trigger code
We can't create files in sysfs, hence don't bother. Also if we ignore
the return value, do so explicitly by casting to void.
2017-11-29 12:32:57 +01:00
Lennart Poettering
98bad05e75 generators: be more careful when writing unit settings that support specifier expansion
Let's always escape strings we receive from the user before writing them
out to unit file settings that suppor specifier expansion, so that user
strings are transported as-is.
2017-11-29 12:32:57 +01:00
Lennart Poettering
b238be1e0d core: enable specifier expansion for What=/Where=/Type=/SourcePath= too
Using specifiers in these settings isn't particularly useful by itself,
but it unifies behaviour a bit. It's kinda surprising that What= in
mount units resolves specifies, but Where= does not. Hence let's add
that too. Also, it's surprising Where=/What= in mount units behaves
differently than in automount and swap units, hence resolve specifiers
there too. Then, Type= in mount units is nowadays an arbitrary,
sometimes non-trivial string (think fuse!), hence let's also expand
specifiers there, to match the rest of the mount settings.

This has the benefit that when writing code that generates unit files,
less care has to be taken to check whether escaping of specifiers is
necessary or not: broadly everything that takes arbitrary user strings
now does specifier expansion, while enums/numerics/booleans do not.
2017-11-29 12:32:57 +01:00
Lennart Poettering
6c5def2ff2 core: process C-style escapes in UnsetEnvironment=
We process C-style escapes in Environment=, hence we should process it
in UnsetEnvironment= too, as the latter accepts assignments much like
the former, including arbitrary values specified by the user.
2017-11-29 12:32:56 +01:00
Lennart Poettering
613613f1ee core: use config_parse_unit_string_printf() for decoding RebootArgument=
All other cases where we accept a reboot argument are decoded with
config_parse_unit_string_printf() rather than
config_parse_unit_path_printf(), and that's really the only thing what
makes sense here, hence adjust this here, too.
2017-11-29 12:32:56 +01:00
Lennart Poettering
35838def32 CODING_STYLE: document the order in which unescaping + specifier expansion should be done 2017-11-29 12:32:56 +01:00
Lennart Poettering
36444d2213 specifier: unify specifier implementations for user-related specifiers
The code in install-printf.c and unit-printf.c for these is pretty much
the same and very generic. Let's move this all over to the generic
specifier.c, and share the implementations.
2017-11-29 12:32:56 +01:00
Lennart Poettering
e82f30d17f specifier: add helper for escaping '%' characters to avoid making them subject for expansion
This is ultimately just a wrapper around strreplace(), but it makes
things a bit more self-descriptive.
2017-11-29 12:32:56 +01:00
Lennart Poettering
9d73565ac0 string-util: update strreplace() a bit, use GREEDY_REALLOC() 2017-11-29 12:32:56 +01:00
Lennart Poettering
6171b8225f core: drop specifier expansion when parsing transient dbus properties
Specifier expansion (much like C escape handling) should be a helper for
writing unit files, but should be nothing we do on programatic APIs. For
those, the client can do the necessary replacements anyway, and we
really should be careful with doing such string processing of data we
get via lower level programmatic APIs.

We currently do specifier expansion only for the env var transient unit
APIs, no other properties do this. Let's remove it here too, to be fully
systematic.

Yes, in a way this is API breakage, but then again this API isn't
documented yet, and an outlier, hence let's clear this up now, before it
is too late.
2017-11-29 12:32:53 +01:00