1
1
mirror of https://github.com/systemd/systemd-stable.git synced 2025-01-31 01:47:15 +03:00

2209 Commits

Author SHA1 Message Date
Daniel Braunwarth
bb2d46fc68 journalctl: respect --quiet flag during file concistency verification
Fixes #24563.

(cherry picked from commit 43deb1a8dce012c67a1285f7ef1bd40d971a4730)
(cherry picked from commit ac805eac156155bb8afdbaf704400e0ef854a6c9)
2022-11-04 13:02:21 +01:00
Franck Bui
077ca08b38 journal: preserve acls when rotating user journals with NOCOW attribute set
When restoring the COW flag for journals on BTRFS, the full journal contents
are copied into new files. But during these operations, the acls of the
previous files were lost and users were not able to access to their old
journal contents anymore.

(cherry picked from commit 11ee11dbb34587edcde5020c5baf1402dcc4ffdf)
2022-04-28 18:43:16 +02:00
Lennart Poettering
c901bc8680 journald: make sure SIGTERM handling doesn't get starved out
Fixes: #22642
(cherry picked from commit 19252b254861d8c9b56e2acaeb182812c8f07e52)
2022-04-01 14:27:11 +09:00
Zbigniew Jędrzejewski-Szmek
7310212021 meson: move files' closing brace to separate line
(cherry picked from commit 2f492a739c5edd09f1c16bc7da84c37a8744121a)
2022-03-05 21:03:32 +00:00
Zbigniew Jędrzejewski-Szmek
c76aba2b24 meson: do not use split() in file lists
The approach to use '''…'''.split() instead of a list of strings was initially
used when converting from automake because it allowed identical blocks of lines
to be used for both, making the conversion easier.

But over the years we have been using normal lists more and more, especially
when there were just a few filenames listed. This converts the rest.

No functional change.

(cherry picked from commit f1b98127ff6320648cc3dc876f3b6a5aa3af204b)
2022-03-05 21:03:32 +00:00
Daan De Meyer
1ef5fd61bd shared: Add more dlopen() tests
Add dlopen_dw(), dlopen_elf() and dlopen_pcre2() to the dlopen test.
To enable adding dlopen_pcre2(), we move pcre2-dlopen.h/c from
src/journal to src/shared.

(cherry picked from commit ee48779e05831a0ec5e1ba5e7ed5fe92aaca1d9e)
2022-03-05 21:03:32 +00:00
YmrDtnJu
88c8a48dfd Fix journald audit logging with fields > N_IOVEC_AUDIT_FIELDS.
ELEMENTSOF(iovec) is not the correct value for the newly introduced parameter m
to function map_all_fields because it is the maximum number of elements in the
iovec array, including those reserved for N_IOVEC_META_FIELDS. The correct
value is the current number of already used elements in the array plus the
maximum number to use for fields decoded from the kernel audit message.

(cherry picked from commit df4ec48f45f518b6926e02ef4d77c8ed1a8b4e2c)
2022-02-14 21:56:48 +00:00
Daan De Meyer
ef108015f3 journal: Remove unused arguments from journal_file_next_entry_for_data()
(cherry picked from commit f2eceb5268f0d26d53db5f9dad318dd45fd900bd)
2022-01-18 12:17:32 +01:00
Daan De Meyer
d72db2d7ca journal: Don't discard -b arg when followed by -e
Allowing -e to be used to view the last logs of a previous boot seems
like a useful feature so let's not discard -b options anymore when
followed by -e.

Fixes #22107

(cherry picked from commit 4d6455c0754e31ddc9590c7b9c9a373d82ec0ed4)
2022-01-18 12:11:20 +01:00
Daan De Meyer
c86461782f journal: Log filename when we fail to write an entry
(cherry picked from commit 5b4a634a03c54a4d18b01686ac86b1133b54a939)
2022-01-07 15:53:17 +01:00
Daan De Meyer
bdaa1b2dda journal: Log a better message when we're rotating because a file is full
The previous message was confusing errors. When we're rotating because
we've reached the file size limit, let's log a better message.

Fixes #22007.

(cherry picked from commit eff79e4e22e7c745fea259c4414f685363d9f16a)
2022-01-07 15:53:12 +01:00
Daan De Meyer
94c5a83c6e journal: Handle partially read HashItem's when punching holes 2021-12-21 02:29:57 +09:00
Yu Watanabe
d9338387d9
Merge pull request #21786 from keszybz/dirent-work
Make FOREACH_DIRENT and FOREACH_DIRENT_ALL declare the iterator variables
2021-12-17 04:07:35 +09:00
Zbigniew Jędrzejewski-Szmek
03f7ddf0bf meson: stop building out convenience libraries by default
The meson default for static_library() are:
build_by_default=true, install=false. We never interact with the
static libraries, and we only care about them as a stepping-stone towards
the installable executables or libraries. Thus let's only build them if
they are a dependency of something else we are building.

While at it, let's drop install:false, since this appears to be the default.

This change would have fixed the issue with lib_import_common failing
to build too: we wouldn't attempt to build it.

In practice this changes very little, because we generally only declare static
libraries where there's something in the default target that will make use of
them. But it seems to be a better pattern to set build_by_default to false.
2021-12-16 11:01:08 +01:00
Luca Boccassi
58f62d7079
Merge pull request #21784 from DaanDeMeyer/issue-21675
journal: Hole punching improvements
2021-12-15 22:38:00 +00:00
Zbigniew Jędrzejewski-Szmek
af3b864d76 Define FOREACH_DIRENT through FOREACH_DIRENT_ALL
As in the previous commit, 'de' is used as the iterator variable name.
2021-12-15 22:50:00 +01:00
Daan De Meyer
24040269ee journal: Stop reading in increments of block size during hole punching
Let's not try to be overly clever here. This code path is not overly
performance sensitive and we should avoid trying to outsmart the kernel
without proper benchmarking.
2021-12-15 18:24:29 +01:00
Daan De Meyer
d951ac5578 journal: Use 16kb buffer during hole punching
Let's use the same buffer size as used in as copy.h.
2021-12-15 18:22:17 +01:00
Daan De Meyer
cdbba44878 journal: Correctly advance offset when iterating hash table entries
pread() is not guaranteed to completely fill up the given buffer with
data which we assumed until now. Instead, only increment the offset by
the number of bytes that were actually read.
2021-12-15 18:21:19 +01:00
Daan De Meyer
a2799cc556 journal: Add a minimum hole size for hole punching
Let's not bother punching extremely small holes to avoid unnecessary
file fragmentation.
2021-12-15 18:17:22 +01:00
Zbigniew Jędrzejewski-Szmek
c7f0d9e5ac tree-wide: make FOREACH_DIRENT_ALL define the iterator variable
The variable is not useful outside of the loop (it'll always be null
after the loop is finished), so we can declare it inline in the loop.
This saves one variable declaration and reduces the chances that somebody
tries to use the variable outside of the loop.

For consistency, 'de' is used everywhere for the var name.
2021-12-15 16:19:13 +01:00
Zbigniew Jędrzejewski-Szmek
a2d7654f99
journactl: show info about journal range only at debug level (#21775)
The message that the "journal begins … ends …" has been always confusing to
users. (Before b91ae210e62 it was "logs begin … end …" which was arguably even
more confusing, but really the change in b91ae210e62 didn't substantially change
this.)

When the range shown is limited (by -e, -f, --since, or other options), it
doesn't really matter to the user what the oldest entries are, since they are
purposefully limiting the range. In fact, if we are showing the last few
entries with -e or -f, knowing that many months the oldest entries have is
completely useless.

And when such options are *not* used, the first entry generally corresponds to
the beginning of the range shown, and the last entry corresponds to the end of
that range. So again, it's not particularly useful, except when debugging
journalctl or such. Let's just treat it as a debug message.

Fixes #21491.
2021-12-15 06:30:15 +09:00
наб
2dd9285bac journalctl: have -f and -e imply no-value -b
Both of these take multiple seconds, best-case, to get the last few
entries, but do so instantly with -b
2021-12-10 07:27:09 +01:00
наб
a96fc72dbf journal: succes[s]fully typo
As found by fossies codespell:
https://fossies.org/linux/test/systemd-main.bb916f3.211208.tar.gz/codespell.html
2021-12-08 13:23:56 +00:00
Vito Caputo
74fb5be62d journal-file: require MMapCache* for journal_file_open()
Previously the MMapCache* was optionally NULL, which open would
handle by creating a new MMapCache* for the occasion.

This produced some slightly circuitous refcount-handling code in
the function, as well as arguably creating opportunities for
weirdness where an MMapCache* was intended to be supplied but
happened to be NULL, which this magic would then paper over.

In any case, this was basically only being utilized by tests,
apparently just to avoid having to create an MMapCache.  So
update the relevant tests to supply an MMapCache and make
journal_file_open() treat a NULL MMapCache* as fatal w/assert.
2021-12-07 14:39:20 -08:00
Vito Caputo
8b4fbbb0a1 journal: stop using JournalFile.mmap everywhere
Preparatory commit; before JournalFile can stop hanging onto its
copy of MMapCache, all these users need to find another way.

Most of the time these callers already have the MMapCache onhand,
so it's no big deal for them to just supply it.

journal_file_rotate() in particular needed to change, and it
seemed wise to not use the mmap_cache_fd_cache() accessor on
f->cache_fd, instead requiring the caller supply the cache to
use.  This was done with an eye towards a potential future where
the journal_file_archive() isolates the cache_fd to a private
cache, which the newly rotated-to file wouldn't be allowed to
use.  It's no biggie for the existing callers to just provide the
appropriate surviving cache.

Basically the mmap_cache_fd_cache() accessor was added just for
journal-verify.c's (ab)use of the mmap-cache.  Which, if the
ugly singleton MMapCache assumption ever goes away, can be
cleaned up to simply use a separate MMapCache for those search
arrays.
2021-12-07 13:00:25 -08:00
Daan De Meyer
d71ece3f0b journal: Fix disabling NO_COW on btrfs filesystems
Disabling NOCOW when data has been written to a file doesn't work.
Instead, when we're done writing to a journal file (after archiving),
let's rewrite the file with COW enabled. This also takes care of
properly defragmenting the file.

With zstd compression level 3, journal files are compressed to 12%
of their original size with default journal settings.

As rewriting the file might take a while since we also do an fsync()
after the rewrite, this work is done in the offline thread to avoid
blocking the journald event loop.
2021-12-06 22:17:41 +01:00
Daan De Meyer
461955ef4c journal: Update the JournalFile path when archiving
When we archive a path, we rename the file to indicate this. However,
until now, we didn't actually update the path member of the corresponding
JournalFile instance. Let's make sure we also update this to avoid
misuse of the old path later on.

This change also requires we save the previous path in journal_file_rotate()
since we need to open a new file at the previous path.
2021-12-06 22:17:40 +01:00
Daan De Meyer
764721cc07 journal: Move offlining logic to journald-file.c
With this change, the logic to write the final tag, emit the final
change notification and to offline the file moves from journal_file_close()
to journald_file_close(). Since all this logic is only executed when
the journal file is writable and all code that writes journal files
already uses journald_file_close() instead of journal_file_close(), this
change should not introduce any changes in behaviour.

Moving the offline related logic to journald-file.c allows us to use
code from src/shared in the offlining logic, more specifically, we can
use the file copying logic from copy.h to fix BTRFS filesystem compression
for journal files when archiving.
2021-12-06 22:17:40 +01:00
Daan De Meyer
035b0f8fe8 journal: Introduce journald-file.c for journal file write related logic
Currently, all the logic related to writing journal files lives in
journal-file.c which is part of libsystemd (sd-journal). Because it's
part of libsystemd, we can't depend on any code from src/shared.

To allow using code from src/shared when writing journal files, let's
gradually move the write related logic from journal-file.c to
journald-file.c in src/journal. This directory is not part of libsystemd
and as such can use code from src/shared.

We can safely remove any journal write related logic from libsystemd as
it's not used by any public APIs in libsystemd.

This commit introduces the new file along with the JournaldFile struct
which wraps an instance of JournalFile. The goal is to gradually move
more functions from journal-file.c and fields from JournalFile to
journald-file.c and JournaldFile respectively.

This commit also modifies all call sites that write journal files to
use JournaldFile instead of JournalFile. All sd-journal tests that
write journal files are moved to src/journal so they can make use of
journald-file.c.

Because the deferred closes logic is only used by journald, we move it
out of journal-file.c as well. In journal_file_open(), we would wait for
any remaining deferred closes for the file we're about to open to complete
before continuing if the file was not newly created. In journald_file_open(),
we call this logic unconditionally since it stands that if a file is newly
created, it can't have any outstanding deferred closes.

No changes in behavior are introduced aside from the earlier execution
of waiting for any deferred closes to complete when opening a new journal
file.
2021-12-06 22:17:38 +01:00
Yu Watanabe
8add30a03c tree-wide: use ERRNO_IS_TRANSIENT() 2021-11-30 23:06:43 +09:00
Zbigniew Jędrzejewski-Szmek
58c34be864 basic/util: rename u64log2 to log2u64
u64log2 was strangely named. We even have log2i and log2u right below
in that file.
2021-11-26 15:29:06 +01:00
Luca Boccassi
26bae8109d
Merge pull request #21431 from DaanDeMeyer/issue-19799
journal: Limit the number of audit fields we add to a message
2021-11-20 17:58:51 +00:00
Daan De Meyer
66e2bb282b journal: Limit the number of audit fields per log message
Similar to the kmsg handler, let's also limit the number of fields
we parse from audit messages.

Fixes #19799
2021-11-19 18:50:59 +00:00
Daan De Meyer
9c41618008 journal: Don't discard kmsg messages coming from journald itself
Previously, we discarded any kmsg messages coming from journald
itself to avoid infinite loops where potentially the processing
of a kmsg message causes journald to log one or more messages to
kmsg which then get read again by the kmsg handler, ...

However, if we completely disable logging whenever we're processing
a kmsg message coming from journald itself, we also prevent any
infinite loops as we can be sure that journald won't accidentally
generate logging messages while processing a kmsg log message.

This change allows us to store all journald logs generated during
the processing of log messages from other services in the system
journal. Previously these could only be found in kmsg which has
low retention, can't be queried using journalctl and whose logs
don't survive reboots.
2021-11-18 19:37:17 +00:00
Daan De Meyer
ceb4192df6 journal: Use mf as variable name for MapField
So we can have a variable m for the max iovec size in the next
commit like we do in the rest of the journal logic.
2021-11-18 13:28:14 +00:00
Daan De Meyer
4cdb970b5b journal: Use consistent naming for iovec in audit logic
Let's use iovec and n for the iovec variable and it's size just like
we do in the rest of the journal code.
2021-11-18 13:28:08 +00:00
Lennart Poettering
b085d22406 shared: split out UID allocation range stuff from user-record.h
user-record.[ch] are about the UserRecord JSON stuff, and the UID
allocation range stuff (i.e. login.defs handling) is a very different
thing, and complex enough on its own, let's give it its own c/h files.

No code changes, just some splitting out of code.
2021-11-13 08:10:11 +01:00
Lennart Poettering
4f538d7b22 tree-wide: use sd_event_source_disable_unref() where we can 2021-11-09 12:52:53 +01:00
Zbigniew Jędrzejewski-Szmek
384c2c3239 Make pager_open() return void 2021-11-03 15:24:56 +01:00
Zbigniew Jędrzejewski-Szmek
d7737416f5 journald: modernize code to send notifications
No change except making things a bit less verbose.
2021-11-03 11:29:49 +01:00
Zbigniew Jędrzejewski-Szmek
33539ffe59
Merge pull request #20988 from DaanDeMeyer/rotate-reason
journal: Improve rotation logging
2021-10-19 09:53:31 +02:00
Daan De Meyer
b2b7cf1df5 journal: Expand rotate log messages in journald
Make sure we always log when we rotate journals and always do so at
least at INFO log level. Doing so we make sure there's always a clear
reason available explaining why we rotated a journal.
2021-10-18 16:48:13 +01:00
Daan De Meyer
c8e6e1f10d journal: Add log level argument to journal_file_rotate_suggested()
When journald is rotating a file, we'd like to log the reason at
LOG_INFO or higher instead of LOG_DEBUG. For journalctl --header,
logging the reason at a level higher than LOG_DEBUG doesn't really
make sense. To accomodate both use cases, make the log level used
by journal_file_rotate_suggested() configurable.
2021-10-18 15:26:06 +02:00
Lennart Poettering
2f82562bad alloc-util: add strdupa_safe() + strndupa_safe() and use it everywhere
Let's define two helpers strdupa_safe() + strndupa_safe() which do the
same as their non-safe counterparts, except that they abort if called
with allocations larger than ALLOCA_MAX.

This should ensure that all our alloca() based allocations are subject
to this limit.

afaics glibc offers three alloca() based APIs: alloca() itself,
strndupa() + strdupa(). With this we have now replacements for all of
them, that take the limit into account.
2021-10-14 15:57:52 +02:00
Lennart Poettering
698cec65c4 tree-wide: port all calls to alloca() to alloca_safe() 2021-10-14 15:57:52 +02:00
Lennart Poettering
a995ce4768 util: define initializer for 'struct ucred' that properly invalidates all fields
i.e. let's make sure to invalid uid/gid to UID_INVAID + GID_INVALID
instead of zero.
2021-10-11 15:37:37 +02:00
Lennart Poettering
f435195925 basic: spit out chase_symlinks() from fs-util.[ch] → chase-symlinks.[ch] 2021-10-05 16:14:37 +02:00
Yu Watanabe
f782eee68a journal,network,timesync: fix segfault on 32bit timeval/timespec systems
Fixes #20741.
2021-09-16 11:47:44 +02:00
Yu Watanabe
73bf0f2ace
Merge pull request #20738 from mrc0mmand/ci-llvm-13
ci: build with clang-13
2021-09-16 09:32:21 +09:00