shaba/systemd-stable
1
1
Fork 0
mirror of https://github.com/systemd/systemd-stable.git synced 2024-12-23 17:34:00 +03:00
Code

Merge pull request #16514 from keszybz/zstd-decompress-fix

Browse Source 
Fix coredumpctl operation with zstd-compressed journals
This commit is contained in:
Zbigniew Jędrzejewski-Szmek 2020-07-22 10:40:19 +02:00 committed by GitHub
commit f25e9eda52
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
10 changed files with 268 additions and 137 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
TODO
View File

@ -1029,7 +1029,7 @@ Features:
- journal: add a setgid "systemd-journal" utility to invoke from libsystemd-journal, which passes fds via STDOUT and does PK access - journal: add a setgid "systemd-journal" utility to invoke from libsystemd-journal, which passes fds via STDOUT and does PK access
- journactl: support negative filtering, i.e. FOOBAR!="waldo", - journactl: support negative filtering, i.e. FOOBAR!="waldo",
and !FOOBAR for events without FOOBAR. and !FOOBAR for events without FOOBAR.
- journal: store timestamp of journal_file_set_offline() int he header, - journal: store timestamp of journal_file_set_offline() in the header,
so it is possible to display when the file was last synced. so it is possible to display when the file was last synced.
- journal-send.c, log.c: when the log socket is clogged, and we drop, count this and write a message about this when it gets unclogged again. - journal-send.c, log.c: when the log socket is clogged, and we drop, count this and write a message about this when it gets unclogged again.
- journal: find a way to allow dropping history early, based on priority, other rules - journal: find a way to allow dropping history early, based on priority, other rules
@ -1071,6 +1071,7 @@ Features:
them via machined, and also watch containers coming and going. them via machined, and also watch containers coming and going.
Benefit: nspawn --ephemeral would start working nicely with the journal. Benefit: nspawn --ephemeral would start working nicely with the journal.
- assign MESSAGE_ID to log messages about failed services - assign MESSAGE_ID to log messages about failed services
- check if loop in decompress_blob_xz() is necessary
* add a test if all entries in the catalog are properly formatted. * add a test if all entries in the catalog are properly formatted.
(Adding dashes in a catalog entry currently results in the catalog entry (Adding dashes in a catalog entry currently results in the catalog entry

143
man/sd_journal_get_data.xml
View File

@ -18,6 +18,7 @@
<refnamediv> <refnamediv>
<refname>sd_journal_get_data</refname> <refname>sd_journal_get_data</refname>
<refname>sd_journal_enumerate_data</refname> <refname>sd_journal_enumerate_data</refname>
<refname>sd_journal_enumerate_available_data</refname>
<refname>sd_journal_restart_data</refname> <refname>sd_journal_restart_data</refname>
<refname>SD_JOURNAL_FOREACH_DATA</refname> <refname>SD_JOURNAL_FOREACH_DATA</refname>
<refname>sd_journal_set_data_threshold</refname> <refname>sd_journal_set_data_threshold</refname>
@ -44,6 +45,13 @@
<paramdef>size_t *<parameter>length</parameter></paramdef> <paramdef>size_t *<parameter>length</parameter></paramdef>
</funcprototype> </funcprototype>
<funcprototype>
<funcdef>int <function>sd_journal_enumerate_available_data</function></funcdef>
<paramdef>sd_journal *<parameter>j</parameter></paramdef>
<paramdef>const void **<parameter>data</parameter></paramdef>
<paramdef>size_t *<parameter>length</parameter></paramdef>
</funcprototype>
<funcprototype> <funcprototype>
<funcdef>void <function>sd_journal_restart_data</function></funcdef> <funcdef>void <function>sd_journal_restart_data</function></funcdef>
<paramdef>sd_journal *<parameter>j</parameter></paramdef> <paramdef>sd_journal *<parameter>j</parameter></paramdef>
@ -73,24 +81,18 @@
<refsect1> <refsect1>
<title>Description</title> <title>Description</title>
<para><function>sd_journal_get_data()</function> gets the data <para><function>sd_journal_get_data()</function> gets the data object associated with a specific field
object associated with a specific field from the current journal from the current journal entry. It takes four arguments: the journal context object, a string with the
entry. It takes four arguments: the journal context object, a field name to request, plus a pair of pointers to pointer/size variables where the data object and its
string with the field name to request, plus a pair of pointers to size shall be stored in. The field name should be an entry field name. Well-known field names are listed in
pointer/size variables where the data object and its size shall be <citerefentry><refentrytitle>systemd.journal-fields</refentrytitle><manvolnum>7</manvolnum></citerefentry>,
stored in. The field name should be an entry field name. but any field can be specified. The returned data is in a read-only memory map and is only valid until
Well-known field names are listed in the next invocation of <function>sd_journal_get_data()</function>,
<citerefentry><refentrytitle>systemd.journal-fields</refentrytitle><manvolnum>7</manvolnum></citerefentry>. <function>sd_journal_enumerate_data()</function>,
The returned data is in a read-only memory map and is only valid <function>sd_journal_enumerate_available_data()</function>, or when the read pointer is altered. Note
until the next invocation of that the data returned will be prefixed with the field name and <literal>=</literal>. Also note that, by
<function>sd_journal_get_data()</function> or default, data fields larger than 64K might get truncated to 64K. This threshold may be changed and turned
<function>sd_journal_enumerate_data()</function>, or the read off with <function>sd_journal_set_data_threshold()</function> (see below).</para>
pointer is altered. Note that the data returned will be prefixed
with the field name and '='. Also note that, by default, data fields
larger than 64K might get truncated to 64K. This threshold may be
changed and turned off with
<function>sd_journal_set_data_threshold()</function> (see
below).</para>
<para><function>sd_journal_enumerate_data()</function> may be used <para><function>sd_journal_enumerate_data()</function> may be used
to iterate through all fields of the current entry. On each to iterate through all fields of the current entry. On each
@ -99,15 +101,18 @@
format as with <function>sd_journal_get_data()</function> and also format as with <function>sd_journal_get_data()</function> and also
follows the same life-time semantics.</para> follows the same life-time semantics.</para>
<para><function>sd_journal_enumerate_available_data()</function> is similar to
<function>sd_journal_enumerate_data()</function>, but silently skips any fields which may be valid, but
are too large or not supported by current implementation.</para>
<para><function>sd_journal_restart_data()</function> resets the <para><function>sd_journal_restart_data()</function> resets the
data enumeration index to the beginning of the entry. The next data enumeration index to the beginning of the entry. The next
invocation of <function>sd_journal_enumerate_data()</function> invocation of <function>sd_journal_enumerate_data()</function>
will return the first field of the entry again.</para> will return the first field of the entry again.</para>
<para>Note that the <function>SD_JOURNAL_FOREACH_DATA()</function> <para>Note that the <function>SD_JOURNAL_FOREACH_DATA()</function> macro may be used as a handy wrapper
macro may be used as a handy wrapper around around <function>sd_journal_restart_data()</function> and
<function>sd_journal_restart_data()</function> and <function>sd_journal_enumerate_available_data()</function>.</para>
<function>sd_journal_enumerate_data()</function>.</para>
<para>Note that these functions will not work before <para>Note that these functions will not work before
<citerefentry><refentrytitle>sd_journal_next</refentrytitle><manvolnum>3</manvolnum></citerefentry> <citerefentry><refentrytitle>sd_journal_next</refentrytitle><manvolnum>3</manvolnum></citerefentry>
@ -139,18 +144,88 @@
<refsect1> <refsect1>
<title>Return Value</title> <title>Return Value</title>
<para><function>sd_journal_get_data()</function> returns 0 on <para><function>sd_journal_get_data()</function> returns 0 on success or a negative errno-style error
success or a negative errno-style error code. If the current entry code. <function>sd_journal_enumerate_data()</function> and
does not include the specified field, -ENOENT is returned. If <function>sd_journal_enumerate_available_data()</function> return a positive integer if the next field
<citerefentry><refentrytitle>sd_journal_next</refentrytitle><manvolnum>3</manvolnum></citerefentry> has been read, 0 when no more fields remain, or a negative errno-style error code.
has not been called at least once, -EADDRNOTAVAIL is returned. <function>sd_journal_restart_data()</function> doesn't return anything.
<function>sd_journal_enumerate_data()</function> returns a <function>sd_journal_set_data_threshold()</function> and <function>sd_journal_get_threshold()</function>
positive integer if the next field has been read, 0 when no more return 0 on success or a negative errno-style error code.</para>
fields are known, or a negative errno-style error code.
<function>sd_journal_restart_data()</function> returns nothing. <refsect2>
<function>sd_journal_set_data_threshold()</function> and <title>Errors</title>
<function>sd_journal_get_threshold()</function> return 0 on
success or a negative errno-style error code.</para> <para>Returned errors may indicate the following problems:</para>
<variablelist>
<varlistentry id='EINVAL'>
<term><constant>-EINVAL</constant></term>
<listitem><para>One of the required parameters is <constant>NULL</constant> or invalid.
</para></listitem>
</varlistentry>
<varlistentry id='ECHILD'>
<term><constant>-ECHILD</constant></term>
<listitem><para>The journal object was created in a different process.</para></listitem>
</varlistentry>
<varlistentry id='EADDRNOTAVAIL'>
<term><constant>-EADDRNOTAVAIL</constant></term>
<listitem><para>The read pointer is not positioned at a valid entry;
<citerefentry><refentrytitle>sd_journal_next</refentrytitle><manvolnum>3</manvolnum></citerefentry>
or a related call has not been called at least once.</para></listitem>
</varlistentry>
<varlistentry id='ENOENT'>
<term><constant>-ENOENT</constant></term>
<listitem><para>The current entry does not include the specified field.</para>
</listitem>
</varlistentry>
<varlistentry id='ENOMEM'>
<term><constant>-ENOMEM</constant></term>
<listitem><para>Memory allocation failed.</para></listitem>
</varlistentry>
<varlistentry id='ENOBUFS'>
<term><constant>-ENOBUFS</constant></term>
<listitem><para>A compressed entry is too large.</para></listitem>
</varlistentry>
<varlistentry id='E2BIG'>
<term><constant>-E2BIG</constant></term>
<listitem><para>The data field is too large for this computer architecture (e.g. above 4 GB on a
32-bit architecture).</para></listitem>
</varlistentry>
<varlistentry id='EPROTONOSUPPORT'>
<term><constant>-EPROTONOSUPPORT</constant></term>
<listitem><para>The journal is compressed with an unsupported method or the journal uses an
unsupported feature.</para></listitem>
</varlistentry>
<varlistentry id='EBADMSG'>
<term><constant>-EBADMSG</constant></term>
<listitem><para>The journal is corrupted (possibly just the entry being iterated over).
</para></listitem>
</varlistentry>
<varlistentry id='EIO'>
<term><constant>-EIO</constant></term>
<listitem><para>An I/O error was reported by the kernel.</para></listitem>
</varlistentry>
</variablelist>
</refsect2>
</refsect1> </refsect1>
<refsect1> <refsect1>

103
man/sd_journal_query_unique.xml
View File

@ -18,6 +18,7 @@
<refnamediv> <refnamediv>
<refname>sd_journal_query_unique</refname> <refname>sd_journal_query_unique</refname>
<refname>sd_journal_enumerate_unique</refname> <refname>sd_journal_enumerate_unique</refname>
<refname>sd_journal_enumerate_available_unique</refname>
<refname>sd_journal_restart_unique</refname> <refname>sd_journal_restart_unique</refname>
<refname>SD_JOURNAL_FOREACH_UNIQUE</refname> <refname>SD_JOURNAL_FOREACH_UNIQUE</refname>
<refpurpose>Read unique data fields from the journal</refpurpose> <refpurpose>Read unique data fields from the journal</refpurpose>
@ -33,6 +34,13 @@
<paramdef>const char *<parameter>field</parameter></paramdef> <paramdef>const char *<parameter>field</parameter></paramdef>
</funcprototype> </funcprototype>
<funcprototype>
<funcdef>int <function>sd_journal_enumerate_available_unique</function></funcdef>
<paramdef>sd_journal *<parameter>j</parameter></paramdef>
<paramdef>const void **<parameter>data</parameter></paramdef>
<paramdef>size_t *<parameter>length</parameter></paramdef>
</funcprototype>
<funcprototype> <funcprototype>
<funcdef>int <function>sd_journal_enumerate_unique</function></funcdef> <funcdef>int <function>sd_journal_enumerate_unique</function></funcdef>
<paramdef>sd_journal *<parameter>j</parameter></paramdef> <paramdef>sd_journal *<parameter>j</parameter></paramdef>
@ -58,33 +66,31 @@
<refsect1> <refsect1>
<title>Description</title> <title>Description</title>
<para><function>sd_journal_query_unique()</function> queries the <para><function>sd_journal_query_unique()</function> queries the journal for all unique values the
journal for all unique values the specified field can take. It specified field can take. It takes two arguments: the journal to query and the field name to look
takes two arguments: the journal to query and the field name to for. Well-known field names are listed on
look for. Well-known field names are listed on <citerefentry><refentrytitle>systemd.journal-fields</refentrytitle><manvolnum>7</manvolnum></citerefentry>,
<citerefentry><refentrytitle>systemd.journal-fields</refentrytitle><manvolnum>7</manvolnum></citerefentry>. but any field can be specified. Field names must be specified without a trailing
Field names must be specified without a trailing '='. After this <literal>=</literal>. After this function has been executed successfully the field values may be queried
function has been executed successfully the field values may be using <function>sd_journal_enumerate_unique()</function> and
queried using <function>sd_journal_enumerate_unique()</function>. <function>sd_journal_enumerate_available_unique()</function>. Invoking one of those calls will change the
Invoking this call a second time will change the field name being field name being queried and reset the enumeration index to the first field value that matches.</para>
queried and reset the enumeration index to the first field value
that matches.</para>
<para><function>sd_journal_enumerate_unique()</function> may be <para><function>sd_journal_enumerate_unique()</function> may be used to iterate through all data fields
used to iterate through all data fields which match the previously which match the previously selected field name as set with
selected field name as set with <function>sd_journal_query_unique()</function>. On each invocation the next field data matching the field
<function>sd_journal_query_unique()</function>. On each invocation name is returned. The order of the returned data fields is not defined. It takes three arguments: the
the next field data matching the field name is returned. The order journal object, plus a pair of pointers to pointer/size variables where the data object and its size
of the returned data fields is not defined. It takes three shall be stored. The returned data is in a read-only memory map and is only valid until the next
arguments: the journal context object, plus a pair of pointers to invocation of <function>sd_journal_enumerate_unique()</function>. Note that the data returned will be
pointer/size variables where the data object and its size shall be prefixed with the field name and <literal>=</literal>. Note that this call is subject to the data field
stored in. The returned data is in a read-only memory map and is size threshold as controlled by <function>sd_journal_set_data_threshold()</function> and only the initial
only valid until the next invocation of part of the field up to the threshold is returned. An error is returned for fields which cannot be
<function>sd_journal_enumerate_unique()</function>. Note that the retrieved. See the error list below for details.</para>
data returned will be prefixed with the field name and '='. Note
that this call is subject to the data field size threshold as <para><function>sd_journal_enumerate_available_unique()</function> is similar to
controlled by <function>sd_journal_enumerate_unique()</function>, but silently skips any fields which may be valid, but
<function>sd_journal_set_data_threshold()</function>.</para> are too large or not supported by current implementation.</para>
<para><function>sd_journal_restart_unique()</function> resets the <para><function>sd_journal_restart_unique()</function> resets the
data enumeration index to the beginning of the list. The next data enumeration index to the beginning of the list. The next
@ -92,11 +98,9 @@
will return the first field data matching the field name will return the first field data matching the field name
again.</para> again.</para>
<para>Note that the <para>Note that the <function>SD_JOURNAL_FOREACH_UNIQUE()</function> macro may be used as a handy wrapper
<function>SD_JOURNAL_FOREACH_UNIQUE()</function> macro may be used around <function>sd_journal_restart_unique()</function> and
as a handy wrapper around <function>sd_journal_enumerate_available_unique()</function>.</para>
<function>sd_journal_restart_unique()</function> and
<function>sd_journal_enumerate_unique()</function>.</para>
<para>Note that these functions currently are not influenced by <para>Note that these functions currently are not influenced by
matches set with <function>sd_journal_add_match()</function> but matches set with <function>sd_journal_add_match()</function> but
@ -111,13 +115,29 @@
<refsect1> <refsect1>
<title>Return Value</title> <title>Return Value</title>
<para><function>sd_journal_query_unique()</function> returns 0 on <para><function>sd_journal_query_unique()</function> returns 0 on success or a negative errno-style error
success or a negative errno-style error code. code. <function>sd_journal_enumerate_unique()</function> and and
<function>sd_journal_enumerate_unique()</function> returns a <function>sd_journal_query_available_unique()</function> return a positive integer if the next field data
positive integer if the next field data has been read, 0 when no has been read, 0 when no more fields remain, or a negative errno-style error code.
more fields are known, or a negative errno-style error code. <function>sd_journal_restart_unique()</function> doesn't return anything.</para>
<function>sd_journal_restart_unique()</function> returns
nothing.</para> <refsect2>
<title>Errors</title>
<para>Returned errors may indicate the following problems:</para>
<variablelist>
<xi:include href="sd_journal_get_data.xml" xpointer="EINVAL"/>
<xi:include href="sd_journal_get_data.xml" xpointer="ECHILD"/>
<xi:include href="sd_journal_get_data.xml" xpointer="EADDRNOTAVAIL"/>
<xi:include href="sd_journal_get_data.xml" xpointer="ENOENT"/>
<xi:include href="sd_journal_get_data.xml" xpointer="ENOBUFS"/>
<xi:include href="sd_journal_get_data.xml" xpointer="E2BIG"/>
<xi:include href="sd_journal_get_data.xml" xpointer="EPROTONOSUPPORT"/>
<xi:include href="sd_journal_get_data.xml" xpointer="EBADMSG"/>
<xi:include href="sd_journal_get_data.xml" xpointer="EIO"/>
</variablelist>
</refsect2>
</refsect1> </refsect1>
<refsect1> <refsect1>
@ -131,10 +151,9 @@
<refsect1> <refsect1>
<title>Examples</title> <title>Examples</title>
<para>Use the <function>SD_JOURNAL_FOREACH_UNIQUE</function> macro <para>Use the <function>SD_JOURNAL_FOREACH_UNIQUE</function> macro to iterate through all values a field
to iterate through all values a field of the journal can take. The of the journal can take (and which can be accessed on the given architecture and are not compressed with
following example lists all unit names referenced in the an unsupported mechanism). The following example lists all unit names referenced in the journal:</para>
journal:</para>
<programlisting><xi:include href="journal-iterate-unique.c" parse="text" /></programlisting> <programlisting><xi:include href="journal-iterate-unique.c" parse="text" /></programlisting>
</refsect1> </refsect1>

2
src/home/homectl-pkcs11.c
View File

@ -16,12 +16,12 @@ struct pkcs11_callback_data {
X509 *cert; X509 *cert;
}; };
#if HAVE_P11KIT
static void pkcs11_callback_data_release(struct pkcs11_callback_data *data) { static void pkcs11_callback_data_release(struct pkcs11_callback_data *data) {
erase_and_free(data->pin_used); erase_and_free(data->pin_used);
X509_free(data->cert); X509_free(data->cert);
} }
#if HAVE_P11KIT
static int pkcs11_callback( static int pkcs11_callback(
CK_FUNCTION_LIST *m, CK_FUNCTION_LIST *m,
CK_SESSION_HANDLE session, CK_SESSION_HANDLE session,

103
src/journal/compress.c
View File

@ -259,10 +259,10 @@ int decompress_blob_lz4(const void *src, uint64_t src_size,
int decompress_blob_zstd( int decompress_blob_zstd(
const void *src, uint64_t src_size, const void *src, uint64_t src_size,
void **dst, size_t *dst_alloc_size, size_t* dst_size, size_t dst_max) { void **dst, size_t *dst_alloc_size, size_t *dst_size, size_t dst_max) {
#if HAVE_ZSTD #if HAVE_ZSTD
size_t space; uint64_t size;
assert(src); assert(src);
assert(src_size > 0); assert(src_size > 0);
@ -271,38 +271,40 @@ int decompress_blob_zstd(
assert(dst_size); assert(dst_size);
assert(*dst_alloc_size == 0 || *dst); assert(*dst_alloc_size == 0 || *dst);
if (src_size > SIZE_MAX/2) /* Overflow? */ size = ZSTD_getFrameContentSize(src, src_size);
return -ENOBUFS; if (IN_SET(size, ZSTD_CONTENTSIZE_ERROR, ZSTD_CONTENTSIZE_UNKNOWN))
space = src_size * 2; return -EBADMSG;
if (dst_max > 0 && space > dst_max)
space = dst_max;
if (!greedy_realloc(dst, dst_alloc_size, space, 1)) if (dst_max > 0 && size > dst_max)
size = dst_max;
if (size > SIZE_MAX)
return -E2BIG;
if (!(greedy_realloc(dst, dst_alloc_size, MAX(ZSTD_DStreamOutSize(), size), 1)))
return -ENOMEM; return -ENOMEM;
for (;;) { _cleanup_(ZSTD_freeDCtxp) ZSTD_DCtx *dctx = ZSTD_createDCtx();
size_t k; if (!dctx)
return -ENOMEM;
k = ZSTD_decompress(*dst, *dst_alloc_size, src, src_size); ZSTD_inBuffer input = {
if (!ZSTD_isError(k)) { .src = src,
*dst_size = k; .size = src_size,
return 0; };
} ZSTD_outBuffer output = {
if (ZSTD_getErrorCode(k) != ZSTD_error_dstSize_tooSmall) .dst = *dst,
return zstd_ret_to_errno(k); .size = *dst_alloc_size,
};
if (dst_max > 0 && space >= dst_max) /* Already at max? */ size_t k = ZSTD_decompressStream(dctx, &output, &input);
return -ENOBUFS; if (ZSTD_isError(k)) {
if (space > SIZE_MAX / 2) /* Overflow? */ log_debug("ZSTD decoder failed: %s", ZSTD_getErrorName(k));
return -ENOBUFS; return zstd_ret_to_errno(k);
space *= 2;
if (dst_max > 0 && space > dst_max)
space = dst_max;
if (!greedy_realloc(dst, dst_alloc_size, space, 1))
return -ENOMEM;
} }
assert(output.pos >= size);
*dst_size = size;
return 0;
#else #else
return -EPROTONOSUPPORT; return -EPROTONOSUPPORT;
#endif #endif
@ -326,7 +328,7 @@ int decompress_blob(
src, src_size, src, src_size,
dst, dst_alloc_size, dst_size, dst_max); dst, dst_alloc_size, dst_size, dst_max);
else else
return -EBADMSG; return -EPROTONOSUPPORT;
} }
int decompress_startswith_xz(const void *src, uint64_t src_size, int decompress_startswith_xz(const void *src, uint64_t src_size,
@ -456,9 +458,6 @@ int decompress_startswith_zstd(
const void *prefix, size_t prefix_len, const void *prefix, size_t prefix_len,
uint8_t extra) { uint8_t extra) {
#if HAVE_ZSTD #if HAVE_ZSTD
_cleanup_(ZSTD_freeDCtxp) ZSTD_DCtx *dctx = NULL;
size_t k;
assert(src); assert(src);
assert(src_size > 0); assert(src_size > 0);
assert(buffer); assert(buffer);
@ -466,7 +465,14 @@ int decompress_startswith_zstd(
assert(prefix); assert(prefix);
assert(*buffer_size == 0 || *buffer); assert(*buffer_size == 0 || *buffer);
dctx = ZSTD_createDCtx(); uint64_t size = ZSTD_getFrameContentSize(src, src_size);
if (IN_SET(size, ZSTD_CONTENTSIZE_ERROR, ZSTD_CONTENTSIZE_UNKNOWN))
return -EBADMSG;
if (size < prefix_len + 1)
return 0; /* Decompressed text too short to match the prefix and extra */
_cleanup_(ZSTD_freeDCtxp) ZSTD_DCtx *dctx = ZSTD_createDCtx();
if (!dctx) if (!dctx)
return -ENOMEM; return -ENOMEM;
@ -481,30 +487,17 @@ int decompress_startswith_zstd(
.dst = *buffer, .dst = *buffer,
.size = *buffer_size, .size = *buffer_size,
}; };
size_t k;
for (;;) { k = ZSTD_decompressStream(dctx, &output, &input);
k = ZSTD_decompressStream(dctx, &output, &input); if (ZSTD_isError(k)) {
if (ZSTD_isError(k)) { log_debug("ZSTD decoder failed: %s", ZSTD_getErrorName(k));
log_debug("ZSTD decoder failed: %s", ZSTD_getErrorName(k)); return zstd_ret_to_errno(k);
return zstd_ret_to_errno(k);
}
if (output.pos >= prefix_len + 1)
return memcmp(*buffer, prefix, prefix_len) == 0 &&
((const uint8_t*) *buffer)[prefix_len] == extra;
if (input.pos >= input.size)
return 0;
if (*buffer_size > SIZE_MAX/2)
return -ENOBUFS;
if (!(greedy_realloc(buffer, buffer_size, *buffer_size * 2, 1)))
return -ENOMEM;
output.dst = *buffer;
output.size = *buffer_size;
} }
assert(output.pos >= prefix_len + 1);
return memcmp(*buffer, prefix, prefix_len) == 0 &&
((const uint8_t*) *buffer)[prefix_len] == extra;
#else #else
return -EPROTONOSUPPORT; return -EPROTONOSUPPORT;
#endif #endif

9
src/journal/journal-internal.h
View File

@ -127,3 +127,12 @@ void journal_print_header(sd_journal *j);
#define JOURNAL_FOREACH_DATA_RETVAL(j, data, l, retval) \ #define JOURNAL_FOREACH_DATA_RETVAL(j, data, l, retval) \
for (sd_journal_restart_data(j); ((retval) = sd_journal_enumerate_data((j), &(data), &(l))) > 0; ) for (sd_journal_restart_data(j); ((retval) = sd_journal_enumerate_data((j), &(data), &(l))) > 0; )
/* All errors that we might encounter while extracting a field that are not real errors,
* but only mean that the field is too large or we don't support the compression. */
static inline bool JOURNAL_ERRNO_IS_UNAVAILABLE_FIELD(int r) {
return IN_SET(abs(r),
ENOBUFS, /* Field or decompressed field too large */
E2BIG, /* Field too large for pointer width */
EPROTONOSUPPORT); /* Unsupported compression */
}

27
src/journal/sd-journal.c
View File

@ -2462,6 +2462,19 @@ _public_ int sd_journal_enumerate_data(sd_journal *j, const void **data, size_t
return 1; return 1;
} }
_public_ int sd_journal_enumerate_available_data(sd_journal *j, const void **data, size_t *size) {
for (;;) {
int r;
r = sd_journal_enumerate_data(j, data, size);
if (r >= 0)
return r;
if (!JOURNAL_ERRNO_IS_UNAVAILABLE_FIELD(r))
return r;
j->current_field++; /* Try with the next field */
}
}
_public_ void sd_journal_restart_data(sd_journal *j) { _public_ void sd_journal_restart_data(sd_journal *j) {
if (!j) if (!j)
return; return;
@ -3002,6 +3015,20 @@ _public_ int sd_journal_enumerate_unique(sd_journal *j, const void **data, size_
} }
} }
_public_ int sd_journal_enumerate_available_unique(sd_journal *j, const void **data, size_t *size) {
for (;;) {
int r;
r = sd_journal_enumerate_unique(j, data, size);
if (r >= 0)
return r;
if (!JOURNAL_ERRNO_IS_UNAVAILABLE_FIELD(r))
return r;
/* Try with the next field. sd_journal_enumerate_unique() modifies state, so on the next try
* we will access the next field. */
}
}
_public_ void sd_journal_restart_unique(sd_journal *j) { _public_ void sd_journal_restart_unique(sd_journal *j) {
if (!j) if (!j)
return; return;

2
src/journal/test-compress.c
View File

@ -232,6 +232,8 @@ static void test_lz4_decompress_partial(void) {
int r; int r;
_cleanup_free_ char *huge = NULL; _cleanup_free_ char *huge = NULL;
log_debug("/* %s */", __func__);
assert_se(huge = malloc(HUGE_SIZE)); assert_se(huge = malloc(HUGE_SIZE));
memcpy(huge, "HUGE=", STRLEN("HUGE=")); memcpy(huge, "HUGE=", STRLEN("HUGE="));
memset(&huge[STRLEN("HUGE=")], 'x', HUGE_SIZE - STRLEN("HUGE=") - 1); memset(&huge[STRLEN("HUGE=")], 'x', HUGE_SIZE - STRLEN("HUGE=") - 1);

3
src/libsystemd/libsystemd.sym
View File

@ -717,4 +717,7 @@ global:
sd_path_lookup_strv; sd_path_lookup_strv;
sd_notify_barrier; sd_notify_barrier;
sd_journal_enumerate_available_data;
sd_journal_enumerate_available_unique;
} LIBSYSTEMD_245; } LIBSYSTEMD_245;

10
src/systemd/sd-journal.h
View File

@ -105,6 +105,7 @@ int sd_journal_get_data_threshold(sd_journal *j, size_t *sz);
int sd_journal_get_data(sd_journal *j, const char *field, const void **data, size_t *l); int sd_journal_get_data(sd_journal *j, const char *field, const void **data, size_t *l);
int sd_journal_enumerate_data(sd_journal *j, const void **data, size_t *l); int sd_journal_enumerate_data(sd_journal *j, const void **data, size_t *l);
int sd_journal_enumerate_available_data(sd_journal *j, const void **data, size_t *l);
void sd_journal_restart_data(sd_journal *j); void sd_journal_restart_data(sd_journal *j);
int sd_journal_add_match(sd_journal *j, const void *data, size_t size); int sd_journal_add_match(sd_journal *j, const void *data, size_t size);
@ -128,6 +129,7 @@ int sd_journal_get_usage(sd_journal *j, uint64_t *bytes);
int sd_journal_query_unique(sd_journal *j, const char *field); int sd_journal_query_unique(sd_journal *j, const char *field);
int sd_journal_enumerate_unique(sd_journal *j, const void **data, size_t *l); int sd_journal_enumerate_unique(sd_journal *j, const void **data, size_t *l);
int sd_journal_enumerate_available_unique(sd_journal *j, const void **data, size_t *l);
void sd_journal_restart_unique(sd_journal *j); void sd_journal_restart_unique(sd_journal *j);
int sd_journal_enumerate_fields(sd_journal *j, const char **field); int sd_journal_enumerate_fields(sd_journal *j, const char **field);
@ -156,13 +158,13 @@ int sd_journal_has_persistent_files(sd_journal *j);
if (sd_journal_seek_tail(j) < 0) { } \ if (sd_journal_seek_tail(j) < 0) { } \
else while (sd_journal_previous(j) > 0) else while (sd_journal_previous(j) > 0)
/* Iterate through the data fields of the current journal entry */ /* Iterate through all available data fields of the current journal entry */
#define SD_JOURNAL_FOREACH_DATA(j, data, l) \ #define SD_JOURNAL_FOREACH_DATA(j, data, l) \
for (sd_journal_restart_data(j); sd_journal_enumerate_data((j), &(data), &(l)) > 0; ) for (sd_journal_restart_data(j); sd_journal_enumerate_available_data((j), &(data), &(l)) > 0; )
/* Iterate through the all known values of a specific field */ /* Iterate through all available values of a specific field */
#define SD_JOURNAL_FOREACH_UNIQUE(j, data, l) \ #define SD_JOURNAL_FOREACH_UNIQUE(j, data, l) \
for (sd_journal_restart_unique(j); sd_journal_enumerate_unique((j), &(data), &(l)) > 0; ) for (sd_journal_restart_unique(j); sd_journal_enumerate_available_unique((j), &(data), &(l)) > 0; )
/* Iterate through all known field names */ /* Iterate through all known field names */
#define SD_JOURNAL_FOREACH_FIELD(j, field) \ #define SD_JOURNAL_FOREACH_FIELD(j, field) \