1
0
mirror of https://gitlab.gnome.org/GNOME/libxml2.git synced 2025-01-15 23:24:06 +03:00

4614 Commits

Author SHA1 Message Date
Patrick Monnerat
d1f9062486 small changes for OS/400
OS/400 version V5R3 is not supported by IBM anymore.
In addition, the iSeries system I have here for development has been changed
and the new system is not able to compile for an OS version lower than V6R1.

Thus I made some updates to the libxml2 os400 scripts accordingly:

- Oldest supported OS version is now V6R1.
- Adjust ILE/RPG wrappers comments.
- Update copyright year range.
- Do not log compiler informational messages.
2017-07-27 11:20:02 +08:00
Nick Wellnhofer
899a5d9f0e Detect infinite recursion in parameter entities
When expanding a parameter entity in a DTD, infinite recursion could
lead to an infinite loop or memory exhaustion.

Thanks to Wei Lei for the first of many reports.

Fixes bug 759579.
2017-07-25 15:21:12 +02:00
David Kilzer
fb56f80eef Heap-buffer-overflow read of size 1 in xmlFAParsePosCharGroup
Credit to OSS-Fuzz.

Add a check to xmlFAParseCharRange() for the end of the buffer
to prevent reading past the end of it.

This fixes Bug 784017.
2017-07-04 18:51:29 +02:00
Nick Wellnhofer
8a0c66986e Fix NULL pointer deref in xmlFAParseCharClassEsc
Found with libFuzzer.
2017-07-04 18:51:29 +02:00
Nick Wellnhofer
52ceced6e7 Fix infinite loops with push parser in recovery mode
Make sure that the input pointer advances in case of errors. Otherwise,
the push parser can loop infinitely.

Found with libFuzzer.
2017-07-04 18:51:29 +02:00
Nick Wellnhofer
f435365b59 Send xmllint usage error to stderr
Fixes bug 781459.
2017-07-04 18:51:28 +02:00
Nick Wellnhofer
3eef3f39a6 Fix NULL deref in xmlParseExternalEntityPrivate
If called from xmlParseExternalEntity, oldctxt is NULL which leads to
a NULL deref if an error occurs. This only affects external code that
calls xmlParseExternalEntity.

Patch from David Kilzer with minor changes.

Fixes bug 780159.
2017-06-20 16:13:57 +02:00
Nick Wellnhofer
872fea9485 Get rid of "blanks wrapper" for parameter entities
Now that replacement of parameter entities goes exclusively through
xmlSkipBlankChars, we can account for the surrounding space characters
there and remove the "blanks wrapper" hack.
2017-06-20 13:19:47 +02:00
Nick Wellnhofer
d9e43c7db5 Make sure not to call IS_BLANK_CH when parsing the DTD
This is required to get rid of the "blanks wrapper" hack. Checking the
return value of xmlSkipBlankChars is more efficient, too.
2017-06-20 13:19:47 +02:00
Nick Wellnhofer
453dff1e3b Remove unnecessary calls to xmlPopInput
It's enough if xmlPopInput is called from xmlSkipBlankChars. Since the
replacement text of a parameter entity is surrounded with space
characters, that's the only place where the replacement can end in a
well-formed document.

This is also required to get rid of the "blanks wrapper" hack.
2017-06-20 13:19:47 +02:00
Nick Wellnhofer
aa267cd127 Simplify handling of parameter entity references
There are only two places where parameter entity references must be
handled. For the internal subset in xmlParseInternalSubset. For the
external subset or content from other external PEs in xmlSkipBlankChars.

Make sure that xmlSkipBlankChars skips over sequences of PEs and
whitespace. Rely on xmlSkipBlankChars instead of calling
xmlParsePEReference directly when in the external subset or a
conditional section.

xmlParserHandlePEReference is unused now.
2017-06-20 13:19:47 +02:00
Nick Wellnhofer
24246c7626 Fix xmlHaltParser
Pop all extra input streams before resetting the input. Otherwise,
a call to xmlPopInput could make input available again.

Also set input->end to input->cur.

Changes the test output for some error tests. Unfortunately, some
fuzzed test cases were added to the test suite without manual cleanup.
This makes it almost impossible to review the impact of later changes
on the test output.
2017-06-20 13:15:43 +02:00
Nick Wellnhofer
e5107772ff Fix pathological performance when outputting charrefs
If a character can't be represented in the output encoding, it is
converted to a character reference. This used to to replace the
character in the input stream by calling xmlBufAddHead or
xmlBufferAddHead. These functions shifted the entire input array
around, leading to quadratic performance when converting a run of
non-representable characters. This is most pronounced when dumping to
memory.

Output the charref directly instead.

Found with libFuzzer.
2017-06-19 16:06:21 +02:00
Nick Wellnhofer
c9ccbd6a6d Deduplicate code in encoding.c
Introduce static functions xmlEncInputChunk and xmlEncOutputChunk
that handle the internal/iconv/ICU branching.
2017-06-19 16:06:21 +02:00
Nick Wellnhofer
5a0ae66d72 Documentation fixes
Fixes bug 347465, bug 599433, bug 624550, bug 698253.
2017-06-18 17:58:38 +02:00
Nick Wellnhofer
8bbe4508ef Spelling and grammar fixes
Fixes bug 743172, bug 743489, bug 769632, bug 782400 and a few other
misspellings.
2017-06-17 16:34:23 +02:00
Nick Wellnhofer
576912fa04 Make HTML parser functions take const pointers
The 'cur' parameter of htmlParseDoc and htmlSAXParseDoc should be
'const xmlChar *'.

Fixes bug 770650.
2017-06-17 15:59:13 +02:00
Nick Wellnhofer
988a5a3b6b Build test programs only when needed
Add test programs to 'check_PROGRAMS' instead of 'noinst_PROGRAMS'.

Fixes bug 760457.
2017-06-17 15:44:50 +02:00
Nick Wellnhofer
b9b4b6b513 Fix doc/examples/index.py
In my previous commit that silenced some test output I didn't realize
that doc/examples/Makefile.am was autogenerated.

Also make index.py output deterministic by sorting the glob results.
2017-06-17 15:43:29 +02:00
Nick Wellnhofer
1f09aea26d Fix compiler warnings in threads.c
Use '#pragma weak' to declare weak functions.
2017-06-17 15:05:34 +02:00
Nick Wellnhofer
629e47e7a2 Fix empty-body warning in nanohttp.c 2017-06-17 14:56:53 +02:00
Nick Wellnhofer
1a595cd173 Fix cast-align warnings
- Suppress warnings in xmlmemory.c by casting to 'void *'.
- Remove unneeded cast in xmlschemas.c that caused a macro precedence
  error.
- Add dummy fields to short structs in xmlschemas.c. This increases the
  size of the structs, but I can't see a better solution without using
  C11's _Alignof operator.

There are still a couple of cast-align warnings in encoding.c. These
are legitimate portability issues that can't be fixed without reworking
the conversion functions.
2017-06-17 14:17:27 +02:00
Nick Wellnhofer
81c01ee9ff Fix unused-parameter warnings 2017-06-17 14:12:53 +02:00
Nick Wellnhofer
1ce1f7852c Fix invalid-source-encoding warnings in testWriter.c
Use hex escapes instead of binary data in source file.
2017-06-17 14:11:03 +02:00
Nick Wellnhofer
5f440d8cad Rework entity boundary checks
Make sure to finish all entities in the internal subset. Nevertheless,
readd a sanity check in xmlParseStartTag2 that was lost in my previous
commit. Also add a sanity check in xmlPopInput. Popping an input
unexpectedly was the source of many recent memory bugs. The check
doesn't mitigate such issues but helps with diagnosis.

Always base entity boundary checks on the input ID, not the input
pointer. The pointer could have been reallocated to the old address.

Always throw a well-formedness error if a boundary check fails. In a
few places, a validity error was thrown.

Fix a few error codes and improve indentation.
2017-06-17 13:25:53 +02:00
Nick Wellnhofer
46dc989080 Don't switch encoding for internal parameter entities
This is only needed for external entities. Trying to switch the encoding
for internal entities could also cause a memory leak in recovery mode.
2017-06-17 13:23:40 +02:00
Nick Wellnhofer
03904159f8 Merge duplicate code paths handling PE references
xmlParsePEReference is essentially a subset of
xmlParserHandlePEReference, so make xmlParserHandlePEReference call
xmlParsePEReference. The code paths in these functions differed
slighty, but the code from xmlParserHandlePEReference seems more solid
and tested.
2017-06-17 13:22:37 +02:00
Nick Wellnhofer
dbaab1f369 Test SAX2 callbacks with entity substitution
This detects regressions like bug 760367.
2017-06-16 21:38:57 +02:00
David Kilzer
3f0627a1ee Fix duplicate SAX callbacks for entity content
Reset 'was_checked' to prevent entity from being parsed twice and SAX
callbacks being invoked twice if XML_PARSE_NOENT was set.

This regressed in version 2.9.3 and caused problems with WebKit.

Fixes bug 760367.
2017-06-16 21:30:42 +02:00
Nick Wellnhofer
01a4b81a03 Support catalog and threads tests under --without-sax1 2017-06-16 21:27:47 +02:00
Nick Wellnhofer
3939178e4c Treat URIs with scheme as absolute in C14N
Fixes bug 783656.
2017-06-12 20:19:02 +02:00
Nick Wellnhofer
67f9f9d6c8 Misc fixes for 'make tests'
- Silence test output.
- Clean up after doc/examples tests.
- Adjust expected output for script tests.
- Add missing results for relaxng/pattern3

There are still two test failures I can't comment on:

- regexp/bug316338
- schemas/any4_0
2017-06-12 19:46:56 +02:00
Nick Wellnhofer
0b2d5c48e3 Initialize keepBlanks in HTML parser
This caused failures in the HTML push tests but the fix required to
change the expected output of the HTML SAX tests.
2017-06-12 19:11:54 +02:00
David Kilzer
85c112a082 Add test cases for bug 758518
test/HTML/758518-entity.html exposed a bug in pushParseTest() in
runtest.c which assumed that an input file was at least 4 bytes long.
That test case is only 3 bytes, so we now take the minimum of 4 bytes
or the length of the test input.  We also now use 'chunkSize' in place
of the hard-coded value '1024' later in the function.
2017-06-12 18:26:11 +02:00
Nick Wellnhofer
ed48d65b4d Fix copy-paste errors in error messages
Thanks to David Kilzer for the review.
2017-06-11 17:12:21 +02:00
Nick Wellnhofer
9a366a3748 Fix compiler warning in htmlParseElementInternal 2017-06-11 12:43:59 +02:00
Nick Wellnhofer
f39e3be0dd Fix sanity check in htmlParseNameComplex
- (cur - len) can overflow.
- Throw an internal error.

Fixes bug 780077.
2017-06-11 12:43:37 +02:00
Nick Wellnhofer
79c8a6b105 Print error messages for truncated UTF-8 sequences
Before, truncated UTF-8 sequences at the end of a file were treated as
EOF. Create an error message containing the offending bytes.

xmlStringCurrentChar would also print characters from the input stream,
not the string it's working on.
2017-06-10 18:11:58 +02:00
Nick Wellnhofer
fb2f518cc6 Fix potential infinite loop in xmlStringLenDecodeEntities
Make sure that xmlParseStringPEReference advances the "str" pointer
even if the parser was stopped. Otherwise xmlStringLenDecodeEntities
can loop infinitely.
2017-06-10 18:00:55 +02:00
Nick Wellnhofer
4ba8cc856b Remove useless check in xmlParseAttributeListDecl
Since we already successfully parsed the attribute name and other
items, it is guaranteed that we made progress in the input stream.

Comparing the input pointer to a previous value also looks fragile to
me. What if the input buffer was reallocated and the new "cur" pointer
happens to be the same as the old one? There are a couple of similar
checks which also take "consumed" into account. This seems to be safer
but I'm not convinced that it couldn't lead to false alarms in rare
situations.
2017-06-10 17:52:59 +02:00
Nick Wellnhofer
f9e7997e80 Reset parser input pointers on encoding failure
Call xmlBufResetInput before bailing out if switching the encoding
fails. Otherwise, the input pointers are left in an invalid state.
This would typically lead to an internal error in xmlGROW but could also
cause other unforeseen problems.
2017-06-10 17:50:27 +02:00
Nick Wellnhofer
bedbef8065 Fix memory leak in xmlParseEntityDecl error path
When parsing the entity value, it can happen that an external entity
with an unsupported encoding is loaded and the parser is stopped. This
would lead to a memory leak.

A custom SAX callback could also stop the parser.

Found with libFuzzer and ASan.
2017-06-10 17:42:52 +02:00
Nick Wellnhofer
94f6ce838c Allow zero sized memory input buffers
Useful for a fuzz target I'm working on.
2017-06-10 17:42:04 +02:00
Nick Wellnhofer
91e5496780 Fix xmlBuildRelativeURI for URIs starting with './'
If the relative URI started with './', the 'pos' index was increased
which also affected indexing into the base path. Aside from producing
wrong results, this could also lead to a heap overread of the base
path buffer. The data read from beyond the buffer was only compared
to some char values, so this is mostly harmless.

Inside libxml2, xmlBuildRelativeURI is only called from xinclude.c.

Found with libFuzzer and ASan.
2017-06-10 17:41:42 +02:00
Nick Wellnhofer
45ce1ee399 Add TODO comment in xmlSwitchEncoding
It would be nice if we could recover from unsupported encodings in
external entities.
2017-06-10 17:32:44 +02:00
Nick Wellnhofer
92b9e8c8b3 Fix type confusion in xmlValidateOneNamespace
Comment out code that casts xmlNsPtr to xmlAttrPtr. ID types on
namespace declarations make no practical sense anyway.

Fixes bug 780228.

Found with libFuzzer and ASan.
2017-06-07 20:26:32 +02:00
Nick Wellnhofer
8c82f5deeb Fix memory leak in xmlStringLenGetNodeList
Avoid expanding the entity recursively. Use the same prevention
mechanism as in xmlStringGetNodeList.

xmlStringGetNodeList on the other hand wasn't fixing up the 'last'
pointer.

I think the memory leak can only be triggered in recovery mode.

Found with libFuzzer and ASan.
2017-06-07 19:58:26 +02:00
Daniel Veillard
94691dc884 Fix NULL pointer deref in xmlDumpElementContent
Can only be triggered in recovery mode.

Fixes bug 758422 (CVE-2017-5969).
2017-06-07 19:58:26 +02:00
Nick Wellnhofer
362b322934 Fix memory leak in xmlBufAttrSerializeTxtContent
The serializer sets doc->encoding to a temporary value and restores
the original value when it's done. This overwrites the encoding value
set in xmlBufAttrSerializeTxtContent, causing a memory leak.

Don't mess with doc->encoding if invalid UTF-8 is encountered.

Found with libFuzzer and ASan.
2017-06-07 19:58:20 +02:00
Nick Wellnhofer
0db8dc9ddc Stop parser on unsupported encodings
Otherwise, the push parser can loop infinitely in recover mode.

Found with libFuzzer.
2017-06-07 19:30:56 +02:00