IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
To detect EBCDIC code pages, we used to switch the encoding twice and
had to be very careful not to decode data after the XML declaration
before the second switch. This relied on a hard-coded expected size of
the XML declaration and was complicated and unreliable.
Now we convert the first 200 bytes to EBCDIC-US and parse the encoding
declaration manually.
Remove explicit integer casts as final operation
- in assignments
- when passing arguments
- when returning values
Remove casts
- to the same type
- from certain range-bound values
The main motivation is that these explicit casts don't change the result
of operations and only render UBSan's implicit-conversion checks
useless. Removing these casts allows UBSan to detect cases where
truncation or sign-changes occur unexpectedly.
Document some explicit casts as truncating and add a few missing ones.
Private functions were previously declared
- in header files in the root directory
- in public headers guarded with IN_LIBXML
- in libxml.h
- redundantly in source files that used them.
Consolidate all private header files in include/private.
* buf.c:
(xmlBufAvail):
- Return the number of bytes available in the buffer, but do not
include a byte for the NUL terminator so that it is reserved.
* encoding.c:
(xmlCharEncFirstLineInput):
(xmlCharEncInput):
(xmlCharEncOutput):
* xmlIO.c:
(xmlOutputBufferWriteEscape):
- Remove code that subtracts 1 from the return value of
xmlBufAvail(). It was implemented inconsistently anyway.
Similar to 8f5710379, mark more static data structures with
`const` keyword.
Also fix placement of `const` in encoding.c.
Original patch by Sarah Wilkin.
These functions shouldn't be part of the public API. Most init
functions are only thread-safe when called from xmlInitParser. Global
variables should only be cleaned up by calling xmlCleanupParser.
Fix a bug in xmlCharEncOutput return value which will cause
xmlNodeDumpOutput to drop characters randomly.
xmlCharEncOutput returns zero if the length of the input buffer is
zero but ignores the fact that it may already encoded the input buffer
and the input's length is zero due to the fact that xmlEncOutputChunk
returned -2 errors and underlying code tries to fix the error by
encoding the input.
xmlCharEncOutput is collecting the number of bytes written to the
output buffer but is returning zero instead of the total number of
bytes in this situation. This commit will fix this issue by returning
the total number of bytes instead. So the xmlNodeDumpOutput will also
continue writing and will not stop due to the fact that it mistakenly
thinks the output buffer is not changed in that iteration.
Fixes#314
This makes the logic in UTF16BEToUTF8() match UTF16LEToUTF8().
* encoding.c:
(UTF16LEToUTF8):
- Fix comment to describe what the code does.
(UTF16BEToUTF8):
- Fix undefined behavior which was applied to UTF16LEToUTF8() in
2f9382033e.
- Add bounds check to while() loop which was applied to
UTF16LEToUTF8() in be803967db.
- Do not return -2 when (in >= inend) to fix the bug. This was
applied to UTF16LEToUTF8() in 496a1cf592.
- Inline (<< 8) statements to match UTF16LEToUTF8().
Add the following tests and results:
test/text-4-byte-UTF-16-BE-offset.xml
test/text-4-byte-UTF-16-BE.xml
test/text-4-byte-UTF-16-LE-offset.xml
test/text-4-byte-UTF-16-LE.xml
Under certain circumstances, the HTML parser would try to guess and
switch input encodings multiple times, leading to slow processing of
documents with encoding errors. The repeated scanning of the input
buffer when guessing encodings could even lead to quadratic behavior.
The code htmlCurrentChar probably assumed that if there's an encoding
handler, it is guaranteed to produce valid UTF-8. This holds true in
general, but if the detected encoding was "UTF-8", the UTF8ToUTF8
encoding handler simply invoked memcpy without checking for invalid
UTF-8. This still must be fixed, preferably by not using this handler
at all.
Also leave a note that switching encodings twice seems impossible to
implement correctly. Add a check when handling UTF-8 encoding errors
in htmlCurrentChar to avoid this situation, even if encoders produce
invalid UTF-8.
Found by OSS-Fuzz.
The return type of xmlRegisterCharEncodingHandler() is void. The invoker
cannot determine whether xmlRegisterCharEncodingHandler() is executed
successfully. when nbCharEncodingHandler >= MAX_ENCODING_HANDLERS, the
"handler" is not added to the array "handlers". As a result, the memory
of "handler" cannot be managed and released: memory leakage.
so add "xmlfree(handler)" to fix memory leakage on the failure branch of
xmlRegisterCharEncodingHandler().
Reported-by: wuqing <wuqing30@huawei.com>
Signed-off-by: Xiaoming Ni <nixiaoming@huawei.com>
Make xmlEncInputChunk and xmlEncOutputChunk return 0 on success and
never a positive value.
Make xmlCharEncFirstLineInt, xmlCharEncFirstLineInt and
xmlCharEncOutFunc return the number of bytes written.
Commit 407b393d introduced a regression caused by xmlCharEncOutput
returning 0 in case of success instead of the number of bytes written.
Always use its return value for nbchars in xmlOutputBufferWrite.
Fixes#166.
Closes: https://bugzilla.gnome.org/show_bug.cgi?id=793028
It seams this line was accidentally copied over from xmlCharEncOutFunc.
In xmlCharEncOutput output is a pointer so incrementing it by ret can
point it where it wasn't supposed to be pointing. Luckily the current
implementation doesn't dereference the pointer after advancing it.
Signed-off-by: Daniel Veillard <veillard@redhat.com>
By always setting flush=TRUE when doing multiple reads, ICU
will not correctly handle truncated utf8 chars across read
boundaries.
The fix is to set flush=TRUE only on final read, and to
provide a pivot buffer which is maintained by libxml
between calls to ucnv_convertEx.
If a character can't be represented in the output encoding, it is
converted to a character reference. This used to to replace the
character in the input stream by calling xmlBufAddHead or
xmlBufferAddHead. These functions shifted the entire input array
around, leading to quadratic performance when converting a run of
non-representable characters. This is most pronounced when dumping to
memory.
Output the charref directly instead.
Found with libFuzzer.
For https://bugzilla.gnome.org/show_bug.cgi?id=711149
In Function:
int xmlCharEncCloseFunc(xmlCharEncodingHandler *handler)
If the freed handler is any one of handlers[i] list, then it will make that
hanldlers[i] as dangling. This may lead to crash issues at places where
handlers is read.
https://bugzilla.gnome.org/show_bug.cgi?id=692915
the new set of converting functions tried to limit the encoding
conversion of the raw buffer to the consumption one to work in
a more progressive fashion. Unfortunately this was bad for
performances and led to errors on progressive parsing when
a very large chunk was close to the end of the document. Fix
the new internal function and switch back to the old way of
converting. Fix another bug in the process.
Various cleanups
* configure.in: force regeneration of APIs in my environment
* buf.c buf.h enc.h encoding.c include/libxml/tree.h
include/libxml/xmlerror.h save.h tree.c: various comment cleanups
pointed by apibuild
* doc/apibuild.py: added the 3 new internal headers in the excludes
* doc/libxml2-api.xml doc/libxml2-refs.xml: regenerated the API
* doc/symbols.xml: listing new entry points for 2.9.0
* doc/devhelp/*: regenerated
* encoding.c: adds xmlCharEncFirstLineInput, xmlCharEncInput and
xmlCharEncOutput
* enc.h: the functions are not made public but added to this new header
When a node is dumped with a new encoding, we may encounter characters
that are not supported in the new encoding. libxml2 handles this by
replacing the character with character references, but in some encodings
this can result in an infinite loop when the character references
themselves contain unsupported characters.
This fixes the infinite loop by undoing a character reference substitution
when it cannot be inserted, and returning an encoder error.
This bug was noticed when looking into an infinite loop bug report for
the Ruby Nokogiri project. The original bug report, "nokogiri process
hangs on call to inner_html" is here:
https://github.com/tenderlove/nokogiri/issues/400
* encoding.c parser.c parserInternals.c: when we autodetect an encoding
but it's actually not completely compatible with the one declared
great care must be taken to not convert more than just the first line.
Led to some refactoring, more private functions and a bit of cleanup.
* include/libxml/tree.h tree.c: make a new kind of buffer where
shrinking and adding in head can avoid reallocation or full
buffer memmoves
* encoding.c xmlIO.c: use the new kind of buffers for output
buffers
Daniel
svn path=/trunk/; revision=3787
* encoding.c: poblem with encoding detection for UTF-16 reported by
Ashwin and found by Bill
* test/valid/dtds/utf16b.ent test/valid/dtds/utf16l.ent
test/valid/UTF16Entity.xml result/valid/UTF16Entity.xml*: added
the example to the regression tests
Daniel
svn path=/trunk/; revision=3700
* config.h.in configure.in encoding.c: patch from Roumen Petrov
to detect if iconv() needs a const for the second parameter
Daniel
svn path=/trunk/; revision=3693
* encoding.c: Fixed typo in xmlCharEncFirstLine pointed out
by Mark Rowe (bug #440159)
* include/libxml/xmlversion.h.in: Added check for definition of
_POSIX_C_SOURCE to avoid warnings on Apple OS/X (patch from
Wendy Doyle and Mark Rowe, bug #346675)
* schematron.c, testapi.c, tree.c, xmlIO.c, xmlsave.c: minor
changes to fix compilation warnings - no change to logic.
svn path=/trunk/; revision=3618
* HTMLparser.c: remove a warning
* encoding.c: check with uppercase for AIX iconv() should fix#352644
* doc/examples/Makefile.am: partially handle one bug report
Daniel
* encoding.c parserInternals.c: avoid passing a char[] as snprintf
first argument.
* threads.c include/libxml/threads.h: implemented xmlIsThreadsEnabled()
based on Andrew W. Nosenko idea.
* doc/* elfgcchack.h: regenerated the API
Daniel
* encoding.c error.c include/libxml/xmlerror.h: finally converted
the encoding module to the common error reporting mechanism
* doc/* doc/html/libxml-xmlerror.html: rebuilt
Daniel
* HTMLparser.c SAX2.c encoding.c globals.c parser.c relaxng.c
runsuite.c runtest.c schematron.c testHTML.c testReader.c
testRegexp.c testSAX.c testThreads.c valid.c xinclude.c xmlIO.c
xmllint.c xmlmodule.c xmlschemas.c xpath.c xpointer.c: a lot of
small cleanups based on Linus' sparse check output.
Daniel
* doc/apibuild.py doc/elfgcchack.xsl: revamped the elfgcchack.h
format to cope with gcc4 change of aliasing allowed scopes, had
to add extra informations to doc/libxml2-api.xml to separate
the header from the c module source.
* *.c: updated all c library files to add a #define bottom_xxx
and reimport elfgcchack.h thereafter, and a bit of cleanups.
* doc//* testapi.c: regenerated when rebuilding the API
Daniel
* encoding.c: removed a static buffer in xmlByteConsumed(),
as pointed by Ben Maurer, fixes#170086
* xmlschemas.c: remove a potentially uninitialized pointer warning
Daniel
* encoding.c: fix the comment to describe the real return values
* pattern.c xpath.c include/libxml/pattern.h: lot of work on
the patterns, pluggin in the XPath default evaluation, but
disabled right now because it's not yet good enough for XSLT.
pattern.h streaming API are likely to be changed to handle
relative and absolute paths in the same expression.
Daniel
* gentest.py testapi.c: autogenerate a minimal NULL value sequence
for unknown pointer types
* HTMLparser.c SAX2.c chvalid.c encoding.c entities.c parser.c
parserInternals.c relaxng.c valid.c xmlIO.c xmlreader.c
xmlsave.c xmlschemas.c xmlschemastypes.c xmlstring.c xpath.c
xpointer.c: This uncovered an impressive amount of entry points
not checking for NULL pointers when they ought to, closing all
the open gaps.
Daniel
* encoding.c doc/examples/testWriter.c: Fixed bug #153937, making
sure the conversion functions return the number of byte written.
Had to fix one of the examples.
Daniel
* nanohttp.c, include/libxml/nanohttp.h: added the routine
xmlNanoHTTPContentLength to the external API (bug151968).
* parser.c: fixed unnecessary internal error message (bug152060);
also changed call to strncmp over to xmlStrncmp.
* encoding.c: fixed compilation warning (bug152307).
* tree.c: fixed segfault in xmlCopyPropList (bug152368); fixed
a couple of compilation warnings.
* HTMLtree.c, debugXML.c, xmlmemory.c: fixed a few compilation
warnings; no change to logic.
* SAX2.c, encoding.c, error.c, parser.c, tree.c, uri.c, xmlIO.c,
xmlreader.c, include/libxml/tree.h: many further little changes
for OOM problems. Now seems to be getting closer to "ok".
* testOOM.c: added code to intercept more errors, found more
problems with library. Changed method of flagging / counting
errors intercepted.
* parserInternals.c xmlIO.c encoding.c include/libxml/parser.h
include/libxml/xmlIO.h: added xmlByteConsumed() interface
* doc/*: updated the benchmark rebuilt the docs
* python/tests/Makefile.am python/tests/indexes.py: added a
specific regression test for xmlByteConsumed()
* include/libxml/encoding.h rngparser.c tree.c: small cleanups
Daniel
* encoding.c: applied patch supplied by Christophe Dubach
to fix problem with --with-minimum configuration
(bug 133773)
* nanoftp.c: fixed potential buffer overflow problem,
similar to fix just applied to nanohttp.c.
* encoding.c, parser.c, xmlstring.c, Makefile.am,
include/libxml/Makefile.am, include/libxml/catalog.c,
include/libxml/chvalid.h, include/libxml/encoding.h,
include/libxml/parser.h, include/libxml/relaxng.h,
include/libxml/tree.h, include/libxml/xmlwriter.h,
include/libxml/xmlstring.h:
moved string and UTF8 routines out of parser.c and encoding.c
into a new module xmlstring.c with include file
include/libxml/xmlstring.h mostly using patches from Reid
Spencer. Since xmlChar now defined in xmlstring.h, several
include files needed to have a #include added for safety.
* doc/apibuild.py: added some additional sorting for various
references displayed in the APIxxx.html files. Rebuilt the
docs, and also added new file for xmlstring module.
* configure.in: small addition to help my testing; no effect on
normal usage.
* doc/search.php: added $_GET[query] so that persistent globals
can be disabled (for recent versions of PHP)
* encoding.c, include/libxml/encoding.h: Enhanced the handling of UTF-16,
UTF-16LE and UTF-16BE encodings. Now UTF-16 output is handled internally
by default, with proper BOM and UTF-16LE encoding. Native UTF-16LE and
UTF-16BE encoding will not generate a BOM on output, and will be
automatically recognized on input.
* test/utf16lebom.xml, test/utf16bebom.xml, result/utf16?ebom*: added
regression tests for above.
* Makefile.am: cleanup
* encoding.c: fix a funny typo
* error.c xmlschemas.c xmlschemastypes.c include/libxml/xmlerror.h:
converted the Schemas code to the new error handling. PITA,
still need to check output from regression tests.
Daniel
* encoding.c, threads.c, include/libxml/HTMLparser.h,
doc/libxml2-api.xml: Minor changes to comments, etc. for
improving documentation generation
* doc/Makefile.am: further adjustment to auto-generation of
win32/libxml2.def.src
* encoding.c: applied UTF-16 encoding handling patch provided by
Mark Itzcovitz
* encoding.c parser.c: more cleanup and fixes for UTF-16 when
not having iconv support.
Daniel
* encoding.c: applying patch from Peter Jacobi to added
ISO-8859-x encoding support when iconv is not available
* configure.in include/libxml/xmlversion.h.in
include/libxml/xmlwin32version.h.in: added the glue needed
at the configure level and made it the default for Windows
Daniel
* HTMLparser.c: fixed problem with comments reported by Nick Kew
* encoding.c: added routines xmlUTF8Size and xmlUTF8Charcmp for
some future cleanup of UTF8 handling
* DOCBparser.c HTMLparser.c c14n.c catalog.c encoding.c globals.c
nanohttp.c parser.c parserInternals.c relaxng.c tree.c uri.c
xmlmemory.c xmlreader.c xmlregexp.c xpath.c xpointer.c
include/libxml/globals.h include/libxml/xmlmemory.h: added
xmlMallocAtomic() to be used when allocating blocks which
do not contains pointers, add xmlGcMemSetup() and xmlGcMemGet()
to allow registering the full set of functions needed by
a garbage collecting allocator like libgc, ref #109944
Daniel
