IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
The old approach introduced a regression, see issue #312 and the
previous commit. Disable code that tries to recover from invalid start
tags. This only affects "recovery" mode.
Add a comment outlining a better fix in accordance with the HTML5 spec.
Revert part of commit 173a0830 that changed behavior when parsing
malformed start tags with the push parser. This reintroduces quadratic
behavior in recovery mode which will be worked around in the next
commit.
Fixes#312.
Fix regression introduced with b25acce8. Some users like libxslt may
call the HTML output functions on documents with uppercase tag names,
so we must keep case-insensitive string comparison.
Fixes#248.
Switch to binary search. This is the first time bsearch is used in the
libxml2 code base. But it's a standard library function since C89 and
should be portable.
Under certain circumstances, the HTML parser would try to guess and
switch input encodings multiple times, leading to slow processing of
documents with encoding errors. The repeated scanning of the input
buffer when guessing encodings could even lead to quadratic behavior.
The code htmlCurrentChar probably assumed that if there's an encoding
handler, it is guaranteed to produce valid UTF-8. This holds true in
general, but if the detected encoding was "UTF-8", the UTF8ToUTF8
encoding handler simply invoked memcpy without checking for invalid
UTF-8. This still must be fixed, preferably by not using this handler
at all.
Also leave a note that switching encodings twice seems impossible to
implement correctly. Add a check when handling UTF-8 encoding errors
in htmlCurrentChar to avoid this situation, even if encoders produce
invalid UTF-8.
Found by OSS-Fuzz.
Null bytes in the input stream do not necessarily signal an EOF
condition. Check the stream pointers for EOF to avoid quadratic
rescanning of input data.
Note that the CUR_CHAR macro used in functions like htmlParseCharData
calls htmlCurrentChar which translates null bytes.
Found by OSS-Fuzz.
Fix a few remaining cases where the HTML push parser would scan more
content during lookahead than being parsed later.
Make sure that htmlParseDocTypeDecl consumes all content up to the
final '>' in case of errors. The old comment said "We shouldn't try to
resynchronize", but ignoring invalid content is also what the HTML5
spec mandates.
Likewise, make htmlParseEndTag skip to the final '>' in invalid end
tags even if not in recovery mode. This is probably the most visible
change in practice and leads to different output for some tests but is
also more in line with HTML5.
Make sure that htmlParsePI and htmlParseComment don't abort if invalid
characters are encountered but log an error and ignore the character.
Change some other end-of-buffer checks to test for a zero byte instead
of relying on IS_CHAR.
Fix usage of IS_CHAR macro in htmlParseScript.
Make sure that htmlParseStartTag doesn't terminate on characters for
which IS_CHAR_CH is false like control chars.
In htmlParseTryOrFinish, only switch to START_TAG if the next character
starts a valid name. Otherwise, htmlParseStartTag might return without
consuming all characters up to the final '>'.
Found by OSS-Fuzz.
The HTML push parser would look ahead for characters in "; >/" to
terminate an entity reference but actual parsing could stop earlier,
potentially resulting in quadratic runtime.
Parse char data and references alternately in htmlParseTryOrFinish
and only look ahead once for a terminating '<' character.
Found by OSS-Fuzz.
The parsing rules when looking for terminating chars or sequences in
the push parser differed from the actual parsing code. This could
result in the lookahead to overshoot and data being rescanned,
potentially leading to quadratic runtime.
Comments must never be handled during lookahead. Attribute values must
only be skipped for start tags and doctype declarations, not for end
tags, comments, PIs and script content.
The general assumption is that htmlCurrentChar only returns 0 if the
end of the input buffer is reached. The UTF-8 path already logged an
error if a zero byte U+0000 was found and returned a space character
instead. Make the ASCII code path do the same.
htmlParseTryOrFinish skips zero bytes at the beginning of a buffer, so
even if 0 was returned from htmlCurrentChar, the push parser would make
progress. But rescanning the input could cause performance problems.
The pull parser would abort parsing and now handles zero bytes in ASCII
mode the same way as the push parser or as in UTF-8 mode.
It would be better to return the replacement character U+FFFD instead,
but some of the client code assumes that the UTF-8 length of input and
output matches.
Reject sequences starting with a continuation byte as well as overlong
sequences like the XML parser.
Also fixes an infinite loop in connection with previous commit 50078922
since htmlCurrentChar would return 0 even if not at the end of the
buffer.
Found by OSS-Fuzz.
If htmlParseScript returns upon hitting an invalid character,
htmlParseLookupSequence will be called again with checkIndex reset to
zero, potentially resulting in quadratic runtime. Make sure that
htmlParseScript consumes all input in one go and simply skips over
invalid characters similar to htmlParseCharDataInternal.
Found by OSS-Fuzz.
Make sure that checkIndex is set when returning without match from
inside a comment. Also track parser state in htmlParseLookupChars.
Found by OSS-Fuzz.
Commit eeb99329 removed an important optimization avoiding quadratic
runtime when repeatedly scanning the input buffer for terminating
characters in the HTML push parser. The related bug is
https://bugzilla.gnome.org/show_bug.cgi?id=444994
Make sure that ctxt->checkIndex is always written and store additional
parser state in ctxt->inSubset which is unused in the HTML parser.
Found by OSS-Fuzz.
If charset conversion fails, reset the input pointers before reporting
the error and bailing out. Otherwise, the input pointers are left in an
invalid state which could lead to use-after-free and other memory
errors.
Similar to f9e7997e. Found by OSS-Fuzz.
Cast to signed type before subtraction to avoid unsigned integer
overflow. Also use ptrdiff_t to avoid potential integer truncation.
Found with libFuzzer and UBSan.
Call xmlBufResetInput before bailing out if switching the encoding
fails. Otherwise, the input pointers could be left in an invalid state.
Similar to commit f9e7997e80 for the
XML parser.
Thanks to Yunho Kim for the report.
Closes: #27
If libz or liblzma are detected with pkg-config, AC_CHECK_HEADERS must
not be run because the correct CPPFLAGS aren't set. It is actually not
required have separate checks for LIBXML_ZLIB_ENABLED and HAVE_ZLIB_H.
Only check for LIBXML_ZLIB_ENABLED and remove HAVE_ZLIB_H macro.
Fixes bug 764657, bug 787041.
Make sure that all parameters and return values of hash callback
functions exactly match the callback function type. This is required
to pass clang's Control Flow Integrity checks and to allow compilation
to asm.js with Emscripten.
Fixes bug 784861.
For https://bugzilla.gnome.org/show_bug.cgi?id=760263
* HTMLparser.c: Add BASE_PTR convenience macro.
(htmlParseSystemLiteral): Store length and start position instead
of a pointer while iterating through the public identifier since
the underlying buffer may change, resulting in a stale pointer
being used.
(htmlParsePubidLiteral): Ditto.
From https://bugzilla.gnome.org/show_bug.cgi?id=758518
Happens when a file has a name getting parsed, but no valid encoding
set, so libxml has to guess what the encoding is. This patch detects
when the buffer location changes, and if it does, restarts the parsing
of the name.
This slightly change a couple of regression tests output
Reviewed by David Kilzer.
* HTMLparser.c:
(htmlParseName): Add bounds check.
(htmlParseNameComplex): Ditto.
* result/HTML/758605.html: Added.
* result/HTML/758605.html.err: Added.
* result/HTML/758605.html.sax: Added.
* runtest.c:
(pushParseTest): The input for the new test case was so small
(4 bytes) that htmlParseChunk() was never called after
htmlCreatePushParserCtxt(), thereby creating a false positive
test failure. Fixed by using a do-while loop so we always call
htmlParseChunk() at least once.
* test/HTML/758605.html: Added.
For https://bugzilla.gnome.org/show_bug.cgi?id=756372
Error in the code pointing to the codepoint in the stack for the
current char value instead of the pointer in the input that the SAX
callback expects
Reported and fixed by Hugh Davenport
As pointed by Christian Schoenebeck <schoenebeck@crudebyte.com>
on the list and based on some of his early patches, this preserve
content when unescaped opening angle brackets are not escaped in
textual content like:
<p> a < b </p>
<p> a <0 </p>
<p> a <=0 </p>
while still reporting the error.
For https://bugzilla.gnome.org/show_bug.cgi?id=719515
fixes htmlParseTryOrFinish to interpret HTML_PARSE_NODEFDTD,
and updates xmllint to actually pass --nodefdtd to the push
version of the HTML parser
https://bugzilla.gnome.org/show_bug.cgi?id=692915
the new set of converting functions tried to limit the encoding
conversion of the raw buffer to the consumption one to work in
a more progressive fashion. Unfortunately this was bad for
performances and led to errors on progressive parsing when
a very large chunk was close to the end of the document. Fix
the new internal function and switch back to the old way of
converting. Fix another bug in the process.
https://bugzilla.gnome.org/show_bug.cgi?id=690202
Buffer overflow errors originating from xmlBufGetInputBase in 2.9.0
The pointers from the context input were not properly reset after
that call which can do reallocations.
For https://bugzilla.gnome.org/show_bug.cgi?id=681822
Regardless if the option HTML_PARSE_NOBLANKS is set or not, blank nodes
are removed from a HTML document, for example:
<html>
<head>
<title>This is a test.</title>
</head>
<body>
<p>This is a test.</p>
</body>
</html>
is read as:
<html><head><title>This is a test.</title></head><body>
<p>This is a test.</p>
</body></html>
This changes the default behaviour but the old behaviour is available
as expected when using the parser flag HTML_PARSE_NOBLANKS
Based on original patch from Igor Ignatyuk <igor_ignatiouk@hotmail.com>
* HTMLparser.c: change various places in the parser where ignorable_space
SAX callback was called without checking for the parser flag preference
* xmllint.c: make sure we use the new flag even for HTML parsing
* result/HTML/*: this modifies the output of a number of tests
In HTML email it's common to find arbitrary fragments of HTML, the one
that triggered this change was of the form:
<meta><font></font><div>...
Before this change the <font> tag was part of the implicit <head> that
gets created for the <meta> tag, after this change, it is part of the
<body>, which more closely matches the behaviour of modern HTML
implementations.
When calling xmlParserInputBufferPush, the buffer may be reallocated
and at the input level the pointers for base, cur and end need to
be reevaluated.
* buf.c buf.h: add two new functions, one to get the base from the
input of the buffer, and another one to reset the pointers based
on the cur and base inded
* HTMLparser.c parser.c: cleanup to use the new helper functions
as well as making sure size_t is used for the indexes computations
This was scattered in a number of modules, xmlParserInputPtr
have usually their base, cur and end pointer set from an
xmlBuf used as input.
* buf.c buf.h: add a new function implementing this setup
* parser.c HTMLparser.c catalog.c parserInternals.c xmlreader.c
use the new function instead of digging into the buffer in
all those modules
For https://bugzilla.gnome.org/show_bug.cgi?id=615785
When the <noscript> is found, <head> is closed and a <body> element is created.
The real <body id="xxx"> gets skipped over, so I can't see any of the
body's attributes.
Just don't close <head> when encountering a <noscript>
Add a regression test too
For https://bugzilla.gnome.org/show_bug.cgi?id=642916
I just noticed that the HTML_PARSE_NOIMPLIED flag that you can pass to the
HTML-Parser methods doesn't do anything. Its intended purpose is to stop the
HTML-parser from forcibly adding a pair of html/body tags if the stream does
not contain any.
This is highly useful when you don't need this level of strictness.
Unfortunately, specifying it doesn't work, because the option is not
copied into the parsing context.
For https://bugzilla.gnome.org/show_bug.cgi?id=643949
In case of error on an IO creation input the given context
is terminated with the given close function, except if the
error happened in xmlParserInputBufferCreateIO. This can
lead to a resource leak which is fixed by this patch.
For https://bugzilla.gnome.org/show_bug.cgi?id=655218http://www.w3.org/TR/2011/WD-html5-20110525/semantics.html#the-meta-element
"""
The charset attribute specifies the character encoding used by the document.
This is a character encoding declaration. If the attribute is present in an XML
document, its value must be an ASCII case-insensitive match for the string
"UTF-8" (and the document is therefore forced to use UTF-8 as its
encoding).
"""
However, while <meta http-equiv="Content-Type" content="text/html;
charset=utf8"> works, <meta charset="utf8"> does not.
While libxml2 HTML parser is not tuned for HTML5, this is a simple
addition
Also added a testcase
The data in node_seq in xmlParserCtxt was not updated properly
when parsing HTML. This patch fixes the accounting for both
pull and push mode of HTML parsing.
For both XML and HTML, the document can provide an encoding
either in XMLDecl in XML, or as a meta element in HTML head.
This adds options to ignore those encodings if the encoding
is known in advace for example if the content had been converted
before being passed to the parser.
* parser.c include/libxml/parser.h: add XML_PARSE_IGNORE_ENC option
for XML parsing
* include/libxml/HTMLparser.h HTMLparser.c: adds the
HTML_PARSE_IGNORE_ENC for HTML parsing
* HTMLtree.c: fix the handling of saving when an unknown encoding is
defined in meta document header
* xmllint.c: add a --noenc option to activate the new parser options
- include/libxml/HTMLparser.h: defines the new HTML parser option
HTML_PARSE_NODEFDTD
- HTMLparser.c: if option is set don't add a default DTD
- xmllint.c: add the corresponding --nodefdtd option in xmllint
* HTMLparser.c: don't default value of HTML boolean attributes in the
parser
* SAX2.c: move this to SAX2 tree building backend
* result/HTML/doc2.htm.sax result/HTML/doc3.htm.sax
result/HTML/wired.html.sax: this changes a few HTML SAX regression
tests
* HTMLparser.c: new htmlParseElementInternal non recursive, with
htmlParseContentInternal and new function to handle node info
and element end.
* include/libxml/parser.h: add new stack for element info in parser
context
* parserInternals.c: fee element info stack
xmlParseInNodeContext notices that the enclosing document is
an HTML document, so invoke the HTML parser for that fragment, and
the HTML parser finding a "<p>hello world!</p>" document automatically
augment it with defaulted <html> and <body>. This defaulting should
be turned off in the HTML parser for this to work, but there is no
such HTML parser option. There is an htmlOmittedDefaultValue global
variable that you could use, but really we should not rely on global
variable for processing options anymore, best is to add an
HTML_PARSE_NOIMPLIED.
* include/libxml/HTMLparser.h: add the HTML_PARSE_NOIMPLIED parser flag
* HTMLparser.c: do add implied element if HTML_PARSE_NOIMPLIED is set
* parser.c: add HTML_PARSE_NOIMPLIED to options for xmlParseInNodeContext
on HTML documents
* HTMLparser.c c14n.c debugXML.c entities.c nanohttp.c parser.c
testC14N.c uri.c xmlcatalog.c xmllint.c xmlregexp.c xpath.c:
fix unused variables, or unneeded increments as well as a couple
of space issues
* runtest.c: check for NULL before calling unlink()
* HTMLparser.c: fix htmlParseLookupSequence to not save ctxt->checkIndex
when the current buffer ends within an attribute value, as this
information would be missed in next pass.
* HTMLparser.c: check when we see an head or a body tag and avoid
autogenerating them
* include/libxml/parser.h: the values for ctxt->html change depending
on the head or body tags being seen
* HTMLparser.c: fix the problem with detection erroring absolutely, and
properly popping up the stack when in EOF, also passes XML_PARSE_HUGE
when decoding options.
* HTMLparser.c: if we hit an encoding error before parsing a potential
<meta> with the info look in the input buffer to see if we can find
it instead of forcing a blind switch to ISO-8859-1
* configure.in doc/* NEWS: preparing the release of 2.7.2
* dict.c: fix the Solaris portability issue
* parser.c: additional cleanup on #554660 fix
* test/ent13 result/ent13* result/noent/ent13*: added the
example in the regression test suite.
* HTMLparser.c: handle leading BOM in htmlParseElement()
Daniel
svn path=/trunk/; revision=3799
* HTMLparser.c: fix an HTML parsing error on large data sections
reported by Mike Day
* test/HTML/utf8bug.html result/HTML/utf8bug.html.err
result/HTML/utf8bug.html.sax result/HTML/utf8bug.html: add the
reproducer to the test suite
daniel
svn path=/trunk/; revision=3797
* doc/examples/reader3.c: patch from Robert Schwebel , allows to
compile the example if configured without output support fixes
#545582
* Makefile.am: add testrecurse to the make check tests
* HTMLparser.c: if the parser got a encoding argument it should be
used over what the meta specifies, patch fixing #536346
Daniel
svn path=/trunk/; revision=3785
* runxmlconf.c: more progresses against the official regression tests
* runsuite.c: small cleanup for non-leak reports
* include/libxml/tree.h: parsing flags and other properties are
now added to the document node, this is generally useful and
allow to make Name and NmToken validations based on the parser
flags, more specifically the 5th edition of XML or not
* HTMLparser.c tree.c: small side effects for the previous changes
* parser.c SAX2.c valid.c: the bulk of teh changes are here,
the parser and validation behaviour can be affected, parsing
flags need to be copied, lot of changes. Also fixing various
validation problems in the regression tests.
Daniel
svn path=/trunk/; revision=3762
* uri.c: applied patch from Ashwin fixing a number of realloc problems
* HTMLparser.c: improve handling for misplaced html/head/body
Daniel
svn path=/trunk/; revision=3740
* HTMLparser.c: apparently it's okay to forget the semicolumn after
entity refs in HTML, fixing char refs parsing accordingly based on
T. Manske patch, this should fix#517653
Daniel
svn path=/trunk/; revision=3726
* HTMLparser.c: patch from Arnold Hendriks improving parsing of
html within html bogus data, still not a complete fix though
Daniel
svn path=/trunk/; revision=3704
* HTMLparser.c: avoid stopping parsing when encountering
out of range characters in an HTML file, report and
continue processing instead, should fix#472696
Daniel
svn path=/trunk/; revision=3675
* HTMLparser.c: fix definition for <embed> to avoid error
when saving back, patch from Stefan Behnel fixing 495213
Daniel
svn path=/trunk/; revision=3671
* HTMLparser.c: fixed bug #381877, avoid reading over the end
of stream when generating an UTF-8 encoding error.
Daniel
svn path=/trunk/; revision=3627
* HTMLparser.c: change the way script/style are parsed to
not try to detect comments, reported by Mike Day
* result/HTML/doc3.*: affects the result of that test
Daniel
svn path=/trunk/; revision=3598
* HTMLparser.c: fixed memory access error on parsing of meta data
which had errors (bug #382206). Also cleaned up a few warnings
by adding some additional DECL macros.
svn path=/trunk/; revision=3593
* HTMLparser.c: applied patch from Steven Rainwater to fix
UTF8ToHtml behaviour on code points which are not mappable to
predefined HTML entities, fixes#377544
Daniel
* HTMLparser.c: fixing HTML minimized attribute values to be generated
internally if not present, fixes bug #332124
* result/HTML/doc2.htm.sax result/HTML/doc3.htm.sax
result/HTML/wired.html.sax: this affects the SAX event strem for
a few test cases
Daniel
* HTMLparser.c: fixing HTML entities in attributes parsing bug #362552
* result/HTML/entities2.html* test/HTML/entities2.html: added to
the regression suite
Daniel
* HTMLparser.c: fix#348252 if the document clains to be in a
different encoding in the meta tag and it's obviously wrong,
don't screw up the end of the content.
Daniel
* HTMLparser.c: remove a warning
* encoding.c: check with uppercase for AIX iconv() should fix#352644
* doc/examples/Makefile.am: partially handle one bug report
Daniel
* xmllint.c: added --html --memory to test htmlReadMemory to
test #321632
* HTMLparser.c: added various initialization calls which may help
#321632 but not conclusive
* testapi.c tree.c include/libxml/tree.h: fixed compilation with
--with-minimum --with-sax1 and --with-minimum --with-schemas
fixing #326442
Daniel
* HTMLparser.c configure.in parserInternals.c runsuite.c runtest.c
testapi.c xmlschemas.c xmlschemastypes.c xmlstring.c: fixed a number
of warnings shown by HP-UX compiler and reported by Rick Jones
Daniel
* HTMLparser.c: script HTML parser error fix, corrects bug #319715
* result/HTML/53867* test/HTML/53867.html: added test from Michael Day
to the regression suite
Daniel
* HTMLparser.c: Applied the last patch from Gary Coady for #304637
changing the behaviour when text nodes are found in body
* result/HTML/*: this changes the output of some tests
Daniel
* HTMLparser.c parser.c SAX2.c debugXML.c tree.c valid.c xmlreader.c
xmllint.c include/libxml/HTMLparser.h include/libxml/parser.h:
added a parser XML_PARSE_COMPACT option to allocate small
text nodes (less than 8 bytes on 32bits, less than 16bytes on 64bits)
directly within the node, various changes to cope with this.
* result/XPath/tests/* result/XPath/xptr/* result/xmlid/*: this
slightly change the output
Daniel
* HTMLparser.c SAX2.c encoding.c globals.c parser.c relaxng.c
runsuite.c runtest.c schematron.c testHTML.c testReader.c
testRegexp.c testSAX.c testThreads.c valid.c xinclude.c xmlIO.c
xmllint.c xmlmodule.c xmlschemas.c xpath.c xpointer.c: a lot of
small cleanups based on Linus' sparse check output.
Daniel
* HTMLparser.c: applied UTF-8 script parsing bug #310229 fix from
Jiri Netolicky
* result/HTML/script2.html* test/HTML/script2.html: added the test
case from the regression suite
Daniel
* HTMLparser.c: applied patch from James Bursa fixing an html parsing
bug in push mode
* result/HTML/repeat.html* test/HTML/repeat.html: added the test to the
regression suite
Daniel
* doc/apibuild.py doc/elfgcchack.xsl: revamped the elfgcchack.h
format to cope with gcc4 change of aliasing allowed scopes, had
to add extra informations to doc/libxml2-api.xml to separate
the header from the c module source.
* *.c: updated all c library files to add a #define bottom_xxx
and reimport elfgcchack.h thereafter, and a bit of cleanups.
* doc//* testapi.c: regenerated when rebuilding the API
Daniel
Synchronized the header files with the library code in order
to assure that all the various conditionals (LIBXML_xxxx_ENABLED)
were the same in both. Modified the API database content to more
accurately reflect the conditionals. Enhanced the generation
of that database. Although there was no substantial change to
any of the library code's logic, a large number of files were
modified to achieve the above, and the configuration script
was enhanced to do some automatic enabling of features (e.g.
--with-xinclude forces --with-xpath). Additionally, all the format
errors discovered by apibuild.py were corrected.
* configure.in: enhanced cross-checking of options
* doc/apibuild.py, doc/elfgcchack.xsl, doc/libxml2-refs.xml,
doc/libxml2-api.xml, gentest.py: changed the usage of the
<cond> element in module descriptions
* elfgcchack.h, testapi.c: regenerated with proper conditionals
* HTMLparser.c, SAX.c, globals.c, tree.c, xmlschemas.c, xpath.c,
testSAX.c: cleaned up conditionals
* include/libxml/[SAX.h, SAX2.h, debugXML.h, encoding.h, entities.h,
hash.h, parser.h, parserInternals.h, schemasInternals.h, tree.h,
valid.h, xlink.h, xmlIO.h, xmlautomata.h, xmlreader.h, xpath.h]:
synchronized the conditionals with the corresponding module code
* doc/examples/tree2.c, doc/examples/xpath1.c, doc/examples/xpath2.c:
added additional conditions required for compilation
* doc/*.html, doc/html/*.html: rebuilt the docs
* HTMLparser.c parser.c: make sure xmlCtxtReadFile and htmlCtxtReadFile
go through the catalog resolution.
* gentest.py testapi.c: fix a side effect wrning of the change
Daniel
* gentest.py testapi.c: better handling of conditional features
* HTMLparser.c SAX2.c parserInternals.c xmlwriter.c: more testing
on parser contexts closed leaks, error messages
Daniel
* gentest.py testapi.c: autogenerate a minimal NULL value sequence
for unknown pointer types
* HTMLparser.c SAX2.c chvalid.c encoding.c entities.c parser.c
parserInternals.c relaxng.c valid.c xmlIO.c xmlreader.c
xmlsave.c xmlschemas.c xmlschemastypes.c xmlstring.c xpath.c
xpointer.c: This uncovered an impressive amount of entry points
not checking for NULL pointers when they ought to, closing all
the open gaps.
Daniel
