1
0
mirror of https://gitlab.gnome.org/GNOME/libxml2.git synced 2024-10-26 12:25:09 +03:00
Commit Graph

211 Commits

Author SHA1 Message Date
Nick Wellnhofer
dae160c64b encoding: Fix table entry for "UTF16" 2024-09-13 12:08:20 +02:00
Nick Wellnhofer
6e503eb742 encoding: Handle more ICU error codes
U_ILLEGAL_ESCAPE_SEQUENCE and U_UNSUPPORTED_ESCAPE_SEQUENCE can occur
with ISO-2022.
2024-09-10 03:34:46 +02:00
Nick Wellnhofer
55d36c5990 encoding: Fix error code in xmlUconvConvert
Broke in 46ec621e.
2024-09-10 03:11:18 +02:00
Nick Wellnhofer
34c9108f15 encoding: Add sizeOut argument to xmlCharEncInput
When push parsing, we want to convert as much of the input as possible.
When pull parsing memory buffers, we want to convert data chunk by chunk
to save memory.
2024-07-16 17:42:10 +02:00
Nick Wellnhofer
1cfc5b8089 entities: Rework serialization of numeric character references 2024-07-16 17:42:10 +02:00
Nick Wellnhofer
69f12d6d47 encoding: Deprecate xmlByteConsumed
This was only used by Chromium/WebKit to detect whether xmlParseContent
really succeeded. It's a horrible, overcomplicated hack.

See 8c5848bd and #767.
2024-07-13 15:42:02 +02:00
Nick Wellnhofer
d099795611 encoding: Readd some UTF-8 validation to encoders
This isn't strictly needed but avoids generating invalid UTF-16 and
unsigned integer overflows.
2024-07-10 22:26:19 +02:00
Nick Wellnhofer
f48eefe3d0 encoding: Rework xmlByteConsumed
Don't loop infinitely if input buffer is too large. Allocate conversion
buffer on the heap.
2024-07-09 14:25:32 +02:00
Nick Wellnhofer
f86d17c163 encoding: Fix xmlParseCharEncoding
Make "UTF-16" return the UTF16LE handler as before.

Fix error return.
2024-07-04 15:47:20 +02:00
Nick Wellnhofer
46ec621eb7 encoding: Clarify xmlUconvConvert 2024-07-03 16:06:59 +02:00
Nick Wellnhofer
48fec2429b encoding: Remove duplicate code
Fix recent commit.
2024-07-03 15:11:20 +02:00
Nick Wellnhofer
71fb257912 encoding: Fix ICU build 2024-07-03 14:35:49 +02:00
Nick Wellnhofer
9a4770ef84 doc: Improve documentation 2024-07-02 13:34:04 +02:00
Nick Wellnhofer
0b0dd98983 parser: Fix EBCDIC detection 2024-07-01 18:05:40 +02:00
Nick Wellnhofer
37a9ff11d8 encoding: Simplify xmlCharEncCloseFunc 2024-07-01 18:05:40 +02:00
Nick Wellnhofer
1167c3340e encoding: Don't include iconv.h from libxml/encoding.h 2024-07-01 18:05:40 +02:00
Nick Wellnhofer
30be984a0f encoding: Rework ISO-8859-X conversion
Optimize code. Pass tables as context parameter. Check for
XML_ENC_ERR_SPACE.
2024-07-01 18:05:40 +02:00
Nick Wellnhofer
282ec1d548 encoding: Rework xmlCharEncodingHandler layout
Reuse some of the old members.

The "input" and "output" function pointers are actually of type
xmlCharEncConvFunc, accepting an additional argument. For default
handlers, this argument is unused, so this should work with most ABIs.
For iconv handlers, these function pointers used to be NULL but now
point to a function which requires the extra argument.

"iconv_in" and "iconv_out" are made void pointers. "uconv_in" and
"uconv_out" are renamed and made void pointers. This is unlikely to
cause issues.

We now expect that the built-in conversion functions correctly report
XML_ENC_ERR_SPACE. For UTF8ToHtml and the ISO-8859-X code, this will be
done in the following commits.
2024-07-01 18:05:40 +02:00
Nick Wellnhofer
57e37dff4e encoding: Rework UTF-16 conversion functions
Optimize UTF-16 conversion functions. Avoid misaligned memory access.
Don't rely on 'sizeof(short) == 2'. Check for XML_ENC_ERR_SPACE. Add
some tests for UTF-16 conversion.
2024-07-01 18:05:40 +02:00
Nick Wellnhofer
bb8e81c788 encoding: Rework simple conversions function
Use a single function for ASCII conversion. Optimize code. Check for
XML_ENC_ERR_SPACE.
2024-07-01 18:05:40 +02:00
Nick Wellnhofer
501e5d195d encoding: Stop using XML_ENC_ERR_PARTIAL 2024-07-01 18:05:40 +02:00
Nick Wellnhofer
c59c24494d encoding: Support custom implementations 2024-07-01 18:05:40 +02:00
Nick Wellnhofer
1e3da9f4d4 encoding: Start with callbacks 2024-07-01 18:05:40 +02:00
Nick Wellnhofer
6d8427dc97 encoding: Rework encoding lookup
Add missing xmlCharEncoding enum values.

Simplify and speed up encoding lookup by using a table mapping names to
xmlCharEncoding enums and binary search. Rearrange the default handler
table to match the enum layout.

For some encodings we now only lookup the provided or most canonical
name instead of trying several names, expecting that iconv or ICU handle
aliases:

- IBM037 (EBCDIC)
- UCS-2
- UCS-4
- Shift_JIS
2024-07-01 18:05:40 +02:00
Nick Wellnhofer
f4e63f7a4f Regenerate libxml2-api.xml and testapi.c 2024-06-27 15:17:40 +02:00
Nick Wellnhofer
b1a416bf52 encoding: Restore old lookup order in xmlOpenCharEncodingHandler
When looking up encodings with xmlLookupCharEncodingHandler, the
returned handler can have a different name than requested
(capitalization, internal aliases). This should eventually be fixed.
For now we revert part of commit 5b893fa9, start the lookup with
xmlFindHandler and add an explicit check for UTF-8.

Should fix the encoding name issue mentioned in #749.
2024-06-27 12:34:45 +02:00
Nick Wellnhofer
c4d8343b7c encoding: Make xmlFindCharEncodingHandler return UTF-8 handler
xmlFindCharEncodingHandler must always return a handler.

Remove UTF-8 handler from default handler list.

Fixes 5b893fa9.
2024-06-24 20:08:27 +02:00
Nick Wellnhofer
5b893fa999 encoding: Fix encoding lookup with xmlOpenCharEncodingHandler
Make xmlOpenCharEncodingHandler call xmlParseCharEncoding first so we
prefer our own handlers for names like "UTF8". Only UTF-16 needs an
exception.

Make callers check the return value. For UTF-8, a NULL encoding doesn't
mean an error.

Remove unnecessary UTF-8 check from htmlFindOutputEncoder. Don't try to
look up ASCII handler since the HTML handler is always available.

Fix return code of xmlParseCharEncoding.

Should fix #744.
2024-06-22 21:59:03 +02:00
Rosen Penev
2def7b4b28 clang-tidy: move assignments out of if
Found with bugprone-assignment-in-if-condition

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2024-06-20 21:11:44 -07:00
Nick Wellnhofer
63ce5f9aed Make some globals const 2024-04-28 17:53:39 +02:00
Nick Wellnhofer
072facc49e encoding: Don't shrink input too early in xmlCharEncOutput
Some exotic encodings like ISO646-FR don't support '#' characters, so
encoding a character reference can actually fail. Don't skip the
offending input in this case so the error will be reported on the next
call.
2024-03-18 15:14:43 +01:00
Nick Wellnhofer
0821efc8ee encoding: Check whether encoding handlers support input/output
The "HTML" encoding handler doesn't support input which could lead to a
wrong error report.
2024-01-02 19:48:23 +01:00
Nick Wellnhofer
023aecc474 encoding: Support ASCII in xmlLookupCharEncodingHandler
Return our built-in ASCII handler. This was never implemented and
triggered the new and stricter error checks.
2023-12-13 23:58:45 +01:00
Nick Wellnhofer
bd5ad0308d encoding: Report malloc failures
Introduce new API functions that return a separate error code if a
memory allocation fails.

- xmlOpenCharEncodingHandler
- xmlLookupCharEncodingHandler

Fix a few places where malloc failures weren't reported.
2023-12-11 22:05:47 +01:00
Nick Wellnhofer
89d19534de encoding: Fix decoding of large chunks
After 95e81a36, we must support XML_ENC_ERR_SPACE when using built-in
encoding handlers.

Should fix #610.
2023-10-28 03:14:13 +02:00
Nick Wellnhofer
1734d27dca encoding: Suppress -Wcast-align warnings 2023-10-02 15:04:18 +02:00
Nick Wellnhofer
0533daf5d2 encoding: Fix infinite loop in xmlCharEncInput
Short-lived regression from 95e81a36.
2023-09-29 12:43:46 +02:00
Nick Wellnhofer
8c084ebdc7 doc: Make apibuild.py happy 2023-09-21 22:57:33 +02:00
Nick Wellnhofer
699299cae3 globals: Stop including globals.h 2023-09-20 22:07:40 +02:00
Nick Wellnhofer
7909ff08e2 include: Remove unnecessary includes
- Don't include tree.h from encoding.h
- Don't include parser.h from xmlIO.h
2023-09-20 22:06:49 +02:00
Nick Wellnhofer
507f11edf0 encoding: Remove debugging code 2023-08-16 19:50:36 +02:00
Nick Wellnhofer
95e81a360c parser: Decode all data in xmlCharEncInput
Even with flush set to true, xmlCharEncInput didn't guarantee to decode
all data. This complicated the push parser.

Remove the flush flag and always decode all available data.

Also fix ICU code where the flush flag has a different meaning. Always
set flush to false and retry even with empty input buffers.
2023-08-08 15:21:31 +02:00
Nick Wellnhofer
4ee0815514 encoding: Move rawconsumed accounting to xmlCharEncInput 2023-08-08 15:19:51 +02:00
Nick Wellnhofer
b236b7a588 parser: Halt parser when growing buffer results in OOM
Fix short-lived regression from previous commit.

It might be safer to make xmlBufSetInputBaseCur use the original buffer
even in case of errors.

Found by OSS-Fuzz.
2023-06-08 21:59:20 +02:00
Nick Wellnhofer
db21cd5db9 malloc-fail: Handle malloc failures in xmlAddEncodingAlias
Avoid memory errors if an allocation fails.

See #344. Fixes #553.
2023-06-06 14:25:30 +02:00
Nick Wellnhofer
2f12e3a938 encoding: Stop calling xmlEncodingErr
This invokes the global error handler which should be avoided.
2023-04-30 21:45:39 +02:00
Nick Wellnhofer
320f5084cd parser: Improve handling of encoding and IO errors
Make sure that xmlCharEncInput, xmlParserInputBufferPush and
xmlParserInputBufferGrow set the correct error code in the
xmlParserInputBuffer. Handle errors when calling these functions.
2023-04-30 21:31:54 +02:00
Nick Wellnhofer
3ff6abbf58 encoding: Rework error codes
Use an enum instead of magic numbers. Fix a few error codes. Simplify
handling of "space" and "partial" errors.

See #506.
2023-04-30 16:43:29 +02:00
Nick Wellnhofer
33fb297b36 encoding: Fix compiler warning in ICU build 2023-04-17 14:59:47 +02:00
Nick Wellnhofer
a6b9e55a9e encoding: Fix error code in asciiToUTF8
Use correct error code when invalid ASCII bytes are encountered.

Found by OSS-Fuzz.
2023-03-26 15:42:02 +02:00