IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
<li><ahref="encoding.html#extend">How to extend the existing
support</a></li>
</ol><h3><aname="What"id="What">What does internationalization support mean ?</a></h3><p>XML was designed from the start to allow the support of any character set
by using Unicode. Any conformant XML parser has to support the UTF-8 and
UTF-16 default encodings which can both express the full unicode ranges. UTF8
is a variable length encoding whose greatest points are to reuse the same
encoding for ASCII and to save space for Western encodings, but it is a bit
more complex to handle in practice. UTF-16 use 2 bytes per character (and
sometimes combines two pairs), it makes implementation easier, but looks a
bit overkill for Western languages encoding. Moreover the XML specification
allows the document to be encoded in other encodings at the condition that
they are clearly labeled as such. For example the following is a wellformed
XML document encoded in ISO-8859-1 and using accentuated letters that we
French like for both markup and content:</p><pre><?xml version="1.0" encoding="ISO-8859-1"?>
<très>là </très></pre><p>Having internationalization support in libxml2 means the following:</p><ul><li>the document is properly parsed</li>
</html></pre><h3><aname="internal"id="internal">The internal encoding, how and why</a></h3><p>One of the core decisions was to force all documents to be converted to a
default internal encoding, and that encoding to be UTF-8, here are the
rationales for those choices:</p><ul><li>keeping the native encoding in the internal form would force the libxml
users (or the code associated) to be fully aware of the encoding of the
original document, for examples when adding a text node to a document,
the content would have to be provided in the document encoding, i.e. the
client code would have to check it before hand, make sure it's conformant
to the encoding, etc ... Very hard in practice, though in some specific
cases this may make sense.</li>
<li>the second decision was which encoding. From the XML spec only UTF8 and
UTF16 really makes sense as being the two only encodings for which there
is mandatory support. UCS-4 (32 bits fixed size encoding) could be
considered an intelligent choice too since it's a direct Unicode mapping
support. I selected UTF-8 on the basis of efficiency and compatibility
with surrounding software:
<ul><li>UTF-8 while a bit more complex to convert from/to (i.e. slightly
more costly to import and export CPU wise) is also far more compact
than UTF-16 (and UCS-4) for a majority of the documents I see it used
for right now (RPM RDF catalogs, advogato data, various configuration
file formats, etc.) and the key point for today's computer
architecture is efficient uses of caches. If one nearly double the
memory requirement to store the same amount of data, this will trash
caches (main memory/external caches/internal caches) and my take is
that this harms the system far more than the CPU requirements needed
for the conversion to UTF-8</li>
<li>Most of libxml2 version 1 users were using it with straight ASCII
most of the time, doing the conversion with an internal encoding
requiring all their code to be rewritten was a serious show-stopper
for using UTF-16 or UCS-4.</li>
<li>UTF-8 is being used as the de-facto internal encoding standard for
related code like the <ahref="http://www.pango.org/">pango</a>
upcoming Gnome text widget, and a lot of Unix code (yet another place
where Unix programmer base takes a different approach from Microsoft
~/XML -></pre><p>The same processing is applied (and reuse most of the code) for HTML I18N
processing. Looking up and modifying the content encoding is a bit more
difficult since it is located in a <meta> tag under the <head>,
so a couple of functions htmlGetMetaEncoding() and htmlSetMetaEncoding() have
been provided. The parser also attempts to switch encoding on the fly when
detecting such a tag on input. Except for that the processing is the same
(and again reuses the same code).</p><h3><aname="Default"id="Default">Default supported encodings</a></h3><p>libxml2 has a set of default converters for the following encodings
(located in encoding.c):</p><ol><li>UTF-8 is supported by default (null handlers)</li>
<li>HTML, a specific handler for the conversion of UTF-8 to ASCII with HTML
predefined entities like &copy; for the Copyright sign.</li>
</ol><p>More over when compiled on an Unix platform with iconv support the full
set of encodings supported by iconv can be instantly be used by libxml. On a
linux machine with glibc-2.1 the list of supported encodings and aliases fill
3 full pages, and include UCS-4, the full set of ISO-Latin encodings, and the
various Japanese ones.</p><p>To convert from the UTF-8 values returned from the API to another encoding
then it is possible to use the function provided from <ahref="html/libxml-encoding.html">the encoding module</a> like <ahref="html/libxml-encoding.html#UTF8Toisolat1">UTF8Toisolat1</a>, or use the
</ul><h3><aname="extend"id="extend">How to extend the existing support</a></h3><p>Well adding support for new encoding, or overriding one of the encoders
(assuming it is buggy) should not be hard, just write input and output
conversion routines to/from UTF-8, and register them using
xmlNewCharEncodingHandler(name, xxxToUTF8, UTF8Toxxx), and they will be
called automatically if the parser(s) encounter such an encoding name
(register it uppercase, this will help). The description of the encoders,
their arguments and expected return values are described in the encoding.h