1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-26 10:04:02 +03:00

211 Commits

Author SHA1 Message Date
Andrew Bartlett
75d5ba4109 lib/util/charset Fix string termination conditions for UTF16 strings
This punts partial UTF16 strings to iconv() to deal with, as it's not
a fast path any longer if it's got an odd length.

Andrew Bartlett

Signed-off-by: Andrew Tridgell <tridge@samba.org>
2011-04-28 03:24:05 +02:00
Andrew Bartlett
3e85b960fa dynconfig: Have only one dynconfig.o in the common code. 2011-04-27 22:22:26 +10:00
Andrew Bartlett
6f846ddf32 lib/util/charset Always set *converted_size even on failure
The caller may wish to inspect the partially converted string.

Andrew Bartlett

Autobuild-User: Andrew Bartlett <abartlet@samba.org>
Autobuild-Date: Wed Apr 20 05:17:48 CEST 2011 on sn-devel-104
2011-04-20 05:17:48 +02:00
Andrew Bartlett
79e6fea774 lib/util/charset create _handle functions for convert_string() et al
This is now API compatible with the existing code in lib/util/charset

lazy_initialize_conv() is no longer called as init_iconv() is called
when the smb.conf is processed, and get_conv_handle() will auto-init
with defaults if required.

load_case_tables_library() is no longer requried as all binaries and
libraries already load these in their entry points, as otherwise all
the other string functions would fail.

Andrew Bartlett
2011-04-20 04:31:07 +02:00
Andrew Bartlett
e1078b87de lib/util Move alpha_strcpy() from s3 into common code
Autobuild-User: Andrew Bartlett <abartlet@samba.org>
Autobuild-Date: Thu Apr 14 09:21:59 CEST 2011 on sn-devel-104
2011-04-14 09:21:59 +02:00
Volker Lendecke
4be643d4ce lib: Move next_token next to next_token_talloc
Signed-off-by: Jeremy Allison <jra@samba.org>
2011-04-13 14:13:25 -07:00
Andrew Bartlett
4158e9a7e5 s3-charcnv: Move convert_string() et al to lib/util/charset
This is the first step to this being the common convert_string
implementation.

Andrew Bartlett

Signed-off-by: Andrew Tridgell <tridge@samba.org>
2011-04-13 14:47:08 +10:00
Andrew Bartlett
d335b635c2 lib/util/charset Add many more charset tests
This confirms that the behaviour of the convert_string() API (with the
process-wide iconv handle).

Andrew Bartlett

Signed-off-by: Andrew Tridgell <tridge@samba.org>
2011-04-13 14:47:08 +10:00
Andrew Bartlett
748c31dc5d lib/util/charset Add convert_string_error()
This adds an interface that matches the source3/ convert string code.

Andrew Bartlett

Signed-off-by: Andrew Tridgell <tridge@samba.org>
2011-04-13 14:47:08 +10:00
Andrew Bartlett
8db1648f66 lib/util/charset Make ASCII conversion validate it's input
We should not just strip the high bits off unicode strings being
converted to ASCII, we need to actually fail the conversion.

Andrew Bartlett

Signed-off-by: Andrew Tridgell <tridge@samba.org>
2011-04-13 14:47:08 +10:00
Andrew Bartlett
a2c691ab9a lib/util/charset Rename convert_string test to allow a 'non_handle' test
A future commit will test (with a subset of tests) the varient of this
function without _handle.

Andrew Bartlett

Signed-off-by: Andrew Tridgell <tridge@samba.org>
2011-04-13 14:47:08 +10:00
Andrew Bartlett
7bbd701a13 lib/util/charset Add more tests for convert_string_error_handle()
This helps define the semantics of this function very clearly,
particularly for partial and invalid inputs.

Andrew Bartlett

Signed-off-by: Andrew Tridgell <tridge@samba.org>
2011-04-13 14:47:08 +10:00
Andrew Bartlett
93463829af lib/util/charset Preserve 'pull' errors even when converting via UTF16
When we do not have a direct iconv handle between any two charsets, we
must go iva UTF16.  However, we should still return the same buffer
and error code as if we were able to go direct - including the partial
conversion and the error code.

This is important for locating the invalid multibyte character in the
stream, for example.

Andrew Bartlett

Signed-off-by: Andrew Tridgell <tridge@samba.org>
2011-04-13 14:47:08 +10:00
Andrew Bartlett
1efa600144 lib/util/charset Add tests for convert_string_error_handle
These confirm that the errno is set correctly and that we stop on a
partial multibyte character

Andrew Bartlett

Signed-off-by: Andrew Tridgell <tridge@samba.org>
2011-04-13 14:47:08 +10:00
Andrew Bartlett
b21129ae20 lib/util/charset Add expected values for upper/lower case tests
Signed-off-by: Andrew Tridgell <tridge@samba.org>
2011-04-13 14:47:07 +10:00
Andrew Bartlett
cd63c9205e lib/util/charset Fix and add public interface for convert_string_error_handle
It makes much more sense for this to match the source3/ interface and
return a bool.

Andrew Bartlett

Signed-off-by: Andrew Tridgell <tridge@samba.org>
2011-04-13 14:47:07 +10:00
Andrew Bartlett
9941dfe9f6 lib/util/charset Move source3/lib/util_unistr.c to the common code.
This file (largely) contains functions to deal with UTF16 strings.

Andrew Bartlett

Signed-off-by: Andrew Tridgell <tridge@samba.org>
2011-04-13 14:47:07 +10:00
Andrew Bartlett
620921b36b lib/util/charset Add tests for strcasecmp_m(), talloc_strupper() et al
This adds tests for:
 strncasecmp_m
 strcasecmp_m
 strupper_talloc_n
 strlower_talloc
 strhaslower
 strhasupper

The tests can certainly be improved with pre-calculated upper and
lower case text, but this at least puts them though their paces.

Andrew Bartlett

Autobuild-User: Andrew Bartlett <abartlet@samba.org>
Autobuild-Date: Thu Mar 31 14:24:22 CEST 2011 on sn-devel-104
2011-03-31 14:24:22 +02:00
Volker Lendecke
466da70d77 charconv: Use talloc_tos() in the S3 build
In S3 we put a talloc_pool on top of the stack so that an allocation from
talloc_tos() normally is just a pointer increment.

Autobuild-User: Volker Lendecke <vlendec@samba.org>
Autobuild-Date: Thu Mar 31 09:14:01 CEST 2011 on sn-devel-104
2011-03-31 09:14:01 +02:00
Jeremy Allison
7cd5a79cd4 Arg. I hate the "if (xxx) return foo" all on one line style of code.
Fix the talloc leaks I introduced by not spotting these returns.

Autobuild-User: Jeremy Allison <jra@samba.org>
Autobuild-Date: Thu Mar 31 05:19:34 CEST 2011 on sn-devel-104
2011-03-31 05:19:33 +02:00
Jeremy Allison
21193c8eeb Don't burn 2k of stack on every iconv, use the heap when it's a slow call.
Autobuild-User: Jeremy Allison <jra@samba.org>
Autobuild-Date: Thu Mar 31 04:09:09 CEST 2011 on sn-devel-104
2011-03-31 04:09:09 +02:00
Andrew Bartlett
b3ffcf888c lib/util/charset smb_panic() on incorrect use of strlen_m_ext
This may save a developer some time in the future.

Andrew Bartlett

Autobuild-User: Andrew Bartlett <abartlet@samba.org>
Autobuild-Date: Thu Mar 31 02:40:31 CEST 2011 on sn-devel-104
2011-03-31 02:40:31 +02:00
Andrew Bartlett
7fe9bb8c84 lib/util/charset Add tests for strlen_m_ext() and convert_string_talloc()
This convers a few more cases for convert_string_talloc() and
introduces tests for strlen_m_ext() across complex multibyte strings.

Andrew Bartlett
2011-03-31 10:53:24 +11:00
Andrew Bartlett
d4f3a380e4 lib/util/charset Add wrapper to allow testing of strlen_m_ext()
This allows test routines to force in particular character sets, and
not rely on the smb.conf.

Andrew Bartlett
2011-03-31 10:53:24 +11:00
Andrew Bartlett
46db53b159 lib/util/charset correct calculation of UTF8 character sizes
Characters between 0x800 0x0FFFF are 3 bytes long.

Andrew Bartlett
2011-03-31 10:52:42 +11:00
Jeremy Allison
c109a70531 Fix convert_string() to take a *converted_size arg. and return a bool.
Makes these interfaces much harder to misuse and easier to ensure error
checking.

Autobuild-User: Jeremy Allison <jra@samba.org>
Autobuild-Date: Wed Mar 30 23:59:37 CEST 2011 on sn-devel-104
2011-03-30 23:59:37 +02:00
Andrew Bartlett
43e364e61b lib/util/charset Add tests for convert_string_talloc_handle()
Andrew Bartlett

Autobuild-User: Andrew Bartlett <abartlet@samba.org>
Autobuild-Date: Wed Mar 30 13:02:47 CEST 2011 on sn-devel-104
2011-03-30 13:02:47 +02:00
Andrew Bartlett
c395209791 lib/util/charset Remove pointless static bool initialised 2011-03-30 12:17:05 +02:00
Andrew Tridgell
f0922147fe charcnv: ensure convert_string_error doesn't conflict with s3 function
use _handle for handle based functions

Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>

Autobuild-User: Andrew Tridgell <tridge@samba.org>
Autobuild-Date: Fri Mar 25 05:22:24 CET 2011 on sn-devel-104
2011-03-25 05:22:24 +01:00
Andrew Bartlett
b5616adc8a lib/util/charset rename iconv_convenience to iconv_handle
This better reflects what this structure is

Andrew Bartlett
2011-03-25 04:37:06 +01:00
Andrew Tridgell
2643a7ba6b charcnv: added convert_string_error()
this will allow us to remove conv_silent

Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
2011-03-25 04:37:06 +01:00
Andrew Tridgell
aaae4123b9 unistr: moved some UCS2 macros into util_unistr
we need to move towards eliminating smb_ucs2_t. This moves a couple of
the related macros into the only file they are used in

Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
2011-03-25 04:37:06 +01:00
Andrew Tridgell
5a20325c86 libcli: created smb_constants.h
this starts the (long!) process of moving some of the SMB constants
into common files. This just moves the FLAGS2_ defines, which are
needed for common string routines (for FLAGS2_UNICODE_STRINGS)
2011-03-25 04:37:06 +01:00
Andrew Tridgell
15e84a9a09 charcnv: removed the allow_badcharcnv and allow_bad_conv options to convert_string*()
we shouldn't accept bad multi-byte strings, it just hides problems

Autobuild-User: Andrew Tridgell <tridge@samba.org>
Autobuild-Date: Thu Mar 24 01:47:26 CET 2011 on sn-devel-104
2011-03-24 01:47:26 +01:00
Andrew Tridgell
451856698f charset: remove another allow_badcharcnv check
better to fail only if there is a bad character
2011-03-24 01:01:04 +01:00
Andrew Bartlett
41051fd3d3 lib/util: Merge basic string length and comparison functions
These functions now use the codepoints for more accurate string
handling and now form common code.

Andrew Bartlett

Autobuild-User: Andrew Bartlett <abartlet@samba.org>
Autobuild-Date: Wed Mar 23 08:21:54 CET 2011 on sn-devel-104
2011-03-23 08:21:54 +01:00
Andrew Tridgell
cd4306b01e charcnv: remove the automatic fail on allow_badcharcnv true
we should just fail the call if the string really is bad
2011-03-23 12:19:29 +11:00
Jelmer Vernooij
df0b7f1c34 lib/: Fix prototypes for functions. 2011-03-19 03:20:05 +01:00
Günther Deschner
110f11d136 lib/util/charset: ICONV_WRAPPER needs talloc.
Guenther

Autobuild-User: Günther Deschner <gd@samba.org>
Autobuild-Date: Wed Feb 23 12:46:33 CET 2011 on sn-devel-104
2011-02-23 12:46:33 +01:00
Jelmer Vernooij
82fa772fa2 Fix missing dependency on libreplace.
Autobuild-User: Jelmer Vernooij <jelmer@samba.org>
Autobuild-Date: Mon Feb 21 01:17:33 CET 2011 on sn-devel-104
2011-02-21 01:17:33 +01:00
Andrew Bartlett
1ad8e5229f lib/util/charset Add back setlocale(), but only when called from binaries
When called from a library, we don't want to call this, as we may
overwrite some of our calling program's context.

Andrew Bartlett

Autobuild-User: Andrew Bartlett <abartlet@samba.org>
Autobuild-Date: Fri Feb 18 09:29:35 CET 2011 on sn-devel-104
2011-02-18 09:29:35 +01:00
Andrew Bartlett
2a3a86a86f lib/util/charcnv Move iconv handle setup in common
We now use the struct smb_iconv_convenience at the core of all our
iconv code, and use global_iconv_convenience for the callers that
don't specify one.

Andrew Bartlett
2011-02-18 18:41:01 +11:00
Andrew Bartlett
8afc271e2a lib/util/charset Use top level iconv.c in source3
The two files were very similar already, the only change required was
to adopt the s3 module registration fucntion name.

(NTSTATUS wasn't used as the charset code does not otherwise use that
type).

Andrew Bartlett

Signed-off-by: Andrew Tridgell <tridge@samba.org>
2011-02-18 18:41:00 +11:00
Andrew Bartlett
92faeaeea8 charset Remove use of {isupper,islower,toupper,tolower}_w functions
These now call the common _m functions that consider UTF16 code points.

This removes the code which will make up a 'lame' table in memory, as
this can just as correctly be handled by running the algorithm at runtime (which is to call toupper() and tolower() on characters < 128).

When used, a top level waf build will always locate the correct table
- in the build tree or outside - due to relinking the installed
binary.

Andrew Bartlett
2011-02-18 17:00:34 +11:00
Andrew Tridgell
fd000928e3 charset: allow s3 waf build to use lib/util/charset
the only conflict is with 'CHARSET' itself, which now builds
conditionally on _SAMBA_BUILD_==4

Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
2011-02-18 15:09:47 +11:00
Andrew Bartlett
13f287fbc2 lib/util/charset use get_dyn_CODEPAGEDIR(), which is in common
This is provided by the s3 and s4 dynconfig implementations.

Andrew Bartlett
2011-02-18 15:09:47 +11:00
Andrew Bartlett
a47120ae93 lib/util/charset split codepoints.c into it's own subsystem 2011-02-18 15:09:47 +11:00
Andrew Bartlett
d8d81a7786 lib/util/charset use a path to dynconfig.h that works in s3 and s4 2011-02-18 15:09:47 +11:00
Andrew Bartlett
0581a5bb3c lib/util/charset add functions isupper_m and islower_m 2011-02-18 15:09:46 +11:00
Andrew Tridgell
1867a6033c s4-charset: use dyn_CODEPAGEDIR for location of upcase.dat/lowcase.dat
this matches samba3 behaviour

Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
2011-02-07 13:22:02 +11:00