1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-25 06:04:04 +03:00

168 Commits

Author SHA1 Message Date
Ralph Boehme
1c60dc5c32 lib/util/charset: fix conversion failure logging
Move catch-all debug statement with loglevel 0 from behind the switch
clause into the switch clause as default case. Fixes an issue that
resulted in the log being flooded with level 0 messages in case someone
put a file with an illegal UTF8 encoding (eg '\xA0test') on the server.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2015-06-09 18:11:13 +02:00
Volker Lendecke
c51300ad89 lib: load_case_tables() -> smb_init_locale()
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2015-03-24 00:00:20 +01:00
Volker Lendecke
06a727e8f8 lib: Remove load_case_tables_library()
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2015-03-24 00:00:20 +01:00
Volker Lendecke
00df00d8d0 lib: Convert [up|low]case.dat to C
This creates a bit more source code, but it removes the requirement to
explicitly mmap the files

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2015-03-24 00:00:20 +01:00
Volker Lendecke
dee2b958c0 lib: Move get_iconv_handle() call down
We only need this in the slow path. A good compiler might detect this,
but as get_iconv_handle() might have side-effects from a compiler's
point of view, I'm not sure it's legal to skip the call in the fast
path. Might be premature optimization, but it's low hanging fruit :-)

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed by: Ira Cooper <ira@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Mon Mar 23 21:21:15 CET 2015 on sn-devel-104
2015-03-23 21:21:15 +01:00
Volker Lendecke
e2ee1a42cf lib: Fix blank line endings
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-12-09 04:12:07 +01:00
Volker Lendecke
4debc30117 lib: Fix a typo
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-12-09 04:12:07 +01:00
Volker Lendecke
e60433efa1 lib: Simplify load_case_tables_library()
We don't really need a talloc context here

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-12-08 00:33:06 +01:00
Stefan Metzmacher
3f0c7e8794 lib/util/tests: avoid some compiler warnings
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-11-14 23:27:04 +01:00
Martin Schwenke
d55cdc367d charset: Avoid compiler warnings with --enable-developer
These:

  [ 44/309] Compiling ../lib/util/charset/util_unistr_w.c
  ../../lib/util/charset/util_unistr_w.c: In function ‘strlower_w’:
  ../../lib/util/charset/util_unistr_w.c:182:127: warning: right-hand operand of comma expression has no effect [-Wunused-value]
      COPY_UCS2_CHAR(s,&v);

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-09-19 18:11:11 +02:00
Volker Lendecke
64271c493d lib: strings: Simplify strcasecmp
This makes us fallback to strcasecmp early if any INVALID_CODEPOINT
appears. Without this patch we just continue to compare if both strings
happen to have an INVALID_CODEPOINT in the same spot.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-08-06 01:05:14 +02:00
Jeremy Allison
ca1617121a s4: tests: Added local.charset test for Bug 10716 - smbd constantly crashes when filename contains non-ascii character
https://bugzilla.samba.org/show_bug.cgi?id=10716

Signed-off-by: Jeremy Allison <jra@samba.org>
2014-08-06 01:05:14 +02:00
Jeremy Allison
dfe8dd87e1 lib: strings: Fix the behavior of strncasecmp_m_handle() in the face of bad conversions.
When either string has a bad conversion, we fall back to
doing raw ascii byte comparisons using strcasecmp(). This
is wrong - we should fall back to strncasecmp.

The problem is we've already stepped past the character
that failed the conversion, so we're not re-testing those
characters for comparison. This can have the effect of
causing strncasecmp_m_handle() to report that two strings
are identical when they are not, if the failed conversion
takes place at the end of the string.

The correct behavior is to step back to the point of
the string(s) that failed the conversion, and continue
the test from there.

This is a litle trickier than the previous fix, as
it requires converting the incoming n variable from
remaining characters to compare to remaining bytes to
compare.

As bytes are always the smallest character size
(1 byte) then it's safe to convert the remaining
characters to check by decrementing the source string
by the last character length (in bytes) and incrementing
the remaining bytes to scan by the same value, then
calling strncasecmp() with the stepped back strings
remaining.

Signed-off-by: Jeremy Allison <jra@samba.org>
2014-08-06 01:05:14 +02:00
Jeremy Allison
9380478a0b lib: strings: Fix the behavior of strcasecmp_m_handle() in the face of bad conversions.
When either string has a bad conversion, we fall back to
doing raw ascii byte comparisons using strcasecmp().

The problem is we've already stepped past the character
that failed the conversion, so we're not re-testing those
characters for comparison. This can have the effect of
causing strcasecmp_m_handle() to report that two strings
are identical when they are not, if the failed conversion
takes place at the end of the string.

The correct behavior is to step back to the point of
the string(s) that failed the conversion, and continue
the test from there.

Found by <lev@zadarastorage.com> when investigating bug
10716 - smbd constantly crashes when filename contains non-ascii character.

Given the normal character set of utf-8, and an on
disk filename of ISO-8859-1 of file-é on disk hex
value: 66 69 6c 65 2d e9, an incoming open given the
correct utf8 name of file-é will collide when it
should not.

Fixes:

Bug 10716 - smbd constantly crashes when filename contains non-ascii character

https://bugzilla.samba.org/show_bug.cgi?id=10716

Signed-off-by: Jeremy Allison <jra@samba.org>
2014-08-06 01:05:14 +02:00
Stefan Metzmacher
8ba775b0a3 lib/util/tests: add missing #include "torture/local/proto.h"
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2014-04-02 09:03:44 +02:00
Volker Lendecke
70dfb51430 iconv: Use a static buffer in iconf not to spoil the talloc_pool
This is a buffer that is strictly used like a stack variable. This
patch makes it one and while there it fixes an error path memleak.
In the "pull_failed" case we did not talloc_free(cvtbuf). With
talloc_tos(), this does not really matter, but for code without
this it does.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2013-12-14 10:06:28 +01:00
Andrew Bartlett
3902e7332d lib/util/charset: We do not use fucntions from wchar.h any more
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Sep 26 02:13:10 CEST 2012 on sn-devel-104
2012-09-26 02:13:10 +02:00
Andrew Bartlett
1c5c96d7cd lib/util/charset: Try to find iconv on HP-UX 2012-09-26 00:24:20 +02:00
Jelmer Vernooij
ce10a7a673 lib/util: Fix typo in comment. 2012-06-15 01:18:08 +02:00
Andrew Bartlett
fc5a214d71 charset: Remove unused strcmp_w()
Found by callcatcher.

Found by callcatcher: http://www.skynet.ie/~caolan/Packages/callcatcher.html

Andrew Bartlett
2012-02-10 16:45:13 +11:00
Andrew Bartlett
983d205486 charset: Remove unused iconv_talloc()
Found by callcatcher: http://www.skynet.ie/~caolan/Packages/callcatcher.html

Andrew Bartlett
2012-02-10 16:45:11 +11:00
Jelmer Vernooij
d6b4701503 charset/tests: Add prototypes for test functions. 2012-02-07 01:11:07 +01:00
Volker Lendecke
1364eb7bd7 lib/charset: Remove an unused variable
Autobuild-User: Volker Lendecke <vlendec@samba.org>
Autobuild-Date: Sun Dec 25 15:07:56 CET 2011 on sn-devel-104
2011-12-25 15:07:55 +01:00
Volker Lendecke
b2eaa9afd3 s3: Fix fn signatures in charset_macosx.c 2011-12-25 13:31:58 +01:00
Andrew Bartlett
5cb6adb590 lib/util/charset Remove unused header smb_common.h
This should fix a compile issue on hosts without gssapi/gssapi.h.

Andrew Bartlett

Signed-off-by: Stefan Metzmacher <metze@samba.org>
2011-10-24 13:56:45 +02:00
Jeremy Allison
f4a41ce5a6 Fix const warnings. 2011-10-14 13:51:26 -07:00
Andrew Bartlett
2a789c8442 build: Fix waf build on MacOS X
The -framework CoreFoundation is required by the charset_macosxfs module

The system/time.h header is required to access the replacement clock_gettime()

Andrew Bartlett

Autobuild-User: Andrew Bartlett <abartlet@samba.org>
Autobuild-Date: Fri Sep 23 10:58:02 CEST 2011 on sn-devel-104
2011-09-23 10:58:02 +02:00
Andrew Bartlett
f9acf770e9 lib/util/charset: remove charset module loading
Now that the 'table' modules are gone, there is no reason for there to
be charset modules at all.  This builds the macosxfs and weird modules
into the binary at the appropriate times, and changes the tests to
test instead the difference between the remaining internal handlers
and iconv().

Andrew Bartlett

Autobuild-User: Andrew Bartlett <abartlet@samba.org>
Autobuild-Date: Tue Sep 20 06:27:06 CEST 2011 on sn-devel-104
2011-09-20 06:27:06 +02:00
Andrew Bartlett
8316577b75 lib/util/charcnv: Remove broken internal CP850 and CP464 modules
These modules are now known to be faulty, and Samba 3.6.0 didn't
include support for them, so we now require a system iconv if you wish
to support these character sets for the non-ASCII range.

Andrew Bartlett
2011-09-20 04:53:05 +02:00
Andrew Bartlett
86b1f45a5e lib/util/charset: the comparsion must be against our charset modules
There is little point testing this against the system iconv.

Andrew Bartlett

Autobuild-User: Andrew Bartlett <abartlet@samba.org>
Autobuild-Date: Sat Sep 10 15:45:42 CEST 2011 on sn-devel-104
2011-09-10 15:45:42 +02:00
Andrew Bartlett
142f871cef lib/util/charset Use name of ISO-8859-1 that matches our internal implementation 2011-09-10 14:18:06 +02:00
Andrew Bartlett
0022296349 lib/util/charset: Improve toture assertions in iconv test 2011-09-10 14:18:06 +02:00
Andrew Bartlett
2085dffddb lib/util/charset Run charset sets with and without the system iconv
We need to know that we can load the samba-provided modules, and that
they are correct.  However, we must mark a number of tests as
knownfail due to errors in our internal iconv modules.

Andrew Bartlett
2011-09-10 14:18:06 +02:00
Andrew Bartlett
fc6bb5d47a lib/util/charset: add back loading of charset modules
For autoconf builds these remain as modules, for waf builds they are
built into the charset library.

This is required to provide the CP850 charset when iconv is not available.

The charset modules static for the waf builds because with proper
shared libs, there isn't the same need for these to be in seperate .so
files.  The modules are also not produced if a system iconv is found,
except for developers, to allow testing of both code paths.

Andrew Bartlett
2011-09-10 14:18:06 +02:00
Volker Lendecke
1d4de78f56 Fix bug 8433, segfault in iconv.c
Autobuild-User: Volker Lendecke <vlendec@samba.org>
Autobuild-Date: Thu Sep  1 18:25:34 CEST 2011 on sn-devel-104
2011-09-01 18:25:34 +02:00
Jelmer Vernooij
0397b2ef30 charset: Make name lowercase everywhere. 2011-07-23 22:08:10 +02:00
Jeremy Allison
93dcfdea38 Second part of fix for bug 8310 - toupper_ascii() is broken on big-endian systems.
Re-add:
	smb_ucs2_t toupper_w(smb_ucs2_t v);

and ensure it is called whenever we are operating on smb_ucs2_t
variables. I'd like to make the definition of smb_ucs2_t incompatible
with int and codepoint_t so they can't be mixed, but that's a patch
for another time.

Autobuild-User: Jeremy Allison <jra@samba.org>
Autobuild-Date: Tue Jul 19 23:48:05 CEST 2011 on sn-devel-104
2011-07-19 23:48:05 +02:00
Jeremy Allison
ee34c25c8a First part of fix for bug 8310 - toupper_ascii() is broken on big-endian systems
Remove
int toupper_ascii(int c);
int tolower_ascii(int c);
int isupper_ascii(int c);
int islower_ascii(int c);

and replace with their _m equivalents, as they are identical.
2011-07-19 13:19:29 -07:00
Volker Lendecke
a52f179951 lib: Allow NULL converted_size in convert_string_talloc 2011-06-24 21:24:17 +02:00
Andrew Bartlett
de0e011944 lib/util/charset: Remove autodetection of charset from LOCALE
In the past, our LOCALE would set the display charset of Samba.  The
display charset has now been removed.  This patch removes the support
code that detected the locale from the environment.  We cannot safely
have 'unix charset' follow the locale (at it creates files on disk and
entries in databases that must not vary), so this code is unused.

As an example, imagine a database is manipulated in the
administrator's locale, and then read by smbd starting up in the
system default locale.  Or smbd restarted by the administrator rather
than a startup script.  Both of these situations could corrupt
databases or filenames on disk.

Andrew Bartlett
2011-06-23 13:47:27 +02:00
Andrew Bartlett
125a2ff262 lib/util/charset: Remove 'display charset'
As discussed in 'CH_DISPLAY and gettext' on the samba-technical list:
http://lists.samba.org/archive/samba-technical/2011-June/078190.html

Setting this to a value other than 'unix charset' does not make sense,
as any system where the filesytem charset does not equal the terminal
charset will already have problems with programs as simple as 'ls'.
It also means that our output could not be pasted as our input in
interactive programs or onto our command line, as we never did
translate in the DISPLAY -> UNIX direction.

The d_printf() calls are retained in case we need to revisit this, and
to support display_set_stderr().

Andrew Bartlett
2011-06-23 13:47:27 +02:00
Sean Finney
0f8018676a Fix numerous missing dependencies in WAF build scripts
With the recent consolidation of code between s3 and s4, a number of new
dependencies have been implicitly introduced.  For example, previous s3
code gained an implicit dependency on talloc after the charset related
consolidation (lib/util/charset/charset.h now includes talloc.h).  When
building against the embedded version of talloc this isn't a problem
since the paths are automatically added to the search path, but when
building against the external libraries build failures will occur for
all components that don't directly or indirectly include talloc as
a dependency.

Since charset.h is included from util.h, which in turn is included from
includes.h, this means most of the codebase (s3 and s4) has such an
undeclared dependency.

Therefore, samba-util-common and samba-util have been added as
dependencies to the s3 and s4 code respectively, for all cases where
the source would otherwise fail to build.  Additionally, a few other
dependencies are added in specific wscript_build files to address
similar dependency-related problems.

https://bugzilla.samba.org/show_bug.cgi?id=8128

Signed-off-by: Sean Finney <seanius@seanius.net>
Signed-off-by: Matthias Dieter Wallnöfer <mdw@samba.org>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User: Matthias Dieter Wallnöfer <mdw@samba.org>
Autobuild-Date: Wed May 25 19:22:13 CEST 2011 on sn-devel-104
2011-05-25 19:22:13 +02:00
Andrew Bartlett
f19ab5d334 lib/util/charset: Remove unused strcasecmp_w and strncasecmp_w
Autobuild-User: Andrew Bartlett <abartlet@samba.org>
Autobuild-Date: Wed May 18 17:22:15 CEST 2011 on sn-devel-104
2011-05-18 17:22:15 +02:00
Andrew Bartlett
57f41ef150 lib/util/charset use talloc_stackframe() rather than talloc_tos()
This is common code, and we can't assume a talloc_stackframe() so we
must create it.

Andrew Bartlett
2011-05-18 16:12:08 +02:00
Andrew Bartlett
3c8de7dd66 lib/util/charset Don't allow invalid 'dos charset = utf8'
No DOS client used UTF8, and this creates subtle, difficult to
disagnose breakage of schannel (domain membership).

Andrew Bartlett
2011-05-18 16:12:08 +02:00
Günther Deschner
0092240401 lib/util/charset: fix the toplevel MacOS X build.
Guenther

Autobuild-User: Günther Deschner <gd@samba.org>
Autobuild-Date: Tue May 17 16:16:59 CEST 2011 on sn-devel-104
2011-05-17 16:16:59 +02:00
Andrew Bartlett
f18cca9a0a lib/util/charset Move built-in charset modules to the top level
This removes the 'charset' subsystem and allows these modules to be
used across the whole of Samba.

Andrew Bartlett
2011-05-13 18:50:23 +02:00
Jeremy Allison
4f41be356a Fix many const compiler warnings. 2011-05-05 10:41:59 -07:00
Jeremy Allison
1c714850d5 Clean up some const and other compiler warnings.
Autobuild-User: Jeremy Allison <jra@samba.org>
Autobuild-Date: Thu May  5 00:59:40 CEST 2011 on sn-devel-104
2011-05-05 00:59:40 +02:00
Andrew Bartlett
80f1d49b61 lib/util/charset Use push_string and talloc_strupper/strlower from common code
The only caller of push_string() (not to be confused with
push_string_check()) in the common code was encode_pw_buffer(), and it
didn't use the alignment or STR_UPPER flags.

The talloc_strupper() and talloc_strlower() functions are tested in
smbtorture, and are next_codepoint() based.

Andrew Bartlett
2011-05-03 07:37:07 +02:00