mirror of
https://github.com/samba-team/samba.git
synced 2025-01-25 06:04:04 +03:00
e21251926b
If two strings are invalid UTF-8, the string is first compared with
memcmp(), which compares as unsigned char.
If the strings are of different lengths and one is a substring of the
other, the memcmp() returns 0 and a second comparison is made which
assumes the next character in the shorter string is '\0' -- but this
comparison was done using SIGNED chars (on most systems). That leads
to non-transitive comparisons.
Consider the strings {"a\xff", "a", "ab\xff"} under that system.
"a\xff" < "a", because (char)0xff == -1.
"ab\xff" > "a", because 'b' == 98.
"ab\xff" < "a\xff", because memcmp("ab\xff", "a\xff", 2) avoiding the
signed char tiebreaker.
(Before c49c48afe09a1a78989628bbffd49dd3efc154dd, the final character
might br arbitrarily cast into another character -- in latin-1, for
example, the 0xff here would have been seen as 'ÿ', which would be
uppercased to 'Ÿ', which is U+0178, which would be truncated to
'\x78', a positive char.
On the other hand e.g. 0xfe, 'þ', would have mapped to 0xde, 'Þ',
remaining negative).
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15625
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
(cherry picked from commit e2051eebd492a419f840280336eb242d0b4a26ac)