1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-25 23:21:54 +03:00
samba-mirror/source4
Joe Guo 87149445af s4/scripting/bin: open unicode files with utf8 encoding and write unicode string
In files like `libcli/util/werror_err_table.txt` and `libcli/util/ntstatus_err_table.txt`,
there were unicode quote symbols at line 6:

    ...(“this documentation”)...

In `libcli/util/wscript_build`, it will run `gen_werror.py` and `gen_ntstatus.py`
to `open` above files, read content from them and write to other files.

When encoding not specified, `open` in both python 2/3 will guess encoding from locale.

When locale is not set, it defaults to POSIX or C, and then python will use
encoding `ANSI_X3.4-1968`.

So, on a system locale is not set, `make` will fail with encoding error
for both python 2 and 3:

    File "/home/ubuntu/samba/source4/scripting/bin/gen_werror.py", line 139, in main
        errors = parseErrorDescriptions(input_file, True, transformErrorName)
      File "/home/ubuntu/samba/source4/scripting/bin/gen_error_common.py", line 52, in parseErrorDescriptions
        for line in file_contents:
      File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
        return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 318: ordinal not in range(128)

In this case, we have to use `io.open` with `encoding='utf8'`.
However, then we got unicode strs and try to write them with other strs
into new file, which means the new file must also open with utf-8 and
all other strs have to be unicode, too.

Instead of prefix `u` to all strs, a more easier/elegant way is to enable
unicode literals for the python scripts, which we normally didn't do in samba.

Since both `gen_werror.py` and `gen_ntstatus.py` are bin scripts and no
other modules import them, it should be ok for this case.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Fri Feb  8 06:34:47 CET 2019 on sn-devel-144
2019-02-08 06:34:47 +01:00
..
auth s4/pyauth: fix memory leak when context_new() has bad arguments 2019-02-01 03:36:17 +01:00
build/pasn1
cldap_server source4 smdb: Add a post fork hook to the service API 2018-11-01 23:49:24 +01:00
client s4:cifsdd: Use C99 initializers for cifsdd argdef 2019-01-28 10:29:26 +01:00
cluster dbwrap: Remove calls to loadparm 2018-04-24 01:53:19 +02:00
dns_server Decrement references to python objects passed to Py_BuildValue 2019-02-07 13:44:30 +01:00
dsdb s4:dsdb:util: make use of samba_runcmd_export_stdin() 2019-02-07 22:47:22 +01:00
echo_server source4 smdb: Add a post fork hook to the service API 2018-11-01 23:49:24 +01:00
heimdal PY3: fix "TabError: inconsistent use of tabs and spaces" 2018-09-27 01:54:27 +02:00
heimdal_build wscript: separate embedded_heimdal from system_heimdal 2019-01-29 22:09:07 +01:00
include lib: Remove global xfile.h includes 2016-11-20 06:23:19 +01:00
kdc s4:kdc: Fix size type for num_bind in kdc-heimdal 2019-01-19 15:36:51 +01:00
ldap_server CVE-2018-16851 ldap_server: Check ret before manipulating blob 2018-11-28 08:22:24 +01:00
lib Cleanup (decref) some objects added to list. 2019-02-07 13:44:30 +01:00
libcli s4:libcli: Remove (now unused) pysmb.c bindings 2019-01-26 04:05:25 +01:00
libnet Cleanup references to module objects returned from PyImport_ImportModule 2019-02-07 13:44:30 +01:00
librpc Clean up reference used with PyDict_Setxxx 2019-02-07 17:17:46 +01:00
nbt_server nbt_server: Use dom_sid_str_buf 2018-11-02 21:21:14 +01:00
ntp_signd ntpd: Use dom_sid_str_buf 2019-01-08 03:40:28 +01:00
ntvfs s4:ntvfs: Use #ifdef instead of #if for config.h definitions 2018-11-28 23:19:24 +01:00
param decref results of PyStr_FromString 2019-02-07 13:44:30 +01:00
rpc_server s4:rpc_server: implement security context multiplexing 2019-01-12 03:13:41 +01:00
script PY3: change shebang to python3 in source4/dsdb dir 2018-12-14 14:40:20 +01:00
scripting s4/scripting/bin: open unicode files with utf8 encoding and write unicode string 2019-02-08 06:34:47 +01:00
selftest Tests for segfaults in python bindings 2019-02-01 03:36:17 +01:00
setup s4/setup/tests: make sure samba-tool is called with correct py version 2018-12-10 10:38:20 +01:00
smb_server source4 smdb: Add a post fork hook to the service API 2018-11-01 23:49:24 +01:00
smbd samba: Change default process model to prefork 2019-02-01 03:36:16 +01:00
torture s4:torture: Enforce Printer Class 2019-01-28 19:34:41 +01:00
utils s4/utils/test: PY3 make sure we call correct python version for samba-tool 2018-12-10 10:38:20 +01:00
web_server s4:web_server: Fix build error 2018-12-11 13:58:30 +01:00
winbind winbind4: Use dom_sid_str_buf 2019-01-08 03:40:28 +01:00
wrepl_server source4 smdb: Add a post fork hook to the service API 2018-11-01 23:49:24 +01:00
.clang_complete
.valgrind_suppressions
wscript_build