1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-11 05:18:09 +03:00
Commit Graph

121353 Commits

Author SHA1 Message Date
Martin Schwenke
c6f74e590f ctdb-recovery: GET_DBMAP from all nodes
This builds a complete list of databases across the cluster so it can
be used to create databases on the nodes where they are missing.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2020-03-23 23:45:38 +00:00
Martin Schwenke
4c0b9c3605 ctdb-recovery: Replace use of ctdb_dbid_map with local db_list
This will be used to build a merged list of databases from all nodes,
allowing the recovery helper to create missing databases.

It would be possible to also include the db_name field in this
structure but that would cause a lot of churn.  This field is used
locally in the recovery of each database so can continue to live in
the relevant state structure(s).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2020-03-23 23:45:38 +00:00
Martin Schwenke
7e5a8a4884 ctdb-daemon: Respect CTDB_CTRL_FLAG_ATTACH_RECOVERY when attaching databases
This is currently only set by the recovery daemon when it attaches
missing databases, so there is no obvious behaviour change.  However,
attaching missing databases can now be moved to the recovery helper as
long as it sets this flag.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2020-03-23 23:45:38 +00:00
Martin Schwenke
98e3d0db2b ctdb-recovery: Use CTDB_CTRL_FLAG_ATTACH_RECOVERY to attach during recovery
ctdb_ctrl_createdb() is only called by the recovery daemon, so this is
a safe, temporary change.  This is temporary because
ctdb_ctrl_createdb(), create_missing_remote_databases() and
create_missing_local_databases() will all go away soon.

Note that this doesn't cause a change in behaviour.  The main daemon
will still only defer attaches from non-recoverd processes during
recovery.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2020-03-23 23:45:38 +00:00
Martin Schwenke
17ed042590 ctdb-protocol: Add control flag CTDB_CTRL_FLAG_ATTACH_RECOVERY
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2020-03-23 23:45:37 +00:00
Martin Schwenke
fc23cd1b9c ctdb-daemon: Remove unused old client database functions
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2020-03-23 23:45:37 +00:00
Martin Schwenke
c6c89495fb ctdb-daemon: Fix database attach deferral logic
Commit 3cc230b5ee says:

  Dont allow clients to connect to databases untile we are well past
  and through the initial recovery phase

It is unclear what this commit was attempting to do.  The commit
message implies that more attaches should be deferred but the code
change adds a conjunction that causes less attaches to be deferred.
In particular, no attaches will be deferred after startup is complete.
This seems wrong.

To implement what seems to be stated in the commit message an "or"
needs to be used so that non-recovery daemon attaches are deferred
either when in recovery or before startup is complete.  Making this
change highlights that attaches need to be allowed during the
"startup" event because this is when smbd is started.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2020-03-23 23:45:37 +00:00
Amitay Isaacs
1c56d6413f ctdb-recovery: Refactor banning a node into separate computation
If a node is marked for banning, confirm that it's not become inactive
during the recovery.  If yes, then don't ban the node.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2020-03-23 23:45:37 +00:00
Amitay Isaacs
c6a0ff1bed ctdb-recovery: Don't trust nodemap obtained from local node
It's possible to have a node stopped, but recovery master not yet
updated flags on the local ctdb daemon when recovery is started.  So do
not trust the list of active nodes obtained from the local node.  Query
the connected nodes to calculate the list of active nodes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2020-03-23 23:45:37 +00:00
Amitay Isaacs
6e2f8756f1 ctdb-recovery: Consolidate node state
This avoids passing multiple arguments to async computation.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2020-03-23 23:45:37 +00:00
Amitay Isaacs
072ff4d12b ctdb-recovery: Fetched vnnmap is never used, so don't fetch it
New vnnmap is constructed using the information from all the connected
nodes.  So there is no need to fetch the vnnmap from recovery master.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2020-03-23 23:45:37 +00:00
Aurelien Aptel
1114b02a72 s3: libsmbclient.h: add missing time.h include
A recent change added a struct with timespec members in the public
libsmbclient header. This type is defined in time.h which was not
included thus making users of libsmbclient not build properly.

    /.../libsmbclient.h:158:18: error: field 'btime_ts' has incomplete type
      struct timespec btime_ts;

Fixes: bf13fe0f22 ("s3: libsmbclient: Add internal/external structures needed for readdirplus.")
Signed-off-by: Aurelien Aptel <aaptel@suse.com>
Reviewed-by: Noel Power <npower@samba.org>

Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Mon Mar 23 20:47:44 UTC 2020 on sn-devel-184
2020-03-23 20:47:44 +00:00
Andrew Bartlett
4d65e3ad21 s4-librpc: Simplify bytes or unicode input checking in python GUID() bindings
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
2020-03-23 19:12:43 +00:00
Andrew Bartlett
a7633deb89 py3: Remove #define IsPy3BytesOrString(pystr)
This was

  (PyUnicode_Check(pystr) || PyBytes_Check(pystr))

This allows us to end the use of Python 2/3 compatability macros.

The one caller will be simplified in the next commit

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <nopower@samba.org>
2020-03-23 19:12:43 +00:00
Andrew Bartlett
675ab9d6ae py3: Remove #define IsPy3Bytes PyBytes_Check
This allows us to end the use of Python 2/3 compatability macros.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
2020-03-23 19:12:43 +00:00
Andrew Bartlett
5c1867ba45 py3: Remove #define PyInt_FromLong PyLong_FromLong
This allows us to end the use of Python 2/3 compatability macros.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power
2020-03-23 19:12:43 +00:00
Andrew Bartlett
4764e8b4c7 py3: Remove #define PyInt_AsLong PyLong_AsLong
This allows us to end the use of Python 2/3 compatability macros.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
2020-03-23 19:12:43 +00:00
Andrew Bartlett
cc79726d95 py3: Remove #define PyInt_Type PyLong_Type
This allows us to end the use of Python 2/3 compatability macros.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
2020-03-23 19:12:43 +00:00
Andrew Bartlett
08a76ec096 pidl: Remove reference to PyInt_Type from error string when we wanted a sensible Long
PyInt_Type is no longer a thing in Python3, we will remove the compatability
reference shortly.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
2020-03-23 19:12:43 +00:00
Andrew Bartlett
79044e966c pidl: Remove duplicate "if (PyLong_Check($cvar)" clauses
Since we moved to Python3, these have been dead code, and it is clearer
now that we have removed the compatability define.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
2020-03-23 19:12:43 +00:00
Andrew Bartlett
3aea3b1538 py3: Remove #define PyInt_Check PyLong_Check
This will allow us to remove some unused code in the PIDL-generated
python bindings.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
2020-03-23 19:12:43 +00:00
Andrew Bartlett
9bdc5a6742 autobuild: Merge samba-simpleserver into samba-nt4
This saves CI resources (20mins build time) at the expense of making
the samba-nt4 job take 45min (5 min longer).

The new maximum job time is 1:15 by samba-o3 on the different hosts.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Mon Mar 23 14:45:59 UTC 2020 on sn-devel-184
2020-03-23 14:45:59 +00:00
Andrew Bartlett
88c4305ffb selftest: Run smb2.compound_find against filesrever only, not nt4_dc and ad_dc
This is a slow test of SMB2 protocol features, it needs not to be repeated.  It only
got run against both by accident at the original merge and then extended to
run on the [compount_find] share (with smbd:find async delay usec = 10000)
but really does not need to be run three times.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2020-03-23 13:02:32 +00:00
Andrew Bartlett
7a44bd1c42 autobuild: Merge the samba-ktest-heimdal and samba-fileserver jobs
This avoids a full compile of Samba just to test Kerberos with a system Heimdal
while still providing an environment to test other fileserver features.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2020-03-23 13:02:32 +00:00
Andrew Bartlett
fb2918bb04 .gitlab-ci.yml: Move the short samba-xc job into "others"
This job is over in 8mins total and much of that is spent setting up.

It can safely be combined with the others in "others"

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2020-03-23 13:02:32 +00:00
Andrew Bartlett
607c9ab307 build: Require --without-ad-dc for --without-ads
Building an AD DC while setting --without-ads makes no sense and just wastes compile time on our build hosts.

To allow samba-nt4 to build --without-ad-dc we set rpc.spoolss.notify
(which is built on the NTVFS fileserver for the callbacks) to run in
the ad_member environment rather than nt4_dc and ad_dc.

This is also just more realistic in any case.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2020-03-23 13:02:31 +00:00
Jeremy Allison
4337c144bc s3: Remove tevent_wait code. The last user was fsp->deferred_close.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Mon Mar 23 12:06:45 UTC 2020 on sn-devel-184
2020-03-23 12:06:45 +00:00
Jeremy Allison
f6efda217c s3: VFS: Remove deferred_close from struct files_struct.
Bump up the VFS number to 43. Samba 4.13 will ship with that.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2020-03-23 09:40:45 +00:00
Jeremy Allison
76b9a3597c s3: smbd: Remove all references to fsp->deferred_close.
We are now free to remove it from struct files_struct.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2020-03-23 09:40:45 +00:00
Jeremy Allison
b686de4e7b s3: smbd: Convert async SMB2 close to use the wait_queue idiom.
Use the same mechanism to wait for aio on a handle used by
SMB1 reply_tdis(), reply_ulogoffX(), reply_exit(), reply_close()
and SMB2 tree disconnect.

This removes the last user of fsp->deferred_close, allowing
us to remove it.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2020-03-23 09:40:45 +00:00
Jeremy Allison
53ee28381e s3: smbd: Remove old async versions of SMB1 reply_close().
Use the common pattern to wait for aio.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2020-03-23 09:40:45 +00:00
Jeremy Allison
fef2054dd0 s3: smbd: Add async internals of reply_close().
Waits until all aio requests on the closing fsps
before returning to the client.

Slightly modified version of the existing async
reply_close code, updated to use the wait_queue
pattern standardized in reply_tdis, reply_ulogoffX
and reply_exit.

Done this way (commented out) so it is a clean
diff and it's clear what is being added.

The next commit will remove the old version.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2020-03-23 09:40:45 +00:00
Jeremy Allison
20290d02a0 s3: smbd: In asnyc do_smb1_close() use the utility function meant for async SMB1 replies.
Don't use the raw call.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2020-03-23 09:40:45 +00:00
Jeremy Allison
91cdfa3dbb s3: smbd: Update reply_close() to modern DBG_ coding style.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2020-03-23 09:40:45 +00:00
Jeremy Allison
a779f0a58b s3: smbd: In SMB1 reply_close() rename all uses of 'struct smb_request' to smb1req.
Will make further changes easier to see, as we now use
req for tevent_req structs.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2020-03-23 09:40:45 +00:00
Douglas Bagnall
c680daae6a idl/drsblobs: do not overwrite number of schedules == 1
If the struct has zero or two schedules, that is what it has, and we
should let that be.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Sun Mar 22 06:19:51 UTC 2020 on sn-devel-184
2020-03-22 06:19:51 +00:00
Andrew Bartlett
a4cdfbd167 dsdb: Allow delete (directly and over DRS) of an object with a link to itself
Previously this would fail with Unsupported critical extension 1.3.6.1.4.1.7165.4.3.2

Reported by Alexander Harm.  Many thanks for helping make Samba better
and for your patience with patches and providing debugging information.

REF: https://lists.samba.org/archive/samba/2020-February/228153.html
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14306

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2020-03-22 04:39:36 +00:00
Andrew Bartlett
ad750ed10f dsdb: Add test for the case of a link pointing back at its own object
This type of object was not possible to delete in Samba without first removing
the link.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14306

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2020-03-22 04:39:36 +00:00
Andrew Bartlett
b8ed1525d6 selftest: Add test for dangling backlink to ourself, a missing and a real object
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14306

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2020-03-22 04:39:36 +00:00
Andrew Bartlett
1f65f21169 selftest: Add test for dangling backlinks to objects that do not exist
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14306

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2020-03-22 04:39:36 +00:00
Gary Lockyer
83ff052777 ldb build: Remove some PEP8 warnings from wscript
Fix indentation of list members and fix lines > 79 characters to remove
PEP8 warnings.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2020-03-22 04:39:36 +00:00
Gary Lockyer
fc13304d15 ldb tests: Confirm lmdb free list handling
Add cmocka tests to confirm lmdb's handling of the free list.

As a result of lmdb's MVCC (Multiversion Concurrency Control) long
running read transactions or stale readers (read transactions where the
process exited without ending the transaction) can cause the database to
run out of space.

Items in the free list are only reused when they would not be visible in
a read transaction.  So long running read transactions prevent entries
in the free list being reused, and the database can run out of space.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2020-03-22 04:39:36 +00:00
Samuel Cabrero
9496e0523e s3:rpc_server: Remove dead code
Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Samuel Cabrero <scabrero@samba.org>
Autobuild-Date(master): Fri Mar 20 17:11:28 UTC 2020 on sn-devel-184
2020-03-20 17:11:28 +00:00
Samuel Cabrero
93224d4184 s3:rpc_server: Remove unused RPC module init and shutdown callbacks
The setup function registers the endpoint server and RPC core routines
initialize and shutdown it.

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2020-03-20 15:36:36 +00:00
Samuel Cabrero
e8e4ecfb8b s3:rpc_server: Do not generate and build s3 RPC server code
Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2020-03-20 15:36:36 +00:00
Samuel Cabrero
4c09839ec9 s3:rpc_server: Do not include s3 autogenerated headers
Prototype is generated by the server compat parser.

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2020-03-20 15:36:36 +00:00
Samuel Cabrero
f6d41e5b44 s3:rpc_server: Remove api_struct
Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2020-03-20 15:36:36 +00:00
Samuel Cabrero
d891c2d389 s3:rpc_server: Remove dead code and unused struct members
Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2020-03-20 15:36:36 +00:00
Samuel Cabrero
04172e966b s3:rpc_server: Remove s3 rpc server loop
Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2020-03-20 15:36:36 +00:00
Samuel Cabrero
c647c13a54 s3:rpc_server: Remove call to s3 rpc services shutdown code
The rpc_<service>_shutdown function unregisters the legacy api_struct
from the local dispatch table, which is not longer used as local
dispatching is done through dcesrv_interface and will be removed in
following commits.

The dcesrv_shutdown_registered_ep_servers will unregister endpoint servers.

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2020-03-20 15:36:35 +00:00