mirror of
https://github.com/samba-team/samba.git
synced 2024-12-23 17:34:34 +03:00
8f8a9f0190
(This used to be commit 9f672c26d6
)
90 lines
4.4 KiB
XML
90 lines
4.4 KiB
XML
<?xml version="1.0" encoding="iso-8859-1"?>
|
|
<!DOCTYPE chapter PUBLIC "-//Samba-Team//DTD DocBook V4.2-Based Variant V1.0//EN" "http://www.samba.org/samba/DTD/samba-doc">
|
|
<chapter id="largefile">
|
|
<chapterinfo>
|
|
&author.jeremy;
|
|
&author.jht;
|
|
<pubdate>March 5, 2005</pubdate>
|
|
</chapterinfo>
|
|
<title>Handling Large Directories</title>
|
|
|
|
<para>
|
|
<indexterm><primary>performance degradation</primary></indexterm>
|
|
<indexterm><primary>large numbers of files</primary></indexterm>
|
|
<indexterm><primary>large directory</primary></indexterm>
|
|
Samba-3.0.12 and later implements a solution for sites that have experienced performance degradation due to the
|
|
problem of using Samba-3 with applications that need large numbers of files (100,000 or more) per directory.
|
|
</para>
|
|
|
|
<para>
|
|
<indexterm><primary>read directory into memory</primary></indexterm>
|
|
<indexterm><primary>strange delete semantics</primary></indexterm>
|
|
The key was fixing the directory handling to read only the current list requested instead of the old
|
|
(up to samba-3.0.11) behavior of reading the entire directory into memory before doling out names.
|
|
Normally this would have broken OS/2 applications, which have very strange delete semantics, but by
|
|
stealing logic from Samba4 (thanks, Tridge), the current code in 3.0.12 handles this correctly.
|
|
</para>
|
|
|
|
<para>
|
|
<indexterm><primary>large directory</primary></indexterm>
|
|
<indexterm><primary>performance</primary></indexterm>
|
|
To set up an application that needs large numbers of files per directory in a way that does not
|
|
damage performance unduly, follow these steps:
|
|
</para>
|
|
|
|
<para>
|
|
<indexterm><primary>canonicalize files</primary></indexterm>
|
|
First, you need to canonicalize all the files in the directory to have one case, upper or lower &smbmdash; take your
|
|
pick (I chose upper because all my files were already uppercase names). Then set up a new custom share for the
|
|
application as follows:
|
|
<smbconfblock>
|
|
<smbconfsection name="[bigshare]"/>
|
|
<smbconfoption name="path">/data/manyfilesdir</smbconfoption>
|
|
<smbconfoption name="read only">no</smbconfoption>
|
|
<smbconfoption name="case sensitive">True</smbconfoption>
|
|
<smbconfoption name="default case">upper</smbconfoption>
|
|
<smbconfoption name="preserve case">no</smbconfoption>
|
|
<smbconfoption name="short preserve case">no</smbconfoption>
|
|
</smbconfblock>
|
|
</para>
|
|
|
|
<para>
|
|
<indexterm><primary>case options</primary></indexterm>
|
|
<indexterm><primary>match case</primary></indexterm>
|
|
<indexterm><primary>uppercase</primary></indexterm>
|
|
Of course, use your own path and settings, but set the case options to match the case of all the files in your
|
|
directory. The path should point at the large directory needed for the application &smbmdash; any new files created in
|
|
there and in any paths under it will be forced by smbd into uppercase, but smbd will no longer have to scan
|
|
the directory for names: it knows that if a file does not exist in uppercase, then it doesn't exist at all.
|
|
</para>
|
|
|
|
<para>
|
|
<indexterm><primary>case-insensitive</primary></indexterm>
|
|
<indexterm><primary>consistent case</primary></indexterm>
|
|
<indexterm><primary>smbd</primary></indexterm>
|
|
The secret to this is really in the <smbconfoption name="case sensitive">True</smbconfoption>
|
|
line. This tells smbd never to scan for case-insensitive versions of names. So if an application asks for a file
|
|
called <filename>FOO</filename>, and it cannot be found by a simple stat call, then smbd will return file not
|
|
found immediately without scanning the containing directory for a version of a different case. The other
|
|
<filename>xxx case xxx</filename> lines make this work by forcing a consistent case on all files created by
|
|
&smbd;.
|
|
</para>
|
|
|
|
<para>
|
|
<indexterm><primary>uppercase</primary></indexterm>
|
|
<indexterm><primary>stanza</primary></indexterm>
|
|
<indexterm><primary>lowercase filenames</primary></indexterm>
|
|
Remember, all files and directories under the <parameter>path</parameter> directory must be in uppercase
|
|
with this &smb.conf; stanza because &smbd; will not be able to find lowercase filenames with these settings. Also
|
|
note that this is done on a per-share basis, allowing this parameter to be set only for a share servicing an application with
|
|
this problematic behavior (using large numbers of entries in a directory) &smbmdash; the rest of your &smbd; shares
|
|
don't need to be affected.
|
|
</para>
|
|
|
|
<para>
|
|
This makes smbd much faster when dealing with large directories. My test case has over 100,000 files, and
|
|
smbd now deals with this very efficiently.
|
|
</para>
|
|
|
|
</chapter>
|