IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
We return length + offset in page via *size. Don't bother - the caller
can do that arithmetics just as well; just report the length to it.
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
All callers can and should handle iov_iter_get_pages() returning
fewer pages than requested. All in-kernel ones do. And it makes
the arithmetical overflow analysis much simpler...
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
do what we do for iovec/kvec; that ends up generating better code,
AFAICS.
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: Christian Brauner (Microsoft) <brauner@kernel.org>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
we can do copyin/copyout under kmap_local_page(); it shouldn't overflow
the kmap stack - the maximal footprint increase only by one here.
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: Christian Brauner (Microsoft) <brauner@kernel.org>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
The functions copy_page_to_iter_pipe() and push_pipe() can both
allocate a new pipe_buffer, but the "flags" member initializer is
missing.
Fixes: 241699cd72 ("new iov_iter flavour: pipe-backed")
To: Alexander Viro <viro@zeniv.linux.org.uk>
To: linux-fsdevel@vger.kernel.org
To: linux-kernel@vger.kernel.org
Cc: stable@vger.kernel.org
Signed-off-by: Max Kellermann <max.kellermann@ionos.com>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Take advantage of how kmap_local_folio() works to simplify the loop.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: William Kucharski <william.kucharski@oracle.com>
Introduce a new nofault flag to indicate to iov_iter_get_pages not to
fault in user pages.
This is implemented by passing the FOLL_NOFAULT flag to get_user_pages,
which causes get_user_pages to fail when it would otherwise fault in a
page. We'll use the ->nofault flag to prevent iomap_dio_rw from faulting
in pages when page faults are not allowed.
Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
Introduce a new fault_in_iov_iter_writeable helper for safely faulting
in an iterator for writing. Uses get_user_pages() to fault in the pages
without actually writing to them, which would be destructive.
We'll use fault_in_iov_iter_writeable in gfs2 once we've determined that
the iterator passed to .read_iter isn't in memory.
Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
Turn iov_iter_fault_in_readable into a function that returns the number
of bytes not faulted in, similar to copy_to_user, instead of returning a
non-zero value when any of the requested pages couldn't be faulted in.
This supports the existing users that require all pages to be faulted in
as well as new users that are happy if any pages can be faulted in.
Rename iov_iter_fault_in_readable to fault_in_iov_iter_readable to make
sure this change doesn't silently break things.
Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
Turn fault_in_pages_{readable,writeable} into versions that return the
number of bytes not faulted in, similar to copy_to_user, instead of
returning a non-zero value when any of the requested pages couldn't be
faulted in. This supports the existing users that require all pages to
be faulted in as well as new users that are happy if any pages can be
faulted in.
Rename the functions to fault_in_{readable,writeable} to make sure
this change doesn't silently break things.
Neither of these functions is entirely trivial and it doesn't seem
useful to inline them, so move them to mm/gup.c.
Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
Both iov_iter_get_pages and iov_iter_get_pages_alloc return the number
of bytes of the iovec they could get the pages for. When they cannot
get any pages, they're supposed to return 0, but when the start of the
iovec isn't page aligned, the calculation goes wrong and they return a
negative value. Fix both functions.
In addition, change iov_iter_get_pages_alloc to return NULL in that case
to prevent resource leaks.
Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
In an ideal world, when someone is passed an iov_iter and returns X bytes,
then X bytes would have been consumed/advanced from the iov_iter. But we
have use cases that always consume the entire iterator, a few examples
of that are iomap and bdev O_DIRECT. This means we cannot rely on the
state of the iov_iter once we've called ->read_iter() or ->write_iter().
This would be easier if we didn't always have to deal with truncate of
the iov_iter, as rewinding would be trivial without that. We recently
added a commit to track the truncate state, but that grew the iov_iter
by 8 bytes and wasn't the best solution.
Implement a helper to save enough of the iov_iter state to sanely restore
it after we've called the read/write iterator helpers. This currently
only works for IOVEC/BVEC/KVEC as that's all we need, support for other
iterator types are left as an exercise for the reader.
Link: https://lore.kernel.org/linux-fsdevel/CAHk-=wiacKV4Gh-MYjteU0LwNBSGpWrK-Ov25HdqB1ewinrFPg@mail.gmail.com/
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Fix all kernel-doc warnings in lib/iov_iter.c:
lib/iov_iter.c:695: warning: Function parameter or member 'i' not described in '_copy_mc_to_iter'
lib/iov_iter.c:695: warning: Excess function parameter 'iter' description in '_copy_mc_to_iter'
lib/iov_iter.c:695: warning: No description found for return value of '_copy_mc_to_iter'
lib/iov_iter.c:758: warning: Function parameter or member 'i' not described in '_copy_from_iter_flushcache'
lib/iov_iter.c:758: warning: Excess function parameter 'iter' description in '_copy_from_iter_flushcache'
lib/iov_iter.c:758: warning: No description found for return value of '_copy_from_iter_flushcache'
Link: https://lkml.kernel.org/r/20210809051053.6531-1-rdunlap@infradead.org
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This warning was there to catch any architectures that still use
CONFIG_SET_FS, and that would mis-use iov_iter_init() for anything that
wasn't a proper user space pointer. So that
WARN_ON_ONCE(uaccess_kernel());
makes perfect conceptual sense: you really shouldn't use a kernel
pointer with set_fs(KERNEL_DS) and then pass it to iov_iter_init().
HOWEVER.
Guenter Roeck reports that this warning actually triggers in no-mmu
configurations of both ARM and m68k. And the reason isn't that they
pass in a kernel pointer under set_fs(KERNEL_DS) at all: the reason is
that in those configurations, "uaccess_kernel()" is simply not reliable.
Those no-mmu setups set USER_DS and KERNEL_DS to the same values, so you
can't test for the difference.
In particular, the no-mmu case for ARM does
#define USER_DS KERNEL_DS
#define uaccess_kernel() (true)
so USER_DS and KERNEL_DS have the same value, and uaccess_kernel() is
always trivially true.
The m68k case is slightly different and not quite as obvious. It does
(spread out over multiple header files just to be extra exciting:
asm/processor.h, asm/segment.h and asm-generic/uaccess.h):
#define TASK_SIZE (0xFFFFFFFFUL)
#define USER_DS MAKE_MM_SEG(TASK_SIZE)
#define KERNEL_DS MAKE_MM_SEG(~0UL)
#define get_fs() (current_thread_info()->addr_limit)
#define uaccess_kernel() (get_fs().seg == KERNEL_DS.seg)
but the end result is the same: uaccess_kernel() will always be true,
because USER_DS and KERNEL_DS end up having the same value, even if that
value is defined differently.
This is very arguably a misfeature in those implementations, but in the
end we don't really care. All modern architectures have gotten rid of
set_fs() already, and generic kernel code never uses it. And while the
sanity check was a nice idea, an architecture would have to go the extra
mile to actually break this.
So this well-intentioned warning isn't really all that likely to find
anything but these known false positives, and as such just isn't worth
maintaining.
Reported-by: Guenter Roeck <linux@roeck-us.net>
Fixes: 8cd54c1c84 ("iov_iter: separate direction from flavour")
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
1) kmap_atomic() is not needed here, kmap_local_page() is enough.
2) No need to make sum = csum_block_add(sum, next, off); conditional
upon next != 0 - adding 0 is a no-op as far as csum_block_add()
is concerned.
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
kmap_local_page() is enough there. Moreover, we can use _copy_to_iter()
for actual copying in those cases - no useful extra checks on the
address we are copying from in that call.
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
recalculating offset on each iteration is pointless - on all subsequent
passes through the loop it will be zero anyway.
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Iterator macros used to provide the arguments for step callbacks in
a structure matching the flavour - iovec for ITER_IOVEC, kvec for
ITER_KVEC and bio_vec for ITER_BVEC. That already broke down for
ITER_XARRAY (bio_vec there); now that we are using kvec callback
for bvec and xarray cases, we are always passing a pointer + length
(void __user * + size_t for ITER_IOVEC callback, void * + size_t
for everything else).
Note that the original reason for bio_vec (page + offset + len) in
case of ITER_BVEC used to be that we did *not* want to kmap a
page when all we wanted was e.g. to find the alignment of its
subrange. Now all such users are gone and the ones that are left
want the page mapped anyway for actually copying the data.
So in all cases we have pointer + length, and there's no good
reason for keeping those in struct iovec or struct kvec - we
can just pass them to callback separately.
Again, less boilerplate in callbacks...
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Making iterator macros keep track of the amount of data copied is pretty
easy and it has several benefits:
1) we no longer need the mess like (from += v.iov_len) - v.iov_len
in the callbacks - initial value + total amount copied so far would do
just fine.
2) less obviously, we no longer need to remember the initial amount
of data we wanted to copy; the loops in iterator macros are along the lines
of
wanted = bytes;
while (bytes) {
copy some
bytes -= copied
if short copy
break
}
bytes = wanted - bytes;
Replacement is
offs = 0;
while (bytes) {
copy some
offs += copied
bytes -= copied
if short copy
break
}
bytes = offs;
That wouldn't be a win per se, but unlike the initial value of bytes, the amount
copied so far *is* useful in callbacks.
3) in some cases (csum_and_copy_..._iter()) we already had offs manually
maintained by the callbacks. With that change we can drop that.
Less boilerplate and more readable code...
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
After the previous commit we have
* xarray and bvec callbacks idential in all cases
* both equivalent to kvec callback wrapped into
kmap_local_page()/kunmap_local() pair.
So we can pass only two (iovec and kvec) callbacks to
iterate_and_advance() and let iterate_{bvec,xarray} wrap
it into kmap_local_page()/kunmap_local_page().
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
... and now we finally can sort out the mess in _copy_mc_to_iter().
Provide a variant of iterate_and_advance() that does *NOT* ignore
the return values of bvec, xarray and kvec callbacks, use that in
_copy_mc_to_iter(). That gets rid of magic in those callbacks -
we used to need it so we'd get at least the right return value in
case of failure halfway through.
As a bonus, now iterator is advanced by the amount actually copied
for all flavours. That's what the callers expect and it used to do that
correctly in iovec and xarray cases. However, in kvec and bvec cases
the iterator had not been advanced on such failures, breaking the users.
Fixed now...
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
... incidentally, using pointer instead of index in an array
(the only change here) trims half-kilobyte of .text...
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
The differences between iterate_iovec and iterate_kvec are minor:
* kvec callback is treated as if it returned 0
* initialization of __p is with i->iov and i->kvec resp.
which is trivially dealt with.
No code generation changes - compiler is quite capable of turning
left = ((void)(STEP), 0);
__v.iov_len -= left;
(with no accesses to left downstream) and
(void)(STEP);
into the same code.
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Premature optimization is the root of all evil... Trying
to unroll the first pass through the loop makes it harder
to follow and not just for readers - compiler ends up
generating worse code than it would on a "non-optimized"
loop.
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
iov_iter_advance() needs to do some non-trivial work when it's given
0 as argument (skip all empty iovecs, mostly). We used to implement
it via iterate_and_advance(); we no longer do so and for all other
users of iterate_and_advance() zero length is a no-op.
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Namely, have off counted starting from 0 rather than from csstate->off.
To compensate we need to shift the initial value (csstate->sum) (rotate
by 8 bits, as usual for csum) and do the same after we are finished adding
the pieces up.
What we get out of that is a bit more redundancy in our variables - from
is always equal to addr + off, which will be useful several commits down
the road.
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Replacement is called copy_page_from_iter_atomic(); unlike the old primitive the
callers do *not* need to do iov_iter_advance() after it. In case when they end
up consuming less than they'd been given they need to do iov_iter_revert() on
everything they had not consumed. That, however, needs to be done only on slow
paths.
All in-tree callers converted. And that kills the last user of iterate_all_kinds()
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
note that in bvec case pages can be compound ones - we can't just assume
that each segment is covered by one (sub)page
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Here iterate_all_kinds() is used just to find the first (non-empty, in
case of iovec) segment. Which can be easily done explicitly.
Note that in bvec case we now can get more than PAGE_SIZE worth of them,
in case when we have a compound page in bvec and a range that crosses
a subpage boundary. Older behaviour had been to stop on that boundary;
we used to get the right first page (for_each_bvec() took care of that),
but that was all we'd got.
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
For one thing, it's only used for iovec (and makes sense only for those).
For another, here we don't care about iov_offset, since the beginning of
the first segment and the end of the last one are ignored. So it makes
a lot more sense to just walk through the iovec array...
We need to deal with the case of truncated iov_iter, but unlike the
situation with iov_iter_alignment() we don't care where the last
segment ends - just which segment is the last one.
[fixed a braino spotted by Qian Cai <quic_qiancai@quicinc.com>]
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
It's easier to go over the array manually. We need to watch out
for truncated iov_iter, though - iovec array might cover more
than i->count.
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
1) constify iov_iter argument; we are not advancing it in this primitive.
2) cap the amount requested by the amount of data in iov_iter. All
existing callers should've been safe, but the check is really cheap and
doing it here makes for easier analysis, as well as more consistent
semantics among the primitives.
3) don't bother with iterate_iovec(). Explicit loop is not any harder
to follow, and we get rid of standalone iterate_iovec() users - it's
only used by iterate_and_advance() and (soon to be gone) iterate_all_kinds().
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
We can do better than generic iterate_and_advance() for this one;
inspired by bvec_iter_advance() (and massaged into that form by
equivalent transformations).
[fixed a braino caught by kernel test robot <oliver.sang@intel.com>]
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Instead of having them mixed in iter->type, use separate ->iter_type
and ->data_source (u8 and bool resp.) And don't bother with (pseudo-)
bitmap for the former - microoptimizations from being able to check
if the flavour is one of two values are not worth the confusion for
optimizer. It can't prove that we never get e.g. ITER_IOVEC | ITER_PIPE,
so we end up with extra headache.
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
iovec is the most common one; test it first and test explicitly,
rather than "not anything else". Replace all flavour checks with
use of iov_iter_is_...() helpers.
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Use corresponding plain variants, revert on short copy. That's the way it
should've been done from the very beginning, except that we didn't have
iov_iter_revert() back then...
[fixed another braino caught by Qian Cai <quic_qiancai@quicinc.com>]
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
asking to advance by more than we have left in the iov_iter should
move to the very end; it should *not* leave negative i->count and
it should not spew into syslog, etc. - it's a legitimate operation.
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
... and actually should just check it's given an iovec-backed iterator
in the first place.
Cc: stable@vger.kernel.org
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
In situation when copy_page_to_iter() got a compound page the current
code would only work on systems with no CONFIG_HIGHMEM. It *is* the majority
of real-world setups, or we would've drown in bug reports by now. Still needs
fixing.
Current variant works for solitary page; rename that to
__copy_page_to_iter() and turn the handling of compound pages into a loop over
subpages.
Cc: stable@vger.kernel.org
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Remove iov_iter_for_each_range() as it's no longer used with the removal of
lustre.
Signed-off-by: David Howells <dhowells@redhat.com>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
