drm/format-helper: Make conversion_buf_size() support sub-byte pixel fmts

There are DRM fourcc formats that have pixels smaller than a byte, but the
conversion_buf_size() function assumes that pixels are a multiple of bytes
and use the struct drm_format_info .cpp field to calculate the dst_pitch.

Instead, calculate it by using the bits per pixel (bpp) and divide it by 8
to account for formats that have sub-byte pixels.

Signed-off-by: Javier Martinez Canillas <javierm@redhat.com>
Reviewed-by: Arthur Grillo <arthurgrillo@riseup.net>
Reviewed-by: Maíra Canal <mcanal@igalia.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230307215039.346863-1-javierm@redhat.com
This commit is contained in:
Javier Martinez Canillas 2023-03-07 22:50:39 +01:00
parent 96c25b0314
commit 3e853b9f89
No known key found for this signature in database
GPG Key ID: C751E590D63F3D69

View File

@ -409,12 +409,15 @@ static size_t conversion_buf_size(u32 dst_format, unsigned int dst_pitch,
const struct drm_rect *clip)
{
const struct drm_format_info *dst_fi = drm_format_info(dst_format);
unsigned int bpp;
if (!dst_fi)
return -EINVAL;
if (!dst_pitch)
dst_pitch = drm_rect_width(clip) * dst_fi->cpp[0];
if (!dst_pitch) {
bpp = drm_format_info_bpp(dst_fi, 0);
dst_pitch = DIV_ROUND_UP(drm_rect_width(clip) * bpp, 8);
}
return dst_pitch * drm_rect_height(clip);
}