Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 15 additions & 8 deletions sparse_strips/vello_bench/src/fine/image.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ use vello_common::kurbo::Affine;
use vello_common::paint::{Image, ImageSource};
use vello_common::peniko;
use vello_common::peniko::ImageQuality;
use vello_common::peniko::ImageSampler;
use vello_common::pixmap::Pixmap;
use vello_cpu::fine::{Fine, FineKernel};

Expand Down Expand Up @@ -148,10 +149,13 @@ fn get_colr_image(extend: peniko::Extend, quality: ImageQuality) -> Image {

let pixmap = Pixmap::from_png(&data[..]).unwrap();
Image {
source: ImageSource::Pixmap(Arc::new(pixmap)),
x_extend: extend,
y_extend: extend,
quality,
image: ImageSource::Pixmap(Arc::new(pixmap)),
sampler: ImageSampler {
x_extend: extend,
y_extend: extend,
quality,
alpha: 1.0,
},
}
}

Expand All @@ -160,10 +164,13 @@ fn get_small_image(extend: peniko::Extend, quality: ImageQuality) -> Image {

let pixmap = Pixmap::from_png(&data[..]).unwrap();
Image {
source: ImageSource::Pixmap(Arc::new(pixmap)),
x_extend: extend,
y_extend: extend,
quality,
image: ImageSource::Pixmap(Arc::new(pixmap)),
sampler: ImageSampler {
x_extend: extend,
y_extend: extend,
quality,
alpha: 1.0,
},
}
}

Expand Down
28 changes: 15 additions & 13 deletions sparse_strips/vello_common/src/encode.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@ use core::hash::{Hash, Hasher};
use fearless_simd::{Simd, SimdBase, SimdFloat, f32x4, f32x16};
use peniko::color::cache_key::{BitEq, BitHash, CacheKey};
use peniko::{
InterpolationAlphaSpace, LinearGradientPosition, RadialGradientPosition, SweepGradientPosition,
ImageSampler, InterpolationAlphaSpace, LinearGradientPosition, RadialGradientPosition,
SweepGradientPosition,
};
use smallvec::ToSmallVec;
// So we can just use `OnceCell` regardless of which feature is activated.
Expand Down Expand Up @@ -430,7 +431,12 @@ impl EncodeExt for Image {
fn encode_into(&self, paints: &mut Vec<EncodedPaint>, transform: Affine) -> Paint {
let idx = paints.len();

let mut quality = self.quality;
let mut sampler = self.sampler;

if sampler.alpha != 1.0 {
// If the sampler alpha is not 1.0, we need to force alpha compositing.
unimplemented!("Applying opacity to image commands");
}
Comment on lines +436 to +439
Copy link
Contributor

@LaurenzV LaurenzV Oct 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't have to be addressed in this PR, but I think it would be smarter to apply the opacity during encoding time instead of at sampling time (using Arc::make_mut(pixmap))? I'm concerned that adding an if branch for that for sampling will be pretty bad for performance, especially when doing bilinear/bicubic filtering.

I also thought we already support image alphas, but seems like I remember wrong...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And I think using unimplented is fine for now, but before the first minor release we should probably replace all panics with a warning instead.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't have to be addressed in this PR, but I think it would be smarter to apply the opacity during encoding time instead of at sampling time (using Arc::make_mut(pixmap))? I'm concerned that adding an if branch for that for sampling will be pretty bad for performance, especially when doing bilinear/bicubic filtering.

I am not sure this would work, for cases where one wants to put same image with different alpha (this is currently possible in vello classic and is expressed as alpha in ImageSampler).

And I think using unimplented is fine for now, but before the first minor release we should probably replace all panics with a warning instead.

Yes, that's how we already have other stuff, so I just continued the pattern.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure this would work, for cases where one wants to put same image with different alpha (this is currently possible in vello classic and is expressed as alpha in ImageSampler).

Well, in this case can just do the encoding twice? I think that applying an opacity to an image is already pretty rare, placing the same image twice with different alphas is even more rare. I don't think it's worth sacrificing rendering performance for all images just to account for those special cases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure this would work, for cases where one wants to put same image with different alpha (this is currently possible in vello classic and is expressed as alpha in ImageSampler).

Well, in this case can just do the encoding twice? I think that applying an opacity to an image is already pretty rare, placing the same image twice with different alphas is even more rare. I don't think it's worth sacrificing rendering performance for all images just to account for those special cases.

I wonder if we can do this in creation of image sampler (ImagePainterData) using generics rather than doing in sampling (which is hot). This way only do "dispatch" once.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory yes, but the main problem is that we already have lots of generics in vello-cpu which are already impacting compile times (and probably build size) pretty badly. :/ So I would be careful about adding even more, especially since each additional generic serves as a multiplying factor.


let c = transform.as_coeffs();

Expand All @@ -441,9 +447,9 @@ impl EncodeExt for Image {
&& (c[3] as f32 - 1.0).is_nearly_zero()
&& ((c[4] - c[4].floor()) as f32).is_nearly_zero()
&& ((c[5] - c[5].floor()) as f32).is_nearly_zero()
&& quality == ImageQuality::Medium
&& sampler.quality == ImageQuality::Medium
{
quality = ImageQuality::Low;
sampler.quality = ImageQuality::Low;
}

// Similarly to gradients, apply a 0.5 offset so we sample at the center of
Expand All @@ -452,12 +458,11 @@ impl EncodeExt for Image {

let (x_advance, y_advance) = x_y_advances(&transform);

let encoded = match &self.source {
let encoded = match &self.image {
ImageSource::Pixmap(pixmap) => {
EncodedImage {
source: ImageSource::Pixmap(pixmap.clone()),
extends: (self.x_extend, self.y_extend),
quality,
sampler,
// While we could optimize RGB8 images, it's probably not worth the trouble.
has_opacities: true,
transform,
Expand All @@ -467,8 +472,7 @@ impl EncodeExt for Image {
}
ImageSource::OpaqueId(image) => EncodedImage {
source: ImageSource::OpaqueId(*image),
extends: (self.x_extend, self.y_extend),
quality: self.quality,
sampler,
has_opacities: true,
transform,
x_advance,
Expand Down Expand Up @@ -510,10 +514,8 @@ impl From<EncodedBlurredRoundedRectangle> for EncodedPaint {
pub struct EncodedImage {
/// The underlying pixmap of the image.
pub source: ImageSource,
/// The extends in the horizontal and vertical direction.
pub extends: (Extend, Extend),
/// The rendering quality of the image.
pub quality: ImageQuality,
/// Sampler
pub sampler: ImageSampler,
/// Whether the image has opacities.
pub has_opacities: bool,
/// A transform to apply to the image.
Expand Down
114 changes: 36 additions & 78 deletions sparse_strips/vello_common/src/paint.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
use crate::pixmap::Pixmap;
use alloc::sync::Arc;
use peniko::{
Gradient, ImageQuality, ImageSampler,
Gradient,
color::{AlphaColor, PremulRgba8, Srgb},
};

Expand Down Expand Up @@ -77,84 +77,68 @@ pub enum ImageSource {
OpaqueId(ImageId),
}

/// An image.
#[derive(Debug, Clone)]
pub struct Image {
/// The underlying pixmap of the image.
pub source: ImageSource,
/// Extend mode in the horizontal direction.
pub x_extend: peniko::Extend,
/// Extend mode in the vertical direction.
pub y_extend: peniko::Extend,
/// Hint for desired rendering quality.
pub quality: ImageQuality,
}

impl Image {
/// Convert a [`peniko::ImageBrush`] to an [`Image`].
impl ImageSource {
/// Convert a [`peniko::ImageData`] to an [`ImageSource`].
///
/// This is a somewhat lossy conversion, as the image data data is transformed to
/// [premultiplied RGBA8](`PremulRgba8`).
///
/// # Panics
///
/// This panics if `image` has a `width` or `height` greater than `u16::MAX`.
pub fn from_peniko_image(brush: &peniko::ImageBrush) -> Self {
pub fn from_peniko_image_data(image: &peniko::ImageData) -> Self {
// TODO: how do we deal with `peniko::ImageFormat` growing? See also
// <https://github.com/linebender/vello/pull/996#discussion_r2080510863>.
if brush.image.format != peniko::ImageFormat::Rgba8 {
unimplemented!("Unsupported image format: {:?}", brush.image.format);
}
if brush.image.alpha_type != peniko::ImageAlphaType::Alpha {
unimplemented!("Unsupported image alpha type: {:?}", brush.image.alpha_type);
}
let do_alpha_multiply = image.alpha_type != peniko::ImageAlphaType::AlphaPremultiplied;

assert!(
brush.image.width <= u16::MAX as u32 && brush.image.height <= u16::MAX as u32,
image.width <= u16::MAX as u32 && image.height <= u16::MAX as u32,
"The image is too big. Its width and height can be no larger than {} pixels.",
u16::MAX,
);
let width = brush.image.width.try_into().unwrap();
let height = brush.image.height.try_into().unwrap();
let ImageSampler {
x_extend,
y_extend,
quality,
alpha: global_alpha,
} = brush.sampler;

#[expect(clippy::cast_possible_truncation, reason = "deliberate quantization")]
let global_alpha = u16::from((global_alpha * 255. + 0.5) as u8);
let width = image.width.try_into().unwrap();
let height = image.height.try_into().unwrap();

// TODO: SIMD
#[expect(clippy::cast_possible_truncation, reason = "This cannot overflow.")]
let pixels = brush
.image
let pixels = image
.data
.data()
.chunks_exact(4)
.map(|rgba| {
let alpha = ((u16::from(rgba[3]) * global_alpha) / 255) as u8;
let multiply = |component| ((u16::from(alpha) * u16::from(component)) / 255) as u8;
PremulRgba8 {
r: multiply(rgba[0]),
g: multiply(rgba[1]),
b: multiply(rgba[2]),
a: alpha,
.map(|pixel| {
let rgba: [u8; 4] = match image.format {
peniko::ImageFormat::Rgba8 => pixel.try_into().unwrap(),
peniko::ImageFormat::Bgra8 => [pixel[2], pixel[1], pixel[0], pixel[3]],
format => unimplemented!("Unsupported image format: {format:?}"),
};
let alpha = u16::from(rgba[3]);
let multiply = |component| ((alpha * u16::from(component)) / 255) as u8;
if do_alpha_multiply {
PremulRgba8 {
r: multiply(rgba[0]),
g: multiply(rgba[1]),
b: multiply(rgba[2]),
a: rgba[3],
}
} else {
PremulRgba8 {
r: rgba[0],
g: rgba[1],
b: rgba[2],
a: rgba[3],
}
Comment on lines +108 to +129
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @ajakubowicz-canva re: image premultiply/unpremultiply

}
})
.collect();
let pixmap = Pixmap::from_parts(pixels, width, height);

Self {
source: ImageSource::Pixmap(Arc::new(pixmap)),
x_extend,
y_extend,
quality,
}
Self::Pixmap(Arc::new(pixmap))
}
}

/// An image.
pub type Image = peniko::ImageBrush<ImageSource>;

/// A premultiplied color.
#[derive(Debug, Clone, PartialEq, Copy)]
pub struct PremulColor {
Expand Down Expand Up @@ -193,30 +177,4 @@ impl PremulColor {
}

/// A kind of paint that can be used for filling and stroking shapes.
#[derive(Debug, Clone)]
pub enum PaintType {
/// A solid color.
Solid(AlphaColor<Srgb>),
/// A gradient.
Gradient(Gradient),
/// An image.
Image(Image),
}

impl From<AlphaColor<Srgb>> for PaintType {
fn from(value: AlphaColor<Srgb>) -> Self {
Self::Solid(value)
}
}

impl From<Gradient> for PaintType {
fn from(value: Gradient) -> Self {
Self::Gradient(value)
}
}

impl From<Image> for PaintType {
fn from(value: Image) -> Self {
Self::Image(value)
}
}
pub type PaintType = peniko::Brush<Image, Gradient>;
14 changes: 7 additions & 7 deletions sparse_strips/vello_cpu/src/fine/common/image.rs
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ impl<'a, S: Simd> PlainNNImagePainter<'a, S> {
data.x_advances.1,
data.y_advances.1,
),
image.extends.1,
image.sampler.y_extend,
data.height,
data.height_inv,
);
Expand Down Expand Up @@ -68,7 +68,7 @@ impl<S: Simd> Iterator for PlainNNImagePainter<'_, S> {
let x_pos = extend(
self.simd,
self.cur_x_pos,
self.data.image.extends.0,
self.data.image.sampler.x_extend,
self.data.width,
self.data.width_inv,
);
Expand Down Expand Up @@ -116,7 +116,7 @@ impl<S: Simd> Iterator for NNImagePainter<'_, S> {
self.data.x_advances.0,
self.data.y_advances.0,
),
self.data.image.extends.0,
self.data.image.sampler.x_extend,
self.data.width,
self.data.width_inv,
);
Expand All @@ -129,7 +129,7 @@ impl<S: Simd> Iterator for NNImagePainter<'_, S> {
self.data.x_advances.1,
self.data.y_advances.1,
),
self.data.image.extends.1,
self.data.image.sampler.y_extend,
self.data.height,
self.data.height_inv,
);
Expand Down Expand Up @@ -214,7 +214,7 @@ impl<S: Simd> Iterator for FilteredImagePainter<'_, S> {
extend(
self.simd,
x_positions + $offsets[$idx],
self.data.image.extends.0,
self.data.image.sampler.y_extend,
self.data.width,
self.data.width_inv,
)
Expand All @@ -226,14 +226,14 @@ impl<S: Simd> Iterator for FilteredImagePainter<'_, S> {
extend(
self.simd,
y_positions + $offsets[$idx],
self.data.image.extends.1,
self.data.image.sampler.y_extend,
self.data.height,
self.data.height_inv,
)
};
}

match self.data.image.quality {
match self.data.image.sampler.quality {
ImageQuality::Low => unreachable!(),
ImageQuality::Medium => {
// <https://github.com/google/skia/blob/220738774f7a0ce4a6c7bd17519a336e5e5dea5b/src/opts/SkRasterPipeline_opts.h#L5039-L5078>
Expand Down
4 changes: 2 additions & 2 deletions sparse_strips/vello_cpu/src/fine/lowp/image.rs
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ impl<S: Simd> Iterator for BilinearImagePainter<'_, S> {
extend(
self.simd,
x_pos,
self.data.image.extends.0,
self.data.image.sampler.x_extend,
self.data.width,
self.data.width_inv,
)
Expand All @@ -70,7 +70,7 @@ impl<S: Simd> Iterator for BilinearImagePainter<'_, S> {
extend(
self.simd,
y_pos,
self.data.image.extends.1,
self.data.image.sampler.y_extend,
self.data.height,
self.data.height_inv,
)
Expand Down
2 changes: 1 addition & 1 deletion sparse_strips/vello_cpu/src/fine/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -573,7 +573,7 @@ impl<S: Simd, T: FineKernel<S>> Fine<S, T> {

match (i.has_skew(), i.nearest_neighbor()) {
(_, false) => {
if i.quality == ImageQuality::Medium {
if i.sampler.quality == ImageQuality::Medium {
fill_complex_paint!(
i.has_opacities,
T::medium_quality_image_painter(
Expand Down
Loading