Kitty Graphics Protocol - image scaling enhancements

[monsterkodi]

Hello,

here are two ideas how the image scaling of the graphics protocol could be even more versatile:

• add possibility to turn off the linear filtering.
• add possibility to specify bottom and right pixel offset.

reasoning:

One might want to highlight/overlay certain areas with a semi-transparent color.
The most efficient way to do this would be an image with just one pixel, scaled to the desired area.
Right now this doesn't work, since the linear filtering produces a kind of 'cushion' effect.

It is possible to specify pixel perfect top-left corner of the area via the X and Y parameters,
but it is not possible to specify an exact bottom-right corner (at least i don't know how).

It would be really nice, if one could specify the desired image size in pixels with W and H.

[kovidgoyal]

On Thu, Mar 20, 2025 at 09:02:21AM -0700, monsterkodi wrote: Hello, here are two ideas how the image scaling of the graphics protocol could be even more versatile: - add possibility to turn off the linear filtering. - add possibility to specify bottom and right pixel offset. reasoning: One might want to highlight/overlay certain areas with a semi-transparent color. The most efficient way to do this would be an image with just one pixel, scaled to the desired area. Right now this doesn't work, since the linear filtering produces a kind of 'cushion' effect.

You can instead generate an indexed PNG with a single color, it should compress very well and work for this purpose.

It is possible to specify pixel perfect top and left corners of the area via the X and Y parameters, but it is not possible to specify exact bottom and right corners (at least i don't know how).

Top, left and width, height give you exact bottom, right.

[monsterkodi]

Thanks for your fast reply!

I will try out the PNG idea. But i still think it would be nice to be able to opt out of the filtering ;)

Maybe I am stupid, but I don't think it is possible right now to simultaneously scale an image up and place it at an arbitrary pixel location. From what i understand, the only way to scale images up in size is by specifying c and r (cols and rows to span) and
when i do that, the lower right corner is always at a cell border, even when i offset the top left corner with X and Y.

Thanks in advance for any further clarification.

EDIT:

i think i get it now: your idea is to use a large sized png with just a single indexed color in it, so it compresses to something small and then crop that to the desired size by specifying the source rectangle. This could actually work, and i will try it out.

This somewhat overcomplicated solution has the disadvantage, that i can't easily control the color of the overlay.

[kovidgoyal]

On Thu, Mar 20, 2025 at 11:51:35AM -0700, monsterkodi wrote: Thanks for your fast reply! I will try out the PNG idea. But i still think it would be nice to be able to opt out of the filtering ;)

Nice sure, worth the overhead of modifying the protocol and then needing to have some way to detect it across terminal emulators, etc., no. As it is kitty is the only terminal emulator that implements the spec in its entirety, I'd rather not complicate it further for a niche use case. The use case is covered by indexed PNG fairly efficiently.

Maybe I am stupid, but I don't think it is possible right now to simultaneously scale an image up and place it at an arbitrary pixel location. From what i understand, the only way to scale images up in size is by specifying `c` and `r` (cols and rows to span) and when i do that, the lower right corner is always at a cell border, even when i offset the top left corner with `X` and `Y`.

Not in combination with scaling, no. Without scaling, absolutely.

[kovidgoyal]

Oh and incidentally, if you did want to implement this the answer is not scaling, its a new image format that would be even more efficient than indexed PNG for the usecase of creating an image composed of blocks of uniform colors. It could be something as simple as:
RGBA:x,y,w,h:RGBA:...

The first color would specify the default color of all pixels, then you have colon separated sections each specifying a rectangular area and the color of pixels in that area.

We would get feature detection for free since a terminal application can test if a terminal supports it by just sending a testing image with the f key for this format.

However, I am not convinced this represents a sufficient advantage over compressed, indexed PNG to be worth it.

[monsterkodi]

thanks a lot for taking the time to answer. it is really appreciated.

i totally understand that you don't want to complicate the protocol any further.
i also see why my use case might seem niche.

one last thought though :)

it seems to me that modern terminals use the graphics card to display their content,
wouldn't it be nice if one could just ask the terminal to display a triangle at a certain position relative to the cursor?

that would immediately satisfy my use case and essentially almost any other :)

[monsterkodi]

i just wanted to report that i got your suggestions to work!
thanks a lot for your input and the very nice terminal app!
here you can see my use case in action:

scroll.mov

[kovidgoyal]

You are welcome! And looks cool :)