Image transformation filter for upscaled images

Image transformation filter for upscaled images

[Evgeny Zajcev]

[-- Attachment #1: Type: text/plain, Size: 475 bytes --]

Currently, image transformation for upscaled images uses NEAREST filter,
which is fast, but renders very bad results for images with text inside.

Maybe change it to GOOD, which is also fast, but renders more decent results

Here is I've got two screenshots showing the difference:

NEAREST (currently hardcoded in Emacs) -
http://lgarc.narod.ru/pics/upscaled-nearest.png

GOOD (my proposal) - http://lgarc.narod.ru/pics/upscaled-good.png

What do you think?

Thanks

-- 
lg

[-- Attachment #2: Type: text/html, Size: 882 bytes --]

Re: Image transformation filter for upscaled images

[Alan Third]

On Sun, Mar 07, 2021 at 10:12:43PM +0300, Evgeny Zajcev wrote:
> Currently, image transformation for upscaled images uses NEAREST filter,
> which is fast, but renders very bad results for images with text inside.
> 
> Maybe change it to GOOD, which is also fast, but renders more decent results
> 
> Here is I've got two screenshots showing the difference:
> 
> NEAREST (currently hardcoded in Emacs) -
> http://lgarc.narod.ru/pics/upscaled-nearest.png
> 
> GOOD (my proposal) - http://lgarc.narod.ru/pics/upscaled-good.png
> 
> What do you think?

See also bug#38394.

The reason nearest was chosen was because scaled up pixel art (emojis,
mostly, like etc/images/smilies/wry.xpm) looked abominable when using
the "best" filter, but most other types of images look OK when using
nearest. On the other hand the bug report complains that scaled up
pixel art looks abominable with nearest, so clearly there's a
difference of opinion.

I don't know whether "good" is a better compromise, I suspect it looks
quite like "best".

I don't know what the best option is, I suspect there's no clear
one-size-fits-all winning strategy.

-- 
Alan Third

Re: Image transformation filter for upscaled images

[Evgeny Zajcev]

[-- Attachment #1: Type: text/plain, Size: 1949 bytes --]

вс, 7 мар. 2021 г. в 23:46, Alan Third <alan@idiocy.org>:

> On Sun, Mar 07, 2021 at 10:12:43PM +0300, Evgeny Zajcev wrote:
> > Currently, image transformation for upscaled images uses NEAREST filter,
> > which is fast, but renders very bad results for images with text inside.
> >
> > Maybe change it to GOOD, which is also fast, but renders more decent
> results
> >
> > Here is I've got two screenshots showing the difference:
> >
> > NEAREST (currently hardcoded in Emacs) -
> > http://lgarc.narod.ru/pics/upscaled-nearest.png
> >
> > GOOD (my proposal) - http://lgarc.narod.ru/pics/upscaled-good.png
> >
> > What do you think?
>
> See also bug#38394.
>
> The reason nearest was chosen was because scaled up pixel art (emojis,
> mostly, like etc/images/smilies/wry.xpm) looked abominable


In 2k21 any xpm with any filter will look abominable :)

when using
> the "best" filter,



> but most other types of images look OK when using
> nearest.


We (telega.el project) see many images of different kind daily, and only
some of them looks OK when nearest filter is applied. Most of the time,
especially if some contrast diagonals are involved (including character
glyphs) upscaling results are no ok.  Also, doc-view suffers from this,
most pdf files viewed inside Emacs looks no ok.

On the other hand the bug report complains that scaled up
> pixel art looks abominable with nearest, so clearly there's a
> difference of opinion.
>

Just get rid of any xpm :) Emacs has nice support for SVG after all, we can
write xpm to svg converter to keep pixel art precision.

I don't know whether "good" is a better compromise, I suspect it looks
> quite like "best".
>
I don't know what the best option is, I suspect there's no clear
> one-size-fits-all winning strategy.
>

Or maybe we can make the transformation filter configurable, so it could be
changed from elisp side?

-- 
lg

[-- Attachment #2: Type: text/html, Size: 3373 bytes --]

Re: Image transformation filter for upscaled images

[Stefan Monnier]

> See also bug#38394.
>
> The reason nearest was chosen was because scaled up pixel art (emojis,
> mostly, like etc/images/smilies/wry.xpm) looked abominable when using
> the "best" filter, but most other types of images look OK when using
> nearest. On the other hand the bug report complains that scaled up
> pixel art looks abominable with nearest, so clearly there's a
> difference of opinion.
>
> I don't know whether "good" is a better compromise, I suspect it looks
> quite like "best".
>
> I don't know what the best option is, I suspect there's no clear
> one-size-fits-all winning strategy.

FWIW, I think there is no perfect solution, and I'm not sure it's worth
the trouble to give much control to the end user.  So I'd go for the
solution which minimizes the frequency of terrible results.

I'd vote for "good" (my experience is mostly with doc-view where I've
found the current "nearest" to be rather poor), but it's not
a strong vote.


        Stefan

Re: Image transformation filter for upscaled images

[Stefan Kangas]

Evgeny Zajcev <lg.zevlg@gmail.com> writes:

>> The reason nearest was chosen was because scaled up pixel art (emojis,
>> mostly, like etc/images/smilies/wry.xpm) looked abominable

Doesn't it seem like a bad trade-off to improve rendering of smileys at
the cost of rendering PDF:s worse?  I rarely if ever use doc-view, but
testing it now seems to produce less than stellar results (i.e. the text
is barely readable).

> Just get rid of any xpm :) Emacs has nice support for SVG after all, we can
> write xpm to svg converter to keep pixel art precision.

Is there a way to convert from XPM automatically, or are you saying you
volunteer to make the SVG:s?

Another option would perhaps be to just replace the XPM:s we have with
some set of emojis licensed under Creative Commons.  The first one I
found when searching online was this one:

    https://openmoji.org/
    https://github.com/hfg-gmuend/openmoji/blob/master/FAQ.md

Its license is Creative Commons Share Alike License 4.0 (CC BY-SA 4.0).
That is okay for distributing with Emacs, I think?

Or we could use some other set, I don't exactly care which one it is.
If there is more than one candidate perhaps we could distribute more
than one and add an option to choose between them.

Re: Image transformation filter for upscaled images

[Evgeny Zajcev]

[-- Attachment #1: Type: text/plain, Size: 1889 bytes --]

пн, 8 мар. 2021 г. в 02:27, Stefan Kangas <stefankangas@gmail.com>:

> Evgeny Zajcev <lg.zevlg@gmail.com> writes:
>
> >> The reason nearest was chosen was because scaled up pixel art (emojis,
> >> mostly, like etc/images/smilies/wry.xpm) looked abominable
>
> Doesn't it seem like a bad trade-off to improve rendering of smileys at
> the cost of rendering PDF:s worse?  I rarely if ever use doc-view, but
> testing it now seems to produce less than stellar results (i.e. the text
> is barely readable).
>
> > Just get rid of any xpm :) Emacs has nice support for SVG after all, we
> can
> > write xpm to svg converter to keep pixel art precision.
>
> Is there a way to convert from XPM automatically, or are you saying you
> volunteer to make the SVG:s?
>

I can write the converter of course, however, there are already fonts with
colored emojis with "SIL Open Font License" that might be used to get SVG
emojis.  Is that license suitable?  Or keeping pixel art for old xpm's is
crucial?

Nowadays having emojis is easy as evaluating (set-fontset-font t 'symbol
(font-spec :family "<font family with emojis>")) sexp, probably old
packages (that uses emoji xpms) could be updated to use unicode emojis
instead?


> Another option would perhaps be to just replace the XPM:s we have with
> some set of emojis licensed under Creative Commons.  The first one I
> found when searching online was this one:
>
>     https://openmoji.org/
>     https://github.com/hfg-gmuend/openmoji/blob/master/FAQ.md
>
> Its license is Creative Commons Share Alike License 4.0 (CC BY-SA 4.0).
> That is okay for distributing with Emacs, I think?
>
> Or we could use some other set, I don't exactly care which one it is.
> If there is more than one candidate perhaps we could distribute more
> than one and add an option to choose between them.
>


-- 
lg

[-- Attachment #2: Type: text/html, Size: 2808 bytes --]

Re: Image transformation filter for upscaled images

[Stefan Kangas]

Evgeny Zajcev <lg.zevlg@gmail.com> writes:

>> Is there a way to convert from XPM automatically, or are you saying you
>> volunteer to make the SVG:s?
>
> I can write the converter of course, however, there are already fonts with
> colored emojis with "SIL Open Font License" that might be used to get SVG
> emojis.  Is that license suitable?  Or keeping pixel art for old xpm's is
> crucial?

Why not both?  IOW, why not begin by converting the XPM files to SVG,
and then look into if we could use fonts instead?  I think the SVG
conversion would be useful in any case.

IIUC, there should be no issue distributing fonts with Emacs for free
operating systems but there might be some problems on some proprietary
ones.  If this is an insurmountable obstacle, those operating systems
could just continue using the SVG, I suppose.

And even if we can't distribute fonts with Emacs, perhaps we could
detect when these fonts are installed and prefer them when they are?

The SIL Open Font License 1.1 is fine:
https://www.gnu.org/licenses/license-list.html#SILOFL

Re: Image transformation filter for upscaled images

[Eli Zaretskii]

> From: Evgeny Zajcev <lg.zevlg@gmail.com>
> Date: Mon, 8 Mar 2021 01:35:50 +0300
> 
> Or maybe we can make the transformation filter configurable, so it could be changed from elisp side?

That is probably the best way out of this conundrum.

Re: Image transformation filter for upscaled images

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Sun, 7 Mar 2021 17:27:44 -0600
> 
> Another option would perhaps be to just replace the XPM:s we have with
> some set of emojis licensed under Creative Commons.

The disadvantage of using emoji is that you cannot have an icon for
which there's no character in the Unicode database.  I don't think
it's wise for us to assume any icon we could ever want to show will
always be available as an emoticon character.

Re: Image transformation filter for upscaled images

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Sun, 7 Mar 2021 20:31:40 -0600
> Cc: Alan Third <alan@idiocy.org>, emacs-devel <emacs-devel@gnu.org>
> 
> IIUC, there should be no issue distributing fonts with Emacs for free
> operating systems but there might be some problems on some proprietary
> ones.

There's no technical problem with this, but installing a private font
is not trivial on all systems, as you need to perform supporting
steps, like update the font caches etc.  I'd rather we avoided that.

Also see my other message about the disadvantages of using emoji.

Re: Image transformation filter for upscaled images

[Alan Third]

On Sun, Mar 07, 2021 at 05:27:44PM -0600, Stefan Kangas wrote:
> Evgeny Zajcev <lg.zevlg@gmail.com> writes:
> 
> >> The reason nearest was chosen was because scaled up pixel art (emojis,
> >> mostly, like etc/images/smilies/wry.xpm) looked abominable
> 
> Doesn't it seem like a bad trade-off to improve rendering of smileys at
> the cost of rendering PDF:s worse?  I rarely if ever use doc-view, but
> testing it now seems to produce less than stellar results (i.e. the text
> is barely readable).

This is the first I've heard of any complaints about the rendering of
PDFs, and I don't view them in Emacs. Certainly nobody brought it up
when the change was implemented, so the trade-off wasn't considered.

I don't feel strongly about this. Can someone try the "best" filter
and see if good is an improvement over it? We use best for scaling
down, so if we're happy with best for scaling up then we can just
remove the code that sets it completely. Or go with good across the
board, but best is, y'know, better. ;)

Remove the ternary operator in this line (image.c:2372), and probably
the one just below on line 2378 as well:

          XRenderSetPictureFilter (FRAME_X_DISPLAY (f), img->mask_picture,
                                   scale_down ? FilterBest : FilterNearest, 0, 0);

> > Just get rid of any xpm :) Emacs has nice support for SVG after all, we can
> > write xpm to svg converter to keep pixel art precision.
> 
> Is there a way to convert from XPM automatically, or are you saying you
> volunteer to make the SVG:s?

There are only 12 emoji XPMs, so trying to copy them directly into SVG
would probably not take too much effort... They might not look very
good, though. ;)

-- 
Alan Third

Re: Image transformation filter for upscaled images

[Lars Ingebrigtsen]

Alan Third <alan@idiocy.org> writes:

> I don't feel strongly about this. Can someone try the "best" filter
> and see if good is an improvement over it? We use best for scaling
> down, so if we're happy with best for scaling up then we can just
> remove the code that sets it completely. Or go with good across the
> board, but best is, y'know, better. ;)

The problem is that icons that are scaled up look like fuzzy messes when
scaled up with "best", I think?  But scaling up text does indeed look
kinda bad when using the current method.

Text images are usually large, while icons are usually small, so we
could use some heuristic for switching between the methods.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Image transformation filter for upscaled images

[Evgeny Zajcev]

[-- Attachment #1: Type: text/plain, Size: 1498 bytes --]

пн, 8 мар. 2021 г. в 21:58, Alan Third <alan@idiocy.org>:

> On Sun, Mar 07, 2021 at 05:27:44PM -0600, Stefan Kangas wrote:
> > Evgeny Zajcev <lg.zevlg@gmail.com> writes:
> >
> > >> The reason nearest was chosen was because scaled up pixel art (emojis,
> > >> mostly, like etc/images/smilies/wry.xpm) looked abominable
> >
> > Doesn't it seem like a bad trade-off to improve rendering of smileys at
> > the cost of rendering PDF:s worse?  I rarely if ever use doc-view, but
> > testing it now seems to produce less than stellar results (i.e. the text
> > is barely readable).
>
> This is the first I've heard of any complaints about the rendering of
> PDFs, and I don't view them in Emacs. Certainly nobody brought it up
> when the change was implemented, so the trade-off wasn't considered.
>
> I don't feel strongly about this. Can someone try the "best" filter
> and see if good is an improvement over it? We use best for scaling
> down, so if we're happy with best for scaling up then we can just
> remove the code that sets it completely. Or go with good across the
> board, but best is, y'know, better. ;)
>

BEST looks even better then good -
http://lgarc.narod.ru/pics/upscaled-best.png

comparing to http://lgarc.narod.ru/pics/upscaled-good.png and
http://lgarc.narod.ru/pics/upscaled-nearest.png

Is it possible to access image type at the filter application time? in that
case we can apply NEAREST for small xpm files and BEST otherwise

-- 
lg

[-- Attachment #2: Type: text/html, Size: 2278 bytes --]

Re: Image transformation filter for upscaled images

[Alan Third]

On Mon, Mar 08, 2021 at 08:18:21PM +0100, Lars Ingebrigtsen wrote:
> Alan Third <alan@idiocy.org> writes:
> 
> > I don't feel strongly about this. Can someone try the "best" filter
> > and see if good is an improvement over it? We use best for scaling
> > down, so if we're happy with best for scaling up then we can just
> > remove the code that sets it completely. Or go with good across the
> > board, but best is, y'know, better. ;)
> 
> The problem is that icons that are scaled up look like fuzzy messes when
> scaled up with "best", I think?  But scaling up text does indeed look
> kinda bad when using the current method.

Indeed, and I doubt they look much better using good, either.

> Text images are usually large, while icons are usually small, so we
> could use some heuristic for switching between the methods.

I don't know, I'd be inclined to go with adding an image option, like
":filter smooth" or ":filter nearest" and letting mode authors choose
what they think is best.

Although I can see that that won't help with the likes of eww where
you just don't know what type of image you're loading.

(Also, I think :filter may be taken, so perhaps just ":smooth t/nil"
or something...)
-- 
Alan Third

Re: Image transformation filter for upscaled images

[Alan Third]

On Mon, Mar 08, 2021 at 11:05:29PM +0300, Evgeny Zajcev wrote:
> 
> BEST looks even better then good -
> http://lgarc.narod.ru/pics/upscaled-best.png

I suppose we shouldn't really be surprised. ;)

Good to know though. Thanks for checking.

> Is it possible to access image type at the filter application time? in that
> case we can apply NEAREST for small xpm files and BEST otherwise

I think we'd need to do it for pbms and pngs as well. The list may
grow endlessly. I prefer the other idea of providing an option to
whoever is loading the image.
-- 
Alan Third

Re: Image transformation filter for upscaled images

[Lars Ingebrigtsen]

Alan Third <alan@idiocy.org> writes:

> I don't know, I'd be inclined to go with adding an image option, like
> ":filter smooth" or ":filter nearest" and letting mode authors choose
> what they think is best.

Yes, that's an excellent idea.

> Although I can see that that won't help with the likes of eww where
> you just don't know what type of image you're loading.

Well, it'd be up to the application to add heuristics (if any).

> (Also, I think :filter may be taken, so perhaps just ":smooth t/nil"
> or something...)

Sure.  Or...  :scaling-algorithm...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Image transformation filter for upscaled images

[Alan Third]

On Mon, Mar 08, 2021 at 10:31:16PM +0100, Lars Ingebrigtsen wrote:
> Alan Third <alan@idiocy.org> writes:
> 
> > I don't know, I'd be inclined to go with adding an image option, like
> > ":filter smooth" or ":filter nearest" and letting mode authors choose
> > what they think is best.
> 
> Yes, that's an excellent idea.

Not my idea, but certainly a good one. :)

> > (Also, I think :filter may be taken, so perhaps just ":smooth t/nil"
> > or something...)
> 
> Sure.  Or...  :scaling-algorithm...

I was wrong, :filter is available, I'll have a bit more of a think
about what we want to call it.

As for default behaviour, I'm inclined to stick with what we have
where scaling down uses best and scaling up uses nearest neighbour but
any use of :filter (or whatever) over-rides the default. Anyone have
an opinion? Do we just want to use best everywhere by default?
-- 
Alan Third

Re: Image transformation filter for upscaled images

[Evgeny Zajcev]

[-- Attachment #1: Type: text/plain, Size: 1091 bytes --]

вт, 9 мар. 2021 г. в 00:59, Alan Third <alan@idiocy.org>:

> On Mon, Mar 08, 2021 at 10:31:16PM +0100, Lars Ingebrigtsen wrote:
> > Alan Third <alan@idiocy.org> writes:
> >
> > > I don't know, I'd be inclined to go with adding an image option, like
> > > ":filter smooth" or ":filter nearest" and letting mode authors choose
> > > what they think is best.
> >
> > Yes, that's an excellent idea.
>
> Not my idea, but certainly a good one. :)
>
> > > (Also, I think :filter may be taken, so perhaps just ":smooth t/nil"
> > > or something...)
> >
> > Sure.  Or...  :scaling-algorithm...
>
> I was wrong, :filter is available, I'll have a bit more of a think
> about what we want to call it.
>
> As for default behaviour, I'm inclined to stick with what we have
> where scaling down uses best and scaling up uses nearest neighbour but
> any use of :filter (or whatever) over-rides the default. Anyone have
> an opinion? Do we just want to use best everywhere by default?
>

I vote for "best" everywhere by default, making doc-view shine out-of-box

-- 
lg

[-- Attachment #2: Type: text/html, Size: 1693 bytes --]

Re: Image transformation filter for upscaled images

[Alan Third]

On Tue, Mar 09, 2021 at 11:34:28AM +0300, Evgeny Zajcev wrote:
> вт, 9 мар. 2021 г. в 00:59, Alan Third <alan@idiocy.org>:
> 
> > As for default behaviour, I'm inclined to stick with what we have
> > where scaling down uses best and scaling up uses nearest neighbour but
> > any use of :filter (or whatever) over-rides the default. Anyone have
> > an opinion? Do we just want to use best everywhere by default?
> >
> 
> I vote for "best" everywhere by default, making doc-view shine out-of-box

This might be a stupid question, but would it not make more sense to
teach doc-view to generate images that are the correct resolution
already? That would make it look much better than any scaling option.
-- 
Alan Third

Re: Image transformation filter for upscaled images

[Evgeny Zajcev]

[-- Attachment #1: Type: text/plain, Size: 1151 bytes --]

вт, 9 мар. 2021 г. в 12:28, Alan Third <alan@idiocy.org>:

> On Tue, Mar 09, 2021 at 11:34:28AM +0300, Evgeny Zajcev wrote:
> > вт, 9 мар. 2021 г. в 00:59, Alan Third <alan@idiocy.org>:
> >
> > > As for default behaviour, I'm inclined to stick with what we have
> > > where scaling down uses best and scaling up uses nearest neighbour but
> > > any use of :filter (or whatever) over-rides the default. Anyone have
> > > an opinion? Do we just want to use best everywhere by default?
> > >
> >
> > I vote for "best" everywhere by default, making doc-view shine out-of-box
>
> This might be a stupid question, but would it not make more sense to
> teach doc-view to generate images that are the correct resolution
> already? That would make it look much better than any scaling option.
>

This will involve many tricks to be added, doc-view uses image-mode to view
the image, and user can change (enlarge) window size at any moment, so
doc-view must add some handlings for that to regenerate/redisplay images.
Also using larger images makes things slower, then just applying scale
filter at display time.

-- 
lg

[-- Attachment #2: Type: text/html, Size: 1670 bytes --]

Re: Image transformation filter for upscaled images

[Eli Zaretskii]

> From: Evgeny Zajcev <lg.zevlg@gmail.com>
> Date: Tue, 9 Mar 2021 11:34:28 +0300
> 
> I vote for "best" everywhere by default, making doc-view shine out-of-box

IME, doing this will make sure we will have this discussion again in
the near future, when someone will report that "best" is not really
the best in some use case dear to them.

I don't understand why we are still arguing instead of adding a
variable or an attribute to control this?  From where I stand, it's a
no-brainer.

Re: Image transformation filter for upscaled images

[Lars Ingebrigtsen]

Evgeny Zajcev <lg.zevlg@gmail.com> writes:

> I vote for "best" everywhere by default, making doc-view shine out-of-box

doc-view can call create-image with :filter 'best, so what's best for
doc-view is pretty much irrelevant as to what the default should be.

The current default works pretty well for most images (just not on
text), so I think it makes sense to not change the default.

(And, as Alan pointed out, doc-view really should be smarter about this
stuff, and render the PDF in the size it's going to be displayed at.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Image transformation filter for upscaled images

[Evgeny Zajcev]

[-- Attachment #1: Type: text/plain, Size: 678 bytes --]

вт, 9 мар. 2021 г. в 16:09, Eli Zaretskii <eliz@gnu.org>:

> > From: Evgeny Zajcev <lg.zevlg@gmail.com>
> > Date: Tue, 9 Mar 2021 11:34:28 +0300
> >
> > I vote for "best" everywhere by default, making doc-view shine out-of-box
>
> IME, doing this will make sure we will have this discussion again in
> the near future, when someone will report that "best" is not really
> the best in some use case dear to them.
>
> I don't understand why we are still arguing instead of adding a
> variable or an attribute to control this?  From where I stand, it's a
> no-brainer.
>

Yeah, any default value is ok if we have control over the scaling filter.

-- 
lg

[-- Attachment #2: Type: text/html, Size: 1175 bytes --]

Re: Image transformation filter for upscaled images

[Dov Grobgeld]

[-- Attachment #1: Type: text/plain, Size: 1070 bytes --]

Note that there are various smart algorithms for bitmap to contour (svg)
tracing. E.g.

http://potrace.sourceforge.net/

But there are others. One idea would be to use potrace for "small" images
and pixel interpolation (or nearest neighbor) for large images. The
specification for small and large should be settable by the user.


On Tue, Mar 9, 2021 at 3:30 PM Lars Ingebrigtsen <larsi@gnus.org> wrote:

> Evgeny Zajcev <lg.zevlg@gmail.com> writes:
>
> > I vote for "best" everywhere by default, making doc-view shine out-of-box
>
> doc-view can call create-image with :filter 'best, so what's best for
> doc-view is pretty much irrelevant as to what the default should be.
>
> The current default works pretty well for most images (just not on
> text), so I think it makes sense to not change the default.
>
> (And, as Alan pointed out, doc-view really should be smarter about this
> stuff, and render the PDF in the size it's going to be displayed at.)
>
> --
> (domestic pets only, the antidote for overdose, milk.)
>    bloggy blog: http://lars.ingebrigtsen.no
>
>

[-- Attachment #2: Type: text/html, Size: 2107 bytes --]

Re: Image transformation filter for upscaled images

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> I don't understand why we are still arguing instead of adding a
> variable or an attribute to control this?  From where I stand, it's a
> no-brainer.

There's no single value that will give good results in all cases.
Adding a variable to control the default is all well and good, but it's
not sufficient.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Image transformation filter for upscaled images

[Stefan Monnier]

> This might be a stupid question, but would it not make more sense to
> teach doc-view to generate images that are the correct resolution
> already? That would make it look much better than any scaling option.

It does already, in most circumstances.  There are cases where we use
scaling, tho [ can't remember when or why, sorry.  ]


        Stefan

Re: Image transformation filter for upscaled images

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Tue, 09 Mar 2021 14:59:04 +0100
> Cc: alan@idiocy.org, Evgeny Zajcev <lg.zevlg@gmail.com>, stefankangas@gmail.com,
>  emacs-devel@gnu.org
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I don't understand why we are still arguing instead of adding a
> > variable or an attribute to control this?  From where I stand, it's a
> > no-brainer.
> 
> There's no single value that will give good results in all cases.
> Adding a variable to control the default is all well and good, but it's
> not sufficient.

Neither a variable nor an image attribute?  (I suggested both.)  Then
what would you suggest to do to resolve this?

Re: Image transformation filter for upscaled images

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> There's no single value that will give good results in all cases.
>> Adding a variable to control the default is all well and good, but it's
>> not sufficient.
>
> Neither a variable nor an image attribute?  (I suggested both.)  Then
> what would you suggest to do to resolve this?

Sorry; missed that you also mentioned an image attribute, which would
be fine.

Thinking a bit more about this, I wonder whether a general solution is
kinda-sorta possible anyway.  That is, having a function that returns
what algorithm to use.  We normally want shrinking the image to always
use `best', and enlarging an image slightly to use `good' or `best'
(that's the issue with doc-view, I think? we're rendering at one size
and then enlarging it on HiDPI screens?  I haven't actually debugged
it), and enlarging more than (say) 3x in size to use nearest
neighbour...

Having a variable (that can be a function that returns the algo to
use, or be one of the symbols) would allow us to experiment with this
more easily and see whether we can come up with something that works for
most people out of the box.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Image transformation filter for upscaled images

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: alan@idiocy.org,  lg.zevlg@gmail.com,  stefankangas@gmail.com,
>   emacs-devel@gnu.org
> Date: Tue, 09 Mar 2021 18:04:59 +0100
> 
> Thinking a bit more about this, I wonder whether a general solution is
> kinda-sorta possible anyway.  That is, having a function that returns
> what algorithm to use.  We normally want shrinking the image to always
> use `best', and enlarging an image slightly to use `good' or `best'
> (that's the issue with doc-view, I think? we're rendering at one size
> and then enlarging it on HiDPI screens?  I haven't actually debugged
> it), and enlarging more than (say) 3x in size to use nearest
> neighbour...
> 
> Having a variable (that can be a function that returns the algo to
> use, or be one of the symbols) would allow us to experiment with this
> more easily and see whether we can come up with something that works for
> most people out of the box.

If we can DTRT in most cases by default, we should, of course.  I was
under the impression that the "best" choice depended on what people
did with images most of the time and on personal preferences.

Re: Image transformation filter for upscaled images

[Alan Third]

[-- Attachment #1: Type: text/plain, Size: 887 bytes --]

On Tue, Mar 09, 2021 at 02:16:43PM +0100, Lars Ingebrigtsen wrote:
> Evgeny Zajcev <lg.zevlg@gmail.com> writes:
> 
> > I vote for "best" everywhere by default, making doc-view shine out-of-box
> 
> doc-view can call create-image with :filter 'best, so what's best for
> doc-view is pretty much irrelevant as to what the default should be.
> 
> The current default works pretty well for most images (just not on
> text), so I think it makes sense to not change the default.

Attached is a patch that should allow us to set smoothing using
":transform-smoothing t", or nil to disable it and use nearest
neighbour. Also sm in image mode lets you set it directly, and
doc-view will always use it.

I'm not sure about the term "smoothing", but it seemed better than
filter or similar. I dunno.

Anyway, it needs a news entry and documentation written, but I'll do
that later.

-- 
Alan Third

[-- Attachment #2: 0001-Enable-selectable-image-smoothing-bug-38394.patch --]
[-- Type: text/plain, Size: 6800 bytes --]

From 62e13bfaa703bb1b19a3c170ca13a7cfe544c97b Mon Sep 17 00:00:00 2001
From: Alan Third <alan@idiocy.org>
Date: Tue, 9 Mar 2021 18:05:10 +0000
Subject: [PATCH] Enable selectable image smoothing (bug#38394)

* lisp/doc-view.el (doc-view-insert-image): Always use smoothing in
docview.
* lisp/image-mode.el (image-transform-smoothing): New variable.
(image-mode-map): Add smoothing binding.
(image-transform-properties): Apply smoothing when requested.
(image-transform-set-smoothing): New function.
(image-transform-reset): Reset smoothing.
* src/image.c (image_set_transform): Use new :transform-smoothing
attribute.
(syms_of_image): Add :transform-smoothing attribute.
---
 lisp/doc-view.el   |  2 ++
 lisp/image-mode.el | 20 ++++++++++++++++++--
 src/image.c        | 16 +++++++++++-----
 3 files changed, 31 insertions(+), 7 deletions(-)

diff --git a/lisp/doc-view.el b/lisp/doc-view.el
index f6fcfae453..cef09009d9 100644
--- a/lisp/doc-view.el
+++ b/lisp/doc-view.el
@@ -1439,6 +1439,8 @@ doc-view-insert-image
 			    (apply #'create-image file doc-view--image-type nil args)
 			  (unless (member :width args)
 			    (setq args `(,@args :width ,doc-view-image-width)))
+                          (unless (member :transform-smoothing args)
+                            (setq args `(,@args :transform-smoothing t)))
 			  (apply #'create-image file doc-view--image-type nil args))))
 	     (slice (doc-view-current-slice))
 	     (img-width (and image (car (image-size image))))
diff --git a/lisp/image-mode.el b/lisp/image-mode.el
index 7384abf3b2..ba3f104f1e 100644
--- a/lisp/image-mode.el
+++ b/lisp/image-mode.el
@@ -95,6 +95,9 @@ image-transform-scale
 (defvar-local image-transform-rotation 0.0
   "Rotation angle for the image in the current Image mode buffer.")
 
+(defvar-local image-transform-smoothing nil
+  "Whether to use transform smoothing.")
+
 (defvar image-transform-right-angle-fudge 0.0001
   "Snap distance to a multiple of a right angle.
 There's no deep theory behind the default value, it should just
@@ -457,6 +460,7 @@ image-mode-map
     (define-key map "sb" 'image-transform-fit-both)
     (define-key map "ss" 'image-transform-set-scale)
     (define-key map "sr" 'image-transform-set-rotation)
+    (define-key map "sm" 'image-transform-set-smoothing)
     (define-key map "so" 'image-transform-original)
     (define-key map "s0" 'image-transform-reset)
 
@@ -523,6 +527,8 @@ image-mode-map
 	 :help "Rotate the image"]
 	["Set Rotation..." image-transform-set-rotation
 	 :help "Set rotation angle of the image"]
+        ["Set Smoothing..." image-transform-set-smoothing
+        :help "Toggle smoothing"]
 	["Original Size" image-transform-original
 	 :help "Reset image to actual size"]
 	["Reset to Default Size" image-transform-reset
@@ -1474,7 +1480,10 @@ image-transform-properties
 	,@(when (cdr resized)
 	    (list :height (cdr resized)))
 	,@(unless (= 0.0 image-transform-rotation)
-	    (list :rotation image-transform-rotation))))))
+	    (list :rotation image-transform-rotation))
+        ,@(when image-transform-smoothing
+            (list :transform-smoothing
+                  (string= image-transform-smoothing "smooth")))))))
 
 (defun image-transform-set-scale (scale)
   "Prompt for a number, and resize the current image by that amount."
@@ -1507,6 +1516,12 @@ image-transform-set-rotation
   (setq image-transform-rotation (float (mod rotation 360)))
   (image-toggle-display-image))
 
+(defun image-transform-set-smoothing (smoothing)
+  (interactive (list (completing-read "Smoothing: "
+                                      '(none smooth) nil t)))
+  (setq image-transform-smoothing smoothing)
+  (image-toggle-display-image))
+
 (defun image-transform-original ()
   "Display the current image with the original (actual) size and rotation."
   (interactive)
@@ -1519,7 +1534,8 @@ image-transform-reset
   (interactive)
   (setq image-transform-resize image-auto-resize
 	image-transform-rotation 0.0
-	image-transform-scale 1)
+	image-transform-scale 1
+        image-transform-smoothing nil)
   (image-toggle-display-image))
 
 (provide 'image-mode)
diff --git a/src/image.c b/src/image.c
index 8137dbea8d..95ae573354 100644
--- a/src/image.c
+++ b/src/image.c
@@ -2230,7 +2230,12 @@ image_set_transform (struct frame *f, struct image *img)
      operations to use a blended filter, to avoid aliasing and the like.
 
      TODO: implement for Windows.  */
-  bool scale_down = (width < img->width) || (height < img->height);
+  bool smoothing;
+  Lisp_Object s = image_spec_value (img->spec, QCtransform_smoothing, NULL);
+  if (!s)
+    smoothing = (width < img->width) || (height < img->height);
+  else
+    smoothing = !NILP (s);
 # endif
 
   /* Perform scale transformation.  */
@@ -2344,13 +2349,13 @@ image_set_transform (struct frame *f, struct image *img)
   /* Under NS the transform is applied to the drawing surface at
      drawing time, so store it for later.  */
   ns_image_set_transform (img->pixmap, matrix);
-  ns_image_set_smoothing (img->pixmap, scale_down);
+  ns_image_set_smoothing (img->pixmap, smoothing);
 # elif defined USE_CAIRO
   cairo_matrix_t cr_matrix = {matrix[0][0], matrix[0][1], matrix[1][0],
 			      matrix[1][1], matrix[2][0], matrix[2][1]};
   cairo_pattern_t *pattern = cairo_pattern_create_rgb (0, 0, 0);
   cairo_pattern_set_matrix (pattern, &cr_matrix);
-  cairo_pattern_set_filter (pattern, scale_down
+  cairo_pattern_set_filter (pattern, smoothing
                             ? CAIRO_FILTER_BEST : CAIRO_FILTER_NEAREST);
   /* Dummy solid color pattern just to record pattern matrix.  */
   img->cr_data = pattern;
@@ -2369,13 +2374,13 @@ image_set_transform (struct frame *f, struct image *img)
              XDoubleToFixed (matrix[2][2])}}};
 
       XRenderSetPictureFilter (FRAME_X_DISPLAY (f), img->picture,
-                               scale_down ? FilterBest : FilterNearest, 0, 0);
+                               smoothing ? FilterBest : FilterNearest, 0, 0);
       XRenderSetPictureTransform (FRAME_X_DISPLAY (f), img->picture, &tmat);
 
       if (img->mask_picture)
         {
           XRenderSetPictureFilter (FRAME_X_DISPLAY (f), img->mask_picture,
-                                   scale_down ? FilterBest : FilterNearest, 0, 0);
+                                   smoothing ? FilterBest : FilterNearest, 0, 0);
           XRenderSetPictureTransform (FRAME_X_DISPLAY (f), img->mask_picture,
                                       &tmat);
         }
@@ -10693,6 +10698,7 @@ syms_of_image (void)
   DEFSYM (QCrotation, ":rotation");
   DEFSYM (QCmatrix, ":matrix");
   DEFSYM (QCscale, ":scale");
+  DEFSYM (QCtransform_smoothing, ":transform-smoothing");
   DEFSYM (QCcolor_adjustment, ":color-adjustment");
   DEFSYM (QCmask, ":mask");
 
-- 
2.29.2

Re: Image transformation filter for upscaled images

[Alan Third]

On Tue, Mar 09, 2021 at 03:50:08PM +0200, Dov Grobgeld wrote:
> Note that there are various smart algorithms for bitmap to contour (svg)
> tracing. E.g.
> 
> http://potrace.sourceforge.net/
> 
> But there are others. One idea would be to use potrace for "small" images
> and pixel interpolation (or nearest neighbor) for large images. The
> specification for small and large should be settable by the user.

Unfortunately this is when using the (hopefully) hardware accelerated
image transforms, so we're stuck with what the system gives us.
-- 
Alan Third

Re: Image transformation filter for upscaled images

[Basil L. Contovounesios]

Alan Third <alan@idiocy.org> writes:

> Attached is a patch that should allow us to set smoothing using
> ":transform-smoothing t", or nil to disable it and use nearest
> neighbour. Also sm in image mode lets you set it directly, and
> doc-view will always use it.

Thanks, just one nit/question from me:

> +(defun image-transform-set-smoothing (smoothing)
> +  (interactive (list (completing-read "Smoothing: "
> +                                      '(none smooth) nil t)))

I know that completing-read can handle a list of symbols, but its
documentation says only list of strings.  Who's right?  Do we care?

-- 
Basil

Re: Image transformation filter for upscaled images

[Stefan Monnier]

> I know that completing-read can handle a list of symbols, but its
> documentation says only list of strings.  Who's right?  Do we care?

IIRC what it officially supports are lists of strings and alists whose
keys are strings or symbols.

The way the code is implemented, it mostly works with lists of symbols
as well, tho expect breakage if your first symbol happens to be `lambda`
or `closure`.


        Stefan

Re: Image transformation filter for upscaled images

[Yuri Khan]

On Wed, 10 Mar 2021 at 02:32, Lars Ingebrigtsen <larsi@gnus.org> wrote:

> Thinking a bit more about this, I wonder whether a general solution is
> kinda-sorta possible anyway.  That is, having a function that returns
> what algorithm to use.  We normally want shrinking the image to always
> use `best', and enlarging an image slightly to use `good' or `best'
> (that's the issue with doc-view, I think? we're rendering at one size
> and then enlarging it on HiDPI screens?  I haven't actually debugged
> it), and enlarging more than (say) 3x in size to use nearest
> neighbour...

Nearest-neighbor scaling when enlarging by exactly 2x feels very good, too.

Re: Image transformation filter for upscaled images

[Alan Third]

On Tue, Mar 09, 2021 at 10:31:29PM +0000, Basil L. Contovounesios wrote:
> Alan Third <alan@idiocy.org> writes:
> 
> > +(defun image-transform-set-smoothing (smoothing)
> > +  (interactive (list (completing-read "Smoothing: "
> > +                                      '(none smooth) nil t)))
> 
> I know that completing-read can handle a list of symbols, but its
> documentation says only list of strings.  Who's right?  Do we care?

I've changed them to strings. Thanks.
-- 
Alan Third

Re: Image transformation filter for upscaled images

[Lars Ingebrigtsen]

Yuri Khan <yuri.v.khan@gmail.com> writes:

> Nearest-neighbor scaling when enlarging by exactly 2x feels very good, too.

Yup.

I guess all integer scaling might be better without blurring, even if
some things may look a bit odd.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Image transformation filter for upscaled images

[Lars Ingebrigtsen]

Alan Third <alan@idiocy.org> writes:

> Attached is a patch that should allow us to set smoothing using
> ":transform-smoothing t", or nil to disable it and use nearest
> neighbour. Also sm in image mode lets you set it directly, and
> doc-view will always use it.
>
> I'm not sure about the term "smoothing", but it seemed better than
> filter or similar. I dunno.

Sounds good to me, and the patch looks good to me, too.

We should probably introduce something like the defcustom I described
elsewhere in this thread, and use it in `create-image' by default, so
that downscaling images will still use smoothing.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Image transformation filter for upscaled images

[Alan Third]

On Wed, Mar 10, 2021 at 03:49:49PM +0100, Lars Ingebrigtsen wrote:
> Alan Third <alan@idiocy.org> writes:
> 
> > Attached is a patch that should allow us to set smoothing using
> > ":transform-smoothing t", or nil to disable it and use nearest
> > neighbour. Also sm in image mode lets you set it directly, and
> > doc-view will always use it.
> >
> > I'm not sure about the term "smoothing", but it seemed better than
> > filter or similar. I dunno.
> 
> Sounds good to me, and the patch looks good to me, too.

I've pushed it to master as c93447eac6f801d7ff97ed6dad368dc49d55cc46.

> We should probably introduce something like the defcustom I described
> elsewhere in this thread, and use it in `create-image' by default, so
> that downscaling images will still use smoothing.

I like this idea, but I felt somewhat out of my depth trying to
implement it.

-- 
Alan Third

Re: Image transformation filter for upscaled images

[Lars Ingebrigtsen]

Alan Third <alan@idiocy.org> writes:

>> We should probably introduce something like the defcustom I described
>> elsewhere in this thread, and use it in `create-image' by default, so
>> that downscaling images will still use smoothing.
>
> I like this idea, but I felt somewhat out of my depth trying to
> implement it.

OK; I'll take a stab at it.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Image transformation filter for upscaled images

[Lars Ingebrigtsen]

Alan Third <alan@idiocy.org> writes:

> I like this idea, but I felt somewhat out of my depth trying to
> implement it.

Now pushed to Emacs 28.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Image transformation filter for upscaled images

[Evgeny Zajcev]

[-- Attachment #1: Type: text/plain, Size: 338 bytes --]

пт, 12 мар. 2021 г. в 04:38, Lars Ingebrigtsen <larsi@gnus.org>:

> Alan Third <alan@idiocy.org> writes:
>
> > I like this idea, but I felt somewhat out of my depth trying to
> > implement it.
>
> Now pushed to Emacs 28.
>

Thanks, works like a charm!  Very useful feature, thanks all you guys for
the work!

-- 
lg

[-- Attachment #2: Type: text/html, Size: 784 bytes --]

Re: Image transformation filter for upscaled images

[Alan Third]

On Fri, Mar 12, 2021 at 02:38:33AM +0100, Lars Ingebrigtsen wrote:
> Alan Third <alan@idiocy.org> writes:
> 
> > I like this idea, but I felt somewhat out of my depth trying to
> > implement it.
> 
> Now pushed to Emacs 28.

Excellent. Thanks, Lars.
-- 
Alan Third