"Fix" sag scaling for hidpi

"Fix" sag scaling for hidpi

[Yuan Fu]

[-- Attachment #1: Type: text/plain, Size: 614 bytes --]

To get crisp images on hidpi displays, a common approach is to load an image double de dimension and rescale it down. E.g., load an 200x200 image and rescale it to 100x100. In Emacs this can be done by

(insert-image (create-image "200x200img.png" 'png nil :scale 0.5))

And Emacs will display a crisp 100x100 image in the buffer. However, this doesn’t work for sag images. If I do

(progn
  (require 'svg)
  (let ((svg (svg-create 200 200)))
    (svg-text svg "test"
              :font-size 100
              :y 150)
    (insert-image (svg-image svg :scale 0.5))))

The image is still blurry:


[-- Attachment #2: Screen Shot 2021-02-04 at 3.43.05 PM.png --]
[-- Type: image/png, Size: 7332 bytes --]

[-- Attachment #3: Type: text/plain, Size: 206 bytes --]



Turns out image.c resolves the dimension for svg images before generating them and skips rescaling all together. The following patch changes that behavior and allows me to generate crisp svg images:


[-- Attachment #4: Screen Shot 2021-02-04 at 3.40.38 PM.png --]
[-- Type: image/png, Size: 9238 bytes --]

[-- Attachment #5: Type: text/plain, Size: 16 bytes --]



WDYT?

Yuan



[-- Attachment #6: svg-scale.patch --]
[-- Type: application/octet-stream, Size: 1483 bytes --]

From e64330c4dd1519424b9ab20b72071498dc1f5f9b Mon Sep 17 00:00:00 2001
From: Yuan Fu <casouri@gmail.com>
Date: Thu, 4 Feb 2021 15:30:38 -0500
Subject: [PATCH] demo

---
 src/image.c | 19 +++++++++----------
 1 file changed, 9 insertions(+), 10 deletions(-)

diff --git a/src/image.c b/src/image.c
index a124cf91ba..4b2504bb52 100644
--- a/src/image.c
+++ b/src/image.c
@@ -2203,16 +2203,7 @@ image_set_transform (struct frame *f, struct image *img)
   /* Determine size.  */
   int width, height;
 
-#ifdef HAVE_RSVG
-  /* SVGs are pre-scaled to the correct size.  */
-  if (EQ (image_spec_value (img->spec, QCtype, NULL), Qsvg))
-    {
-      width = img->width;
-      height = img->height;
-    }
-  else
-#endif
-    compute_image_size (img->width, img->height, img->spec, &width, &height);
+  compute_image_size (img->width, img->height, img->spec, &width, &height);
 
   /* Determine rotation.  */
   double rotation = 0.0;
@@ -10008,6 +9999,14 @@ svg_load_image (struct frame *f, struct image *img, char *contents,
   compute_image_size (viewbox_width, viewbox_height, img->spec,
                       &width, &height);
 
+  Lisp_Object value = image_spec_value (img->spec, QCscale, NULL);
+  if (NUMBERP (value))
+    {
+      double scale = XFLOATINT(value);
+      width /= scale;
+      height /= scale;
+    }
+
   if (! check_image_size (f, width, height))
     {
       image_size_error ();
-- 
2.24.3 (Apple Git-128)


[-- Attachment #7: Type: text/plain, Size: 2 bytes --]

Re: "Fix" sag scaling for hidpi

[Stefan Monnier]

> And Emacs will display a crisp 100x100 image in the buffer. However, this doesn’t work for sag images. If I do
>
> (progn
>   (require 'svg)
>   (let ((svg (svg-create 200 200)))
>     (svg-text svg "test"
>               :font-size 100
>               :y 150)
>     (insert-image (svg-image svg :scale 0.5))))
>
> The image is still blurry:

I think it's a bug, and that even

    (progn
      (require 'svg)
      (let ((svg (svg-create 100 100)))
        (svg-text svg "test"
                  :font-size 100
                  :y 150)
        (insert-image (svg-image svg :scale 1.0))))

should insert a crisp image.


        Stefan

Re: "Fix" sag scaling for hidpi

[Yuan Fu]

> On Feb 4, 2021, at 5:07 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> 
>> And Emacs will display a crisp 100x100 image in the buffer. However, this doesn’t work for sag images. If I do
>> 
>> (progn
>>  (require 'svg)
>>  (let ((svg (svg-create 200 200)))
>>    (svg-text svg "test"
>>              :font-size 100
>>              :y 150)
>>    (insert-image (svg-image svg :scale 0.5))))
>> 
>> The image is still blurry:
> 
> I think it's a bug, and that even
> 
>    (progn
>      (require 'svg)
>      (let ((svg (svg-create 100 100)))
>        (svg-text svg "test"
>                  :font-size 100
>                  :y 150)
>        (insert-image (svg-image svg :scale 1.0))))
> 
> should insert a crisp image.

Sorry, I should be more specific. This blurry problem only occurs on nsterm with hidpi screens. (That’s because cocoa reports logical pixel sizes instead of physical ones.)

Yuan

Re: "Fix" sag scaling for hidpi

[Lars Ingebrigtsen]

Yuan Fu <casouri@gmail.com> writes:

> Sorry, I should be more specific. This blurry problem only occurs on
> nsterm with hidpi screens. (That’s because cocoa reports logical pixel
> sizes instead of physical ones.)

Should this be fixed (somehow) by changing the cocoa dimensions to be
physical ones, so that Emacs is consistent across architectures?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: "Fix" sag scaling for hidpi

[Robert Pluim]

>>>>> On Fri, 05 Feb 2021 10:02:39 +0100, Lars Ingebrigtsen <larsi@gnus.org> said:

    Lars> Yuan Fu <casouri@gmail.com> writes:
    >> Sorry, I should be more specific. This blurry problem only occurs on
    >> nsterm with hidpi screens. (That’s because cocoa reports logical pixel
    >> sizes instead of physical ones.)

    Lars> Should this be fixed (somehow) by changing the cocoa dimensions to be
    Lars> physical ones, so that Emacs is consistent across architectures?

Thatʼs the wrong direction: pgtk is moving us towards logical pixel
sizes (gtk already uses them, but there are some rough edges).

Robert

Re: "Fix" sag scaling for hidpi

[Lars Ingebrigtsen]

Robert Pluim <rpluim@gmail.com> writes:

>     Lars> Should this be fixed (somehow) by changing the cocoa dimensions to be
>     Lars> physical ones, so that Emacs is consistent across architectures?
>
> Thatʼs the wrong direction: pgtk is moving us towards logical pixel
> sizes (gtk already uses them, but there are some rough edges).

Is there any realistic path we can take to change this in the
Windows/non-gtk cases, though?  We should have the same on all
architectures, and if we can't transition the old ones to logical pixel
sizes, then the new ones will have to use physical ones, too.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: "Fix" sag scaling for hidpi

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Sat, 06 Feb 2021 11:00:51 +0100
> Cc: Yuan Fu <casouri@gmail.com>, Stefan Monnier <monnier@iro.umontreal.ca>,
>  emacs-devel <emacs-devel@gnu.org>
> 
> > Thatʼs the wrong direction: pgtk is moving us towards logical pixel
> > sizes (gtk already uses them, but there are some rough edges).
> 
> Is there any realistic path we can take to change this in the
> Windows/non-gtk cases, though?  We should have the same on all
> architectures, and if we can't transition the old ones to logical pixel
> sizes, then the new ones will have to use physical ones, too.

If someone explains what are the "logical" and "physical" pixel sizes,
I could try figuring out what that means for the MS-Windows case.

Re: "Fix" sag scaling for hidpi

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> If someone explains what are the "logical" and "physical" pixel sizes,
> I could try figuring out what that means for the MS-Windows case.

Physical pixel size is what you'd think -- it's one actual pixel on the
screen.  (If the monitor is 1280 × 1024, you have (* 1280 1024) =>
1310720 physical pixels.)

"Logical" pixels was something that was introduced when people started
making high-resolution screens.  In a 2x HiDPI scenario, the OS will
report that the 1280 × 1024 is really a 640 x 512 screen, but if you
give it an image that's 640x512 to display, it'll scale it up, using
four physical pixels per logical pixel, and this will be blurry.

So you give it a 1280x1024 image if it's a logical 640x512 2x screen if
you don't want blurriness.  :-/

(Of course, with fonts you avoid the blurriness, because it'll just
render the fonts to whatever the physical pixels are.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: "Fix" sag scaling for hidpi

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: rpluim@gmail.com,  casouri@gmail.com,  monnier@iro.umontreal.ca,
>   emacs-devel@gnu.org
> Date: Sat, 06 Feb 2021 12:35:41 +0100
> 
> "Logical" pixels was something that was introduced when people started
> making high-resolution screens.  In a 2x HiDPI scenario, the OS will
> report that the 1280 × 1024 is really a 640 x 512 screen, but if you
> give it an image that's 640x512 to display, it'll scale it up, using
> four physical pixels per logical pixel, and this will be blurry.

Looks like this is called "device-independent pixels" (DIPs) on
MS-Windows?

Re: "Fix" sag scaling for hidpi

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Looks like this is called "device-independent pixels" (DIPs) on
> MS-Windows?

Hm...

https://en.wikipedia.org/wiki/Device-independent_pixel

Similar, but that sounds more well-defined than "logical pixels" -- the
pixel ratio can vary, so there's no standard for how big a logical pixel
is physically.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: "Fix" sag scaling for hidpi

[Robert Pluim]

>>>>> On Sat, 06 Feb 2021 12:35:41 +0100, Lars Ingebrigtsen <larsi@gnus.org> said:

    Lars> Eli Zaretskii <eliz@gnu.org> writes:
    >> If someone explains what are the "logical" and "physical" pixel sizes,
    >> I could try figuring out what that means for the MS-Windows case.

    Lars> Physical pixel size is what you'd think -- it's one actual pixel on the
    Lars> screen.  (If the monitor is 1280 × 1024, you have (* 1280 1024) =>
    Lars> 1310720 physical pixels.)

    Lars> "Logical" pixels was something that was introduced when people started
    Lars> making high-resolution screens.  In a 2x HiDPI scenario, the OS will
    Lars> report that the 1280 × 1024 is really a 640 x 512 screen, but if you
    Lars> give it an image that's 640x512 to display, it'll scale it up, using
    Lars> four physical pixels per logical pixel, and this will be blurry.

    Lars> So you give it a 1280x1024 image if it's a logical 640x512 2x screen if
    Lars> you don't want blurriness.  :-/

Or you use a scalable image format, or you donʼt specify the
dimensions, and let the toolkit work it out.

    Lars> (Of course, with fonts you avoid the blurriness, because it'll just
    Lars> render the fonts to whatever the physical pixels are.)

Font glyphs are just (small) scalable images :-)

Robert

Re: "Fix" sag scaling for hidpi

[Alan Third]

On Sat, Feb 06, 2021 at 12:35:41PM +0100, Lars Ingebrigtsen wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> "Logical" pixels was something that was introduced when people started
> making high-resolution screens.  In a 2x HiDPI scenario, the OS will
> report that the 1280 × 1024 is really a 640 x 512 screen, but if you
> give it an image that's 640x512 to display, it'll scale it up, using
> four physical pixels per logical pixel, and this will be blurry.
> 
> So you give it a 1280x1024 image if it's a logical 640x512 2x screen if
> you don't want blurriness.  :-/

One of the main problems I've had with trying to find a solution is to
work out what we want to actually happen.

If I open an image on an emacs frame it makes sense that the image
should be displayed 1:1 with the physical pixels no matter what the
scale factor is.

If that frame is then moved to another display with a different scale
factor do we then resize the image according to the change in scale
factor (so it's still 1:1 with the physical pixels), or keep it the
same (logical) size?

I think we probably want to keep it the same (logical) size, so I
think we need to, as described elsewhere, expose the scale factor to
lisp so that create-image can calculate :scale rather than trying to
calculate it on-the-fly in C code.

We'll probably have to do more fiddling with SVGs though, since they
can define sizes in real-world units, like cm or inches, so the dpi
has to match the physical pixels, and if we move to a different screen
and regenerate the SVG the DPI will be different but the scale likely
won't be recalculated and the image will change size.

Perhaps we need to let lisp set the DPI for an SVG (and other scalable
image types) as well as the scale.

And this is further complicated by the fact macOS uses a "fake" DPI
that has no relation to physical pixel size at all!

And, of course, none of this will help when the lisp code doesn't use
create-image.

I hope this makes sense, I find this hard to describe.
-- 
Alan Third

Re: "Fix" sag scaling for hidpi

[Lars Ingebrigtsen]

Alan Third <alan@idiocy.org> writes:

> I think we probably want to keep it the same (logical) size, so I
> think we need to, as described elsewhere, expose the scale factor to
> lisp so that create-image can calculate :scale rather than trying to
> calculate it on-the-fly in C code.

Yes, we need to know both how many physical pixels there are, and what
the logical pixels are.  

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: "Fix" sag scaling for hidpi

[Yuan Fu]

> 
> One of the main problems I've had with trying to find a solution is to
> work out what we want to actually happen.
> 
> If I open an image on an emacs frame it makes sense that the image
> should be displayed 1:1 with the physical pixels no matter what the
> scale factor is.
> 
> If that frame is then moved to another display with a different scale
> factor do we then resize the image according to the change in scale
> factor (so it's still 1:1 with the physical pixels), or keep it the
> same (logical) size?
> 
> I think we probably want to keep it the same (logical) size, so I
> think we need to, as described elsewhere, expose the scale factor to
> lisp so that create-image can calculate :scale rather than trying to
> calculate it on-the-fly in C code.

I agrees that we should keep the logical size, i.e., keep the size comparing against text. If we expose correct physical size, packages that generate bitmaps for display can generate crisp bitmaps with correct pixel size.

IIUC, a high-res image with :scale 0.5 should work across high and low-res displays. So ideally any package that wants to generate crisp bitmap can get the physical size and pixel-ratio from Emacs, generate the image and set :scale to 1/pixel-ratio. And this image works across different displays.

> 
> We'll probably have to do more fiddling with SVGs though, since they
> can define sizes in real-world units, like cm or inches, so the dpi
> has to match the physical pixels, and if we move to a different screen
> and regenerate the SVG the DPI will be different but the scale likely
> won't be recalculated and the image will change size.

To simplify things, maybe we can assume DPI is 96. I.e., assume 1 inch = 96 logical pixels = 96 * pixel-ratio physical pixels.

Obviously, if we can get DPI information from all terminals, then we could use that information. But from your previous message it doesn’t seem easy.

> 
> Perhaps we need to let lisp set the DPI for an SVG (and other scalable
> image types) as well as the scale.
> 
> And this is further complicated by the fact macOS uses a "fake" DPI
> that has no relation to physical pixel size at all!
> 
> And, of course, none of this will help when the lisp code doesn't use
> create-image.
> 
> I hope this makes sense, I find this hard to describe.
> -- 
> Alan Third

For SVGs I think we should automatically handle the pixel ratio and dpi so the image is always crisp and lisp doesn’t need to do anything (don’t need to add the :scale attribute or anything).

For bitmap images I think we display them in their physical size and let lisp alter the size by the :scale attribute.

For :width and :height attributes, I think they should be in logical pixels.Because the ratio of logical pixels and other text in a buffer doesn’t change when you drag a frame across different displays. So if the user set an image to have certain :width and drag the frame to a different display, the image doesn’t change its size comparing to everything else in the buffer.

Yuan

Re: "Fix" sag scaling for hidpi

[Yuan Fu]

ns should be able to report both physical and logical pixels, because cocoa reports logical pixels and pixel ratio.

pgtk can report both pixels.

nt should be able to report both? Eli mentioned that it has the concept of device independent pixels

I’m not sure about x11+gtk, any one knows if it can report both physical and logical pixels when on a hidpi display?

Yuan

Re: "Fix" sag scaling for hidpi

[Lars Ingebrigtsen]

Yuan Fu <casouri@gmail.com> writes:

> I’m not sure about x11+gtk, any one knows if it can report both
> physical and logical pixels when on a hidpi display?

For Gtk, we already do:

#if GTK_CHECK_VERSION (3, 22, 0)
      scale = gdk_monitor_get_scale_factor (monitor);
#elif defined HAVE_GTK3
      scale = gdk_screen_get_monitor_scale_factor (gscreen, i);
#endif

So we can use the same to compute logical pixels there.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: "Fix" sag scaling for hidpi

[Robert Pluim]

>>>>> On Wed, 10 Feb 2021 19:17:52 +0100, Lars Ingebrigtsen <larsi@gnus.org> said:

    Lars> Yuan Fu <casouri@gmail.com> writes:
    >> I’m not sure about x11+gtk, any one knows if it can report both
    >> physical and logical pixels when on a hidpi display?

    Lars> For Gtk, we already do:

    Lars> #if GTK_CHECK_VERSION (3, 22, 0)
    Lars>       scale = gdk_monitor_get_scale_factor (monitor);
    Lars> #elif defined HAVE_GTK3
    Lars>       scale = gdk_screen_get_monitor_scale_factor (gscreen, i);
    Lars> #endif

Iʼve not kept up: is that still integer factors only?

    Lars> So we can use the same to compute logical pixels there.

So now non-gtk on x11 will lag even further behind gtk/pgtk & other
platforms. [1]

Eli, Lars, when can we realistically envisage deprecating lucid
support? Followed a release later by the current gtk support.

Robert

Footnotes:
[1] Someone over on github referred to the current gtk support in
    Emacs as 'impure gtk', which I think is a great way to describe
    it.

Re: "Fix" sag scaling for hidpi

[Lars Ingebrigtsen]

Robert Pluim <rpluim@gmail.com> writes:

> Iʼve not kept up: is that still integer factors only?

No, floats.

>     Lars> So we can use the same to compute logical pixels there.
>
> So now non-gtk on x11 will lag even further behind gtk/pgtk & other
> platforms. [1]
>
> Eli, Lars, when can we realistically envisage deprecating lucid
> support? Followed a release later by the current gtk support.

I don't see any hurry -- if people choose to use a version of Emacs
with wonky scaling, that's up to them.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: "Fix" sag scaling for hidpi

[Basil L. Contovounesios]

Robert Pluim <rpluim@gmail.com> writes:

>>>>>> On Wed, 10 Feb 2021 19:17:52 +0100, Lars Ingebrigtsen <larsi@gnus.org> said:
>
>     Lars> So we can use the same to compute logical pixels there.
>
> So now non-gtk on x11 will lag even further behind gtk/pgtk & other
> platforms. [1]
>
> Eli, Lars, when can we realistically envisage deprecating lucid
> support? Followed a release later by the current gtk support.

I thought you'd never ask!  Who needs a GUI anyway.

-- 
Basil ;)

Re: "Fix" sag scaling for hidpi

[Robert Pluim]

>>>>> On Wed, 10 Feb 2021 18:52:09 +0000, "Basil L. Contovounesios" <contovob@tcd.ie> said:
    >> Eli, Lars, when can we realistically envisage deprecating lucid
    >> support? Followed a release later by the current gtk support.

    Basil> I thought you'd never ask!  Who needs a GUI anyway.

Levity aside, I do think it would be better to have *one* Gui toolkit
per platform. On Gnu/Linux we currently have 3.

Robert

Re: "Fix" sag scaling for hidpi

[Basil L. Contovounesios]

Robert Pluim <rpluim@gmail.com> writes:

>>>>>> On Wed, 10 Feb 2021 18:52:09 +0000, "Basil L. Contovounesios" <contovob@tcd.ie> said:
>     >> Eli, Lars, when can we realistically envisage deprecating lucid
>     >> support? Followed a release later by the current gtk support.
>
>     Basil> I thought you'd never ask!  Who needs a GUI anyway.
>
> Levity aside, I do think it would be better to have *one* Gui toolkit
> per platform. On Gnu/Linux we currently have 3.

Of course I'd love to see improvements in this area, and I don't mind
how many we end up with.  But the infamous impure GTK bug means that
there's only one on GNU/Linux for now, as far as I'm concerned: Lucid.

-- 
Basil

Re: "Fix" sag scaling for hidpi

[Robert Pluim]

>>>>> On Thu, 11 Feb 2021 14:30:37 +0000, "Basil L. Contovounesios" <contovob@tcd.ie> said:

    Basil> Robert Pluim <rpluim@gmail.com> writes:
    >>>>>>> On Wed, 10 Feb 2021 18:52:09 +0000, "Basil L. Contovounesios" <contovob@tcd.ie> said:
    >> >> Eli, Lars, when can we realistically envisage deprecating lucid
    >> >> support? Followed a release later by the current gtk support.
    >> 
    Basil> I thought you'd never ask!  Who needs a GUI anyway.
    >> 
    >> Levity aside, I do think it would be better to have *one* Gui toolkit
    >> per platform. On Gnu/Linux we currently have 3.

    Basil> Of course I'd love to see improvements in this area, and I don't mind
    Basil> how many we end up with.  But the infamous impure GTK bug means that
    Basil> there's only one on GNU/Linux for now, as far as I'm concerned: Lucid.

Actually, looking through configure, we also have motif and athena and
'none', so itʼs 6, not 3. I get your point about the GTK bug, but that
issue has been ignored by the GTK folks for at least 15 years.

(and before anyone pins their hopes on pure gtk: pgtk emacs using a
wayland display connection forwarded over ssh with waypipe has the
same failure mode)

Robert

Cleaning out old X11 toolkits?

[chad]

[-- Attachment #1: Type: text/plain, Size: 1021 bytes --]

Subject changed to try to start a new thread. Apologies in advance if my
(dumb) email client does the wrong thing.

On Thu, Feb 11, 2021 at 7:12 AM Robert Pluim <rpluim@gmail.com> wrote:

> Actually, looking through configure, we also have motif and athena and
> 'none', so itʼs 6, not 3.
>

I started a reply mentioning motif and athena/Xaw3d for hysterical reasons,
but then peeked ahead and saw that they had already been mentioned.

In more pragmatic terms, I would guess that it's entirely possible to
excise motif/lesstif, athena, and Xaw3d from main without anyone noticing.
Whether this is worth the effort in a world leaning ever so slowly towards
Cairo and pgtk is a little hard to tell, but a quick grep through src
suggests that it would at least clear up a bunch of #ifdef spaghetti.

Would the maintainers be interested in a branch that tried this? Would it
be better to wait for pgtk to settle first? Is there a big use-case for
those toolkits of which I'm unaware?

Thanks,
~Chad

[-- Attachment #2: Type: text/html, Size: 1439 bytes --]

Re: Cleaning out old X11 toolkits?

[Eli Zaretskii]

> From: chad <yandros@gmail.com>
> Date: Thu, 11 Feb 2021 14:18:27 -0800
> Cc: "Basil L. Contovounesios" <contovob@tcd.ie>, Yuan Fu <casouri@gmail.com>, Alan Third <alan@idiocy.org>, 
> 	emacs-devel <emacs-devel@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca>, 
> 	Lars Ingebrigtsen <larsi@gnus.org>, Eli Zaretskii <eliz@gnu.org>
> 
> In more pragmatic terms, I would guess that it's entirely possible to excise motif/lesstif, athena, and Xaw3d
> from main without anyone noticing. Whether this is worth the effort in a world leaning ever so slowly towards
> Cairo and pgtk is a little hard to tell, but a quick grep through src suggests that it would at least clear up a
> bunch of #ifdef spaghetti.
> 
> Would the maintainers be interested in a branch that tried this? Would it be better to wait for pgtk to settle
> first? Is there a big use-case for those toolkits of which I'm unaware?

I'm not sure I understand how a branch could help.  A branch is
generally used by significantly fewer people than the master branch.
A branch that doesn't bring any new user-visible features, and just
cleans up code, is unlikely to provide motivation for anyone to try
it.  So I'm afraid such a branch will just sit there unused, and will
not bring us any closer to a decision.

I think if we want to move towards removing those toolkits, we should
try a different approach.  Two ideas:

  . analyze the bug report in debbugs DB, and see how many builds are
    with any of these toolkits
  . look at GNU/Linux distros and see if they still provide builds
    with any of these toolkits, and what was the last version of Emacs
    when they did

We should also somehow analyze the usage of these toolkits on other
Posix platforms, although I'm not sure I know how -- do they offer
distros similar to GNU/Linux? if so, we could include them in the 2nd
item above.

Once we have an idea about the usage and popularity of each of these
toolkits, we could decide what changes are reasonable, and make them
on master.

Re: Cleaning out old X11 toolkits?

[Colin Baxter]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

    >> From: chad <yandros@gmail.com> Date: Thu, 11 Feb 2021 14:18:27
    >> -0800 Cc: "Basil L. Contovounesios" <contovob@tcd.ie>, Yuan Fu
    >> <casouri@gmail.com>, Alan Third <alan@idiocy.org>, emacs-devel
    >> <emacs-devel@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca>,
    >> Lars Ingebrigtsen <larsi@gnus.org>, Eli Zaretskii <eliz@gnu.org>
    >> 
    >> In more pragmatic terms, I would guess that it's entirely
    >> possible to excise motif/lesstif, athena, and Xaw3d from main
    >> without anyone noticing. Whether this is worth the effort in a
    >> world leaning ever so slowly towards Cairo and pgtk is a little
    >> hard to tell, but a quick grep through src suggests that it would
    >> at least clear up a bunch of #ifdef spaghetti.
    >> 
    >> Would the maintainers be interested in a branch that tried this?
    >> Would it be better to wait for pgtk to settle first? Is there a
    >> big use-case for those toolkits of which I'm unaware?

    > I'm not sure I understand how a branch could help.  A branch is
    > generally used by significantly fewer people than the master
    > branch.  A branch that doesn't bring any new user-visible
    > features, and just cleans up code, is unlikely to provide
    > motivation for anyone to try it.  So I'm afraid such a branch will
    > just sit there unused, and will not bring us any closer to a
    > decision.

    > I think if we want to move towards removing those toolkits, we
    > should try a different approach.  Two ideas:

    >   . analyze the bug report in debbugs DB, and see how many builds
    > are with any of these toolkits . look at GNU/Linux distros and see
    > if they still provide builds with any of these toolkits, and what
    > was the last version of Emacs when they did

    > We should also somehow analyze the usage of these toolkits on
    > other Posix platforms, although I'm not sure I know how -- do they
    > offer distros similar to GNU/Linux? if so, we could include them
    > in the 2nd item above.

    > Once we have an idea about the usage and popularity of each of
    > these toolkits, we could decide what changes are reasonable, and
    > make them on master.

I would like to put in a plea for the retention of lucid. I build both
lucid and GTK versions of emacs. The former consistently loads about a
third faster and for text applications, e.g. auctex, I prefer it. I
write this knowing that my preferences can be dismissed as personal and
anecdotal.
 
Best wishes,

Re: Cleaning out old X11 toolkits?

[Eli Zaretskii]

> From: Colin Baxter <m43cap@yandex.com>
> Cc: chad <yandros@gmail.com>,  emacs-devel@gnu.org
> Cc: 
> Date: Fri, 12 Feb 2021 08:44:16 +0000
> 
> I would like to put in a plea for the retention of lucid.

Removing Lucid is not on the table.  The toolkits proposed for removal
didn't include it.

Re: Cleaning out old X11 toolkits?

[Robert Pluim]

>>>>> On Fri, 12 Feb 2021 09:09:56 +0200, Eli Zaretskii <eliz@gnu.org> said:


    Eli>   . analyze the bug report in debbugs DB, and see how many builds are
    Eli>     with any of these toolkits

(turns out that athena == lucid, so weʼre down to 5)

Totally unscientific stats from debbugs mailboxes from Jan 2018
through to Sep 2020:

~/.../archives/emacs-bugs
$ grep '^In.*pc-linux-gnu' 2018-* 2019-* 2020-*|wc -l
1594
~/.../archives/emacs-bugs
$ grep '^In.*pc-linux-gnu' 2018-* 2019-* 2020-*|grep -i gtk|wc -l
1274
~/.../archives/emacs-bugs
$ grep '^In.*pc-linux-gnu' 2018-* 2019-* 2020-*|grep -i 'x toolkit'|wc -l
268
~/.../archives/emacs-bugs
$ grep '^In.*pc-linux-gnu' 2018-* 2019-* 2020-*|grep -i 'motif'|wc -l
3

So itʼs ~80% GTK, 16% Lucid, and Motif can be removed.

    Eli>   . look at GNU/Linux distros and see if they still provide builds
    Eli>     with any of these toolkits, and what was the last version of Emacs
    Eli>     when they did

Debian provides gtk, lucid, and no-x builds.

Robert

Re: Cleaning out old X11 toolkits?

[Lars Ingebrigtsen]

Robert Pluim <rpluim@gmail.com> writes:

> ~/.../archives/emacs-bugs
> $ grep '^In.*pc-linux-gnu' 2018-* 2019-* 2020-*|grep -i 'motif'|wc -l
> 3
>
> So itʼs ~80% GTK, 16% Lucid, and Motif can be removed.

Thanks for the analysis, and I agree that if we're going to remove
anything Motif seems like the obvious candidate here.

One possible thing to take into consideration: Are there any systems
(that we care about) where Motif is the only toolkit solution available?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Cleaning out old X11 toolkits?

[Robert Pluim]

>>>>> On Fri, 12 Feb 2021 10:35:46 +0100, Lars Ingebrigtsen <larsi@gnus.org> said:

    Lars> Robert Pluim <rpluim@gmail.com> writes:
    >> ~/.../archives/emacs-bugs
    >> $ grep '^In.*pc-linux-gnu' 2018-* 2019-* 2020-*|grep -i 'motif'|wc -l
    >> 3
    >> 
    >> So itʼs ~80% GTK, 16% Lucid, and Motif can be removed.

    Lars> Thanks for the analysis, and I agree that if we're going to remove
    Lars> anything Motif seems like the obvious candidate here.

    Lars> One possible thing to take into consideration: Are there any systems
    Lars> (that we care about) where Motif is the only toolkit solution available?

All of the motif builds in that time interval were on Gnu/Linux. And
isnʼt lucid available everywhere as a fall-back?

Robert

Re: Cleaning out old X11 toolkits?

[Lars Ingebrigtsen]

Robert Pluim <rpluim@gmail.com> writes:

>     Lars> One possible thing to take into consideration: Are there any systems
>     Lars> (that we care about) where Motif is the only toolkit
>     Lars> solution available?
>
> All of the motif builds in that time interval were on Gnu/Linux.

Right, and there are other toolkits available there, so that sounds like
a plus for removing Motif.

> And isnʼt lucid available everywhere as a fall-back?

That's what I was wondering.  :-)  It does sound unlikely, though.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Cleaning out old X11 toolkits?

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

>> And isnʼt lucid available everywhere as a fall-back?
>
> That's what I was wondering.  :-)  It does sound unlikely, though.

I mean, it does sound unlikely that there's any systems with just Motif
available (that we care about).

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Cleaning out old X11 toolkits?

[Robert Pluim]

>>>>> On Fri, 12 Feb 2021 11:04:00 +0100, Lars Ingebrigtsen <larsi@gnus.org> said:

    Lars> Lars Ingebrigtsen <larsi@gnus.org> writes:
    >>> And isnʼt lucid available everywhere as a fall-back?
    >> 
    >> That's what I was wondering.  :-)  It does sound unlikely, though.

    Lars> I mean, it does sound unlikely that there's any systems with just Motif
    Lars> available (that we care about).

We could just remove support for '--with-x-toolkit=motif' from
configure.ac for a while, and see if anyone complains. If not, we rip
out the actual Motif code.

Robert

Re: Cleaning out old X11 toolkits?

[Lars Ingebrigtsen]

Robert Pluim <rpluim@gmail.com> writes:

> We could just remove support for '--with-x-toolkit=motif' from
> configure.ac for a while, and see if anyone complains. If not, we rip
> out the actual Motif code.

Sounds like a plan.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Cleaning out old X11 toolkits?

[Jean Louis]

* Lars Ingebrigtsen <larsi@gnus.org> [2021-02-12 12:36]:
> Robert Pluim <rpluim@gmail.com> writes:
> 
> > ~/.../archives/emacs-bugs
> > $ grep '^In.*pc-linux-gnu' 2018-* 2019-* 2020-*|grep -i 'motif'|wc -l
> > 3
> >
> > So itʼs ~80% GTK, 16% Lucid, and Motif can be removed.
> 
> Thanks for the analysis, and I agree that if we're going to remove
> anything Motif seems like the obvious candidate here.
> 
> One possible thing to take into consideration: Are there any systems
> (that we care about) where Motif is the only toolkit solution
> available?

Motif:
http://www.opengroup.org/desktop/motif.html

CDE:
https://sourceforge.net/p/cdesktopenv/wiki/Home/

As Motif offers high degree of compatibility for various systems it
would not be good to remove it for the sake of users on those
systems.

Jean

Re: Cleaning out old X11 toolkits?

[Jean Louis]

* Jean Louis <bugs@gnu.support> [2021-02-13 00:54]:
> * Lars Ingebrigtsen <larsi@gnus.org> [2021-02-12 12:36]:
> > Robert Pluim <rpluim@gmail.com> writes:
> > 
> > > ~/.../archives/emacs-bugs
> > > $ grep '^In.*pc-linux-gnu' 2018-* 2019-* 2020-*|grep -i 'motif'|wc -l
> > > 3
> > >
> > > So itʼs ~80% GTK, 16% Lucid, and Motif can be removed.
> > 
> > Thanks for the analysis, and I agree that if we're going to remove
> > anything Motif seems like the obvious candidate here.
> > 
> > One possible thing to take into consideration: Are there any systems
> > (that we care about) where Motif is the only toolkit solution
> > available?

CDE is obviously developed and in use. I hope you have reviewed the
link, yesterday people were developing CDE that uses Motif or Lesstif
and runs on many UNIX systems and GNU/Linux, it would not be good
removing Motif from people who may wish to run it. They may not be
reporting on this mailing list, but review the CDE website.

By the way, I cannot compile lesstif/motif:

  What toolkit should Emacs use?                          MOTIF



~/Programming/Software/emacs
$ make
cd . && ./autogen.sh autoconf
Checking whether you have the necessary tools...
(Read INSTALL.REPO for more details on building Emacs)
Checking for autoconf (need at least version 2.65) ... ok
Your system has the required tools.
Running 'autoreconf -fi -I m4' ...
You can now run './autogen.sh git'.
if [ -x ./config.status ]; then	\
     ./config.status --recheck;	\
else				\
     ./configure --cache-file=/dev/null; \
fi
running CONFIG_SHELL=/bin/sh /bin/sh ./configure --prefix=/package/text/emacs --with-modules --with-x-toolkit=motif PKG_CONFIG_PATH=/home/data1/protected/GNUstep/Library/Libraries/pkgconfig:/usr/lib/pkgconfig --no-create --no-recursion
checking for xcrun... no
checking for GNU Make... make
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
configure: error: '--with-x-toolkit=motif' is invalid;
this option's value should be 'yes', 'no', 'lucid', 'athena', 'gtk',
'gtk2' or 'gtk3'.  'yes' and 'gtk' are synonyms.
'athena' and 'lucid' are synonyms.
make: *** [Makefile:452: config.status] Error 1
~/Programming/Software/emacs
$

So motif is not even option currently. It is bug.

Jean

Re: Cleaning out old X11 toolkits?

[Stefan Kangas]

Jean Louis <bugs@gnu.support> writes:

>> > One possible thing to take into consideration: Are there any systems
>> > (that we care about) where Motif is the only toolkit solution
>> > available?
>
> CDE is obviously developed and in use.

I think he means operating system, not desktop environment.

> By the way, I cannot compile lesstif/motif:

Yes, it has been disabled on master.  I believe it was announced earlier
in this thread.

Re: Cleaning out old X11 toolkits?

[Eli Zaretskii]

> Date: Sat, 13 Feb 2021 00:49:29 +0300
> From: Jean Louis <bugs@gnu.support>
> Cc: Robert Pluim <rpluim@gmail.com>, Eli Zaretskii <eliz@gnu.org>,
>   chad <yandros@gmail.com>, emacs-devel@gnu.org
> 
> > > So itʼs ~80% GTK, 16% Lucid, and Motif can be removed.
> > 
> > Thanks for the analysis, and I agree that if we're going to remove
> > anything Motif seems like the obvious candidate here.
> > 
> > One possible thing to take into consideration: Are there any systems
> > (that we care about) where Motif is the only toolkit solution
> > available?
> 
> Motif:
> http://www.opengroup.org/desktop/motif.html
> 
> CDE:
> https://sourceforge.net/p/cdesktopenv/wiki/Home/
> 
> As Motif offers high degree of compatibility for various systems it
> would not be good to remove it for the sake of users on those
> systems.

Which systems are those where Motif is the _only_ toolkit available,
though?  That was the question.  If you have the answer to that
question, or some resource which could provide the answer, please
tell.  I couldn't find the answer on the pages to which you pointed;
did I miss something?  The pages look outdated, btw.

Re: Cleaning out old X11 toolkits?

[Jean Louis]

* Eli Zaretskii <eliz@gnu.org> [2021-02-13 10:52]:
> > Date: Sat, 13 Feb 2021 00:49:29 +0300
> > From: Jean Louis <bugs@gnu.support>
> > Cc: Robert Pluim <rpluim@gmail.com>, Eli Zaretskii <eliz@gnu.org>,
> >   chad <yandros@gmail.com>, emacs-devel@gnu.org
> > 
> > > > So itʼs ~80% GTK, 16% Lucid, and Motif can be removed.
> > > 
> > > Thanks for the analysis, and I agree that if we're going to remove
> > > anything Motif seems like the obvious candidate here.
> > > 
> > > One possible thing to take into consideration: Are there any systems
> > > (that we care about) where Motif is the only toolkit solution
> > > available?
> > 
> > Motif:
> > http://www.opengroup.org/desktop/motif.html
> > 
> > CDE:
> > https://sourceforge.net/p/cdesktopenv/wiki/Home/
> > 
> > As Motif offers high degree of compatibility for various systems it
> > would not be good to remove it for the sake of users on those
> > systems.
> 
> Which systems are those where Motif is the _only_ toolkit available,
> though?  That was the question.  If you have the answer to that
> question, or some resource which could provide the answer, please
> tell.  I couldn't find the answer on the pages to which you pointed;
> did I miss something?  The pages look outdated, btw.

I do not believe there is any UNIX or UNIX-like system where there is
just one toolkit available.

In making that decision it is better to consider users, not systems.

Then we can say if there is no system exclusively using Lucid, let us
exclude Lucid, until you get only one system exclusively using Gtk,
then there will be no users neither of Lucid neither of
Motif/Lesstiff, of course. That is developers' decision.

When Lucid is removed there will be not any more 0.24% of Debian users
using Emacs with Lucid:
https://qa.debian.org/popcon.php?package=emacs

In that analogy there is no system that uses exclusively Lucid, should
it be removed?

Motif/Lesstif may be used more today on those systems where CDE is
used.

Maybe Debian decided not to provide Lesstif/Motif versions, so there
may not be the package any more.

Gentoo does use Motif:
https://packages.gentoo.org/useflags/motif and offers some
instructions:
https://wiki.gentoo.org/wiki/Xft_support_for_GNU_Emacs#Motif_toolkit
which are indications that there are users using Emacs with motif

There are some benefits (profits) and expenses (impacted users or
systems) in making such decisions.

Is the cost of keeping Motif/Lesstif too high? Maybe developers would
have additional work to cleanup sources, that is also the cost.

Is the principle in use that if certain function is not used by more
than XXX number of people, that such function should be removed?
Removing Motif in Emacs seem to be action that would impact many
involved. With many I think of thousands probable users, but not
hundreds of thousands. Maybe 10000 maybe 2000 maybe 20000 or more. Who
knows, I have not come to research number of users. But Gentoo users
are in that group.

I also believe that Motif version would be used on CDE for reasons of
consistency and harmony with the look a like of the system. Me
personally, in last 3 days I did want to use Motif, I just did not
find easy way to do it. In fact I am using Lucid because I was
thinking it is Motif. But I don't mind if you remove or not as I
failed in last 2 years to come to it, today I have realized it. Here I
am rather considering impact on other users globally.

Project activity of CDE:
https://sourceforge.net/projects/cdesktopenv/

,----
|  Matt Smith posted a comment on ticket #92
| I think it's still worthwhile to commit the fixes you've done since they still fix some obvious build issues (like fpic vs fPIC) and are a good starting point to figuring out the rest of the problems. I left dtwm running for quite awhile and it eventually crashed so I have two core dumps from this most recent attempt. One for dtwm and one for dtdbcache. Both attached.
| 2 days ago
| Lev Kujawski posted a comment on ticket #92
| Hi, I'm sorry to hear that CDE's not working for you. Do you think it is still worthwhile to commit the build fixes I have for SPARC? I'd be willing to look at any backtraces or core dumps if you have them.
| 2 days ago
| Matt Smith posted a comment on ticket #92
| Just a quick followup. make World.dev compiles with -pthread for tcl. Actually running it still fails though. I see dtdbcache segfaulting when I run /usr/dt/bin/Xsessionbut that's not the only problem. Just running dtwm or dtsession will hang with no error messages. I'll probably need to hit this with gdb but I don't really have the time right now to go further into it. Maybe in a month or two
| 3 days ago
| John Garry posted a comment on ticket #104
| Looks like many people are against this, i'm fine with it being optional trays ruin the aesthetic of cde. maybe it doesnt have to be added to cde though if stalonetray had a decoration or mode that made it look more like it belongs there. On 2/9/21, John Garry johngarry@users.sourceforge.net wrote: [tickets:#104] Possible system try blended into the bar? Status: open Created: Tue Feb 09, 2021 07:19 AM UTC by John Garry Last Updated: Tue Feb 09, 2021 07:19 AM UTC Owner: nobody So far, each time i've...
| 1 week ago
| Jon Trulson committed [3685f6]
| Purge unused Freetype dependency from CDE
`----

,----
| Original Reference Platforms
| 
| In 1999 the reference platforms on which CDE was built were;
| Supplier 	Platform 	OS
| Digital 	AlphaStation 200 	Digital UNIX V4.0
| Fujitsu 	DS/90 7000 	UXP/DS V20L10
| HP 	HP9000/7xx 	HP-UX 10.01
| IBM 	RS/6000 	AIX 4.2
| Novell 	Intel 486/Pentium 	UnixWare 2.02
| Sun 	SPARCstation 	Solaris 2.4
`----

Maybe those platforms are where Emacs may play well in its Motif
version as to be harmonized with the rest of the desktop system used
like CDE?

How many users of those platforms are there I would not know, but
there could be some. Would they be able to compile software? Maybe
their settings and packages would be impacted, and administrators
would have hard time in changing definitions on how to compile new
Emacs.

Evidence is there that:

- CDE is still under development, few days ago, unless the Sourceforge
  database is wrong or not functional

- There are reasons why CDE works and it probably works on those
  platforms and GNU/Linux as well, even if not popular by majority,
  but it works.

- that there are Gentoo instructions for Emacs on Lesstif/Motif for
  Gentoo users, I have not verified other GNU/Linux distributions

Deducing by the above facts, number of users could be in thousands. 

Basically you could consider profit and expenses in developer
terms. If it is too much of an expense to keep Lesstif/Motif version
then you may enforce it over probably some few thousands of Emacs
Motif users. But maybe Emacs team and Emacs community can profit and
benefit by providing support for various toolkits and not cutting that
support too early. That sounds more friendly to me.

I would apply that formula, does it give more benefits to provide
Motif/Lesstif versions and open up harmonization or easier use on
those other platforms and let Gentoo and other users in peace?

Or maybe those users using Motif/Lesstif versions could swallow it?
Maybe those platforms which don't work same as GNU/Linux better adopt
Gtk to run Emacs on their system? That is maybe more benefit then
expense? Maybe administrators of those platforms and package
maintainers better learn how to compile Emacs with Gtk or Lucid. Maybe
1000 users is too little, why bother? Then employ Emacs developers to
remove any traces of Motif as it was costly keeping them in sources?

Those are questions to consider by development IMHO.

I do remember that back in time Emacs development was proud that it
works on so many systems. Now that changed. Compatibilities and wider
adoption seem to be of less importance today. All efforts move what is
most popular while forgetting good and well established technologies.

I live in a different surprising new world. Maybe tomorrow NetBSD will
decide not to be compatible with XX+ platforms and will just work for
PC, even that would not be surprising any more.

Widening adoption drives innovation. Narrowing features, excluding
such features, does not grow innovation.

Jean

https://en.wikipedia.org/wiki/NetBSD

Re: Cleaning out old X11 toolkits?

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: chad <yandros@gmail.com>,  emacs-devel@gnu.org
> Date: Fri, 12 Feb 2021 10:30:41 +0100
> 
> So itʼs ~80% GTK, 16% Lucid, and Motif can be removed.

Thanks, that's an important data point, IMO.

> Debian provides gtk, lucid, and no-x builds.

Does that include Emacs 27 and snapshots of Emacs 28?

Re: Cleaning out old X11 toolkits?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 639 bytes --]

On Fri, Feb 12, 2021 at 01:28:42PM +0200, Eli Zaretskii wrote:
> > From: Robert Pluim <rpluim@gmail.com>
> > Cc: chad <yandros@gmail.com>,  emacs-devel@gnu.org
> > Date: Fri, 12 Feb 2021 10:30:41 +0100
> > 
> > So itʼs ~80% GTK, 16% Lucid, and Motif can be removed.
> 
> Thanks, that's an important data point, IMO.
> 
> > Debian provides gtk, lucid, and no-x builds.
> 
> Does that include Emacs 27 and snapshots of Emacs 28?

As far as I can see, 27.1 is in unstable/testing. 28 hasn't arrived
(yet).

Cheers

[1] https://packages.debian.org/search?keywords=emacs&searchon=names&exact=1&suite=all&section=all

 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Cleaning out old X11 toolkits?

[Robert Pluim]

>>>>> On Fri, 12 Feb 2021 13:28:42 +0200, Eli Zaretskii <eliz@gnu.org> said:


    >> Debian provides gtk, lucid, and no-x builds.

    Eli> Does that include Emacs 27 and snapshots of Emacs 28?

Debian stable has emacs-25. Debian testing has emacs-27. They donʼt do
emacs 28 snapshots as far as I can tell.

Robert

Re: Cleaning out old X11 toolkits?

[Basil L. Contovounesios]

Robert Pluim <rpluim@gmail.com> writes:

>>>>>> On Fri, 12 Feb 2021 13:28:42 +0200, Eli Zaretskii <eliz@gnu.org> said:
>
>
>     >> Debian provides gtk, lucid, and no-x builds.
>
>     Eli> Does that include Emacs 27 and snapshots of Emacs 28?
>
> Debian stable has emacs-25.

Current Debian Stable (codename Buster) has Emacs 26.1:

  https://packages.debian.org/buster/emacs

The emacs25 package in Buster is transitional (i.e. just a name kept
around for backward compatibility), and actually provides Emacs 26.1:

  https://packages.debian.org/buster/emacs25

-- 
Basil

Re: Cleaning out old X11 toolkits?

[Sean Whitton]

Hello,

On Fri 12 Feb 2021 at 01:28PM +02, Eli Zaretskii wrote:

>> From: Robert Pluim <rpluim@gmail.com>
>> Cc: chad <yandros@gmail.com>,  emacs-devel@gnu.org
>> Date: Fri, 12 Feb 2021 10:30:41 +0100
>>
>> So itʼs ~80% GTK, 16% Lucid, and Motif can be removed.
>
> Thanks, that's an important data point, IMO.
>
>> Debian provides gtk, lucid, and no-x builds.
>
> Does that include Emacs 27 and snapshots of Emacs 28?

Yes, we haven't stopped providing lucid with Emacs 27 (and a lot of the
Debian Emacsen team use it).  There are several unofficial emacs28.deb
snapshot services and I believe they typically provie lucid too.

-- 
Sean Whitton

Re: Cleaning out old X11 toolkits?

[Ulrich Mueller]

>>>>> On Fri, 12 Feb 2021, Eli Zaretskii wrote:

>   . look at GNU/Linux distros and see if they still provide builds
>     with any of these toolkits,

Data point from Gentoo: We support all of GTK 3, Motif, Lucid/Athena
(with or without Xaw3d), X without any toolkit, and Emacs without X.
Bug reports received during the last year indicate that at least
Lucid/Athena is still in use.

We no longer support GTK 2 at this point. Also no Aqua/Cocoa because
(quoting our main developer for the Mac OS port) "Clang compilers are
sufficiently difficult to build at this point".

>     and what was the last version of Emacs when they did

We support all above-mentioned for the latest released version (27.1),
and we have live ebuilds for master and emacs-27 branches.

Re: Cleaning out old X11 toolkits?

[martin rudalics]

 > In more pragmatic terms, I would guess that it's entirely possible to
 > excise motif/lesstif, athena, and Xaw3d from main without anyone noticing.

I regularly build with Motif.  Here it's just as reliable as Lucid.

 > Whether this is worth the effort in a world leaning ever so slowly towards
 > Cairo and pgtk is a little hard to tell, but a quick grep through src
 > suggests that it would at least clear up a bunch of #ifdef spaghetti.
 >
 > Would the maintainers be interested in a branch that tried this? Would it
 > be better to wait for pgtk to settle first? Is there a big use-case for
 > those toolkits of which I'm unaware?

The only build I'd clean out is GTK-3.  Seriously.  I really ask people
to use the pgtk branch for their daily work on a regular basis.  Once it
has been merged in, we can banish GTK-3 (not necessarily GTK-2 which
works quite satisfactorily here) from the remaining X-parts of our code
base and nobody will complain about those #ifdef spaghetti any more.

I give you one simple example with emacs -Q and the following one-line
initial file:

(setq default-frame-alist '((cursor-color . "red3") (height . 32)))

Here this works well with Lucid, Motif, GTK-2 and without x-toolkit.  It
does not work with GTK-3, which shows four lines below the minibuffer
that apparently do not belong to anyone (it works with a GTK-3 build of
Emacs 25 though).  And the X pgtk build here simply refuses to shrink
the frame to 32 lines but keeps it at 36 lines.  It's behaviors like
this that can keep you busy trying to make GTK-3 builds do something
reasonable at least.

Once all the GTK-3 specific stuff has been moved to pgtk, people can fix
GTK-3 specific bugs there and do not have to care about Motif or other
X11-toolkits any more just as they do not have to care about W32 or NS
now.

martin

Re: Cleaning out old X11 toolkits?

[Robert Pluim]

>>>>> On Fri, 12 Feb 2021 11:00:28 +0100, martin rudalics <rudalics@gmx.at> said:

    >> Whether this is worth the effort in a world leaning ever so slowly towards
    >> Cairo and pgtk is a little hard to tell, but a quick grep through src
    >> suggests that it would at least clear up a bunch of #ifdef spaghetti.
    >> 
    >> Would the maintainers be interested in a branch that tried this? Would it
    >> be better to wait for pgtk to settle first? Is there a big use-case for
    >> those toolkits of which I'm unaware?

    martin> The only build I'd clean out is GTK-3.  Seriously.  I really ask people
    martin> to use the pgtk branch for their daily work on a regular basis.  Once it
    martin> has been merged in, we can banish GTK-3 (not necessarily GTK-2 which
    martin> works quite satisfactorily here) from the remaining X-parts of our code
    martin> base and nobody will complain about those #ifdef spaghetti any more.

If thatʼs what we want, then we should merge pgtk to master and ask
people to test it.

    martin> I give you one simple example with emacs -Q and the following one-line
    martin> initial file:

    martin> (setq default-frame-alist '((cursor-color . "red3") (height . 32)))

    martin> Here this works well with Lucid, Motif, GTK-2 and without x-toolkit.  It
    martin> does not work with GTK-3, which shows four lines below the minibuffer
    martin> that apparently do not belong to anyone (it works with a GTK-3 build of
    martin> Emacs 25 though).  And the X pgtk build here simply refuses to shrink
    martin> the frame to 32 lines but keeps it at 36 lines.  It's behaviors like
    martin> this that can keep you busy trying to make GTK-3 builds do something
    martin> reasonable at least.

Works fine for me here with a GTK build, although my frame ends up
with a height of 30. pgtk generally has issues with frame sizing and
positioning.

    martin> Once all the GTK-3 specific stuff has been moved to pgtk, people can fix
    martin> GTK-3 specific bugs there and do not have to care about Motif or other
    martin> X11-toolkits any more just as they do not have to care about W32 or NS
    martin> now.

Tell you what, you remove impure GTK, and Iʼll remove Xft. Deal? :-)

Robert

Re: Cleaning out old X11 toolkits?

[martin rudalics]

 > If thatʼs what we want, then we should merge pgtk to master and ask
 > people to test it.

Why should people "test" it more when it's been merged?  Checking out
the pgtk is just as easy.

 > Works fine for me here with a GTK build, although my frame ends up
 > with a height of 30. pgtk generally has issues with frame sizing and
 > positioning.

We might be in for some rude awakening here.

martin

Merging native-comp and pgtk

[Stefan Monnier]

BTW,

Where are we w.r.t merging those two branches into `master`?

IIUC the plan is to include those features as "experimental" in
Emacs-28.1, right?  If so, I think it's becoming urgent that we merge
them into `master`.


        Stefan

Re: Merging native-comp and pgtk

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Fri, 12 Feb 2021 13:14:08 -0500
> 
> Where are we w.r.t merging those two branches into `master`?
> 
> IIUC the plan is to include those features as "experimental" in
> Emacs-28.1, right?  If so, I think it's becoming urgent that we merge
> them into `master`.

I don't thin it's ready yet, see my other message.

(And why is it urgent?  Emacs 27.2 is not out yet, and Emacs 28 is
supposed to have native-comp included.  So we still have quite a long
way to go.)

Re: Merging native-comp and pgtk

[Andy Moreton]

On Fri 12 Feb 2021, Eli Zaretskii wrote:

>> From: Stefan Monnier <monnier@iro.umontreal.ca>
>> Date: Fri, 12 Feb 2021 13:14:08 -0500
>> 
>> Where are we w.r.t merging those two branches into `master`?
>> 
>> IIUC the plan is to include those features as "experimental" in
>> Emacs-28.1, right?  If so, I think it's becoming urgent that we merge
>> them into `master`.
>
> I don't thin it's ready yet, see my other message.
>
> (And why is it urgent?  Emacs 27.2 is not out yet, and Emacs 28 is
> supposed to have native-comp included.  So we still have quite a long
> way to go.)

For me, the native-comp branch is not yet usable on mingw64 64bit:

 - The async background compile and intrusive warnings are intrusive and
   make emacs unresponsive to user input. The async background compile
   is difficult to control (other than disabling completely), or to see
   which files are still in the queue to be compiled.

 - The AOT build ignores prebuilt .eln files due to incorrect filename
   comparison (see bug #46256), and so does an async rebuild of all .eln
   files even if AOT prebuilt files already exist. This makes the
   effects of the first problem above more severe.

 - The dependencies for building .eln files do not work. If the .elc
   file for a .el file exists, the .eln will not be built (even if the
   .eln does not exist).

On mingw64 32bit configured with "--wide":

 - the build fails: bootstrap-emacs.exe crashes when compiling lisp).
   gdb does not produce anything informative in the backtrace.

 - the "--wide" configure flag affects ABI, so should be included in the
   native-comp ABI hash identifier in .eln filenames.

As a pre-existing problem (present in master), the .elc files are built
in the source tree, rather than the build tree. That prevents doing
out-of-tree builds for different platforms/ABIs from the same source
tree.

The native-comp branch looks promising, but still needs a little more
work to make it suitable for wider use. Whether that should be done on a
development branch or after merging into master is a separate issue.

    AndyM

Re: Merging native-comp and pgtk

[Stefan Monnier]

> For me, the native-comp branch is not yet usable on mingw64 64bit:

Should "usable on mingw64" a prerequisite to merge a branch for an
experimental feature that will only be activated upon explicit
user request?

> The native-comp branch looks promising, but still needs a little more
> work to make it suitable for wider use. Whether that should be done on a
> development branch or after merging into master is a separate issue.

But this thread is specifically about where that work should take place.
As long as the code is on a separate branch, it incurs the extra
overhead of having to keep merging changes from `master` to keep it up
to date.


        Stefan

Re: Merging native-comp and pgtk

[Andy Moreton]

On Fri 12 Feb 2021, Stefan Monnier wrote:

>> For me, the native-comp branch is not yet usable on mingw64 64bit:
>
> Should "usable on mingw64" a prerequisite to merge a branch for an
> experimental feature that will only be activated upon explicit
> user request?

Not necessarily - my note was a list of some things that still need
attention, not an argument against merging native-comp into master.

>> The native-comp branch looks promising, but still needs a little more
>> work to make it suitable for wider use. Whether that should be done on a
>> development branch or after merging into master is a separate issue.
>
> But this thread is specifically about where that work should take place.
> As long as the code is on a separate branch, it incurs the extra
> overhead of having to keep merging changes from `master` to keep it up
> to date.

I think you misunderstood my meaning: I don't have a strong opinion on
whether it should be be merged.

    AndyM

Re: Merging native-comp and pgtk

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Fri, 12 Feb 2021 18:06:45 -0500
> Cc: emacs-devel@gnu.org
> 
> > For me, the native-comp branch is not yet usable on mingw64 64bit:
> 
> Should "usable on mingw64" a prerequisite to merge a branch for an
> experimental feature that will only be activated upon explicit
> user request?

We may indeed decide that good support for the MS-Windows build is not
a prerequisite, but then we'd need someone to volunteer to fix any
potential problems, including breaking the MS-Windows build, as result
of the merge.  IME, any breakage of that kind on master is left
unfixed until yours truly does it.  If I'm unable to build the
native-comp code (and Andy just said the --with-wide-int
configuration, which is what I use, doesn't build), then I will be
unable to build and test this code, and will not be able to fix any
problems reported for master, whether on MS-Windows or on other
platforms.  Then someone else will have to take over that job.

IOW, having a master branch that one of the co-maintainers cannot
build is not a good situation.  We should try avoiding it, IMO.

Re: Merging native-comp and pgtk

[Stefan Monnier]

> We may indeed decide that good support for the MS-Windows build is not
> a prerequisite, but then we'd need someone to volunteer to fix any
> potential problems, including breaking the MS-Windows build, as result
> of the merge.  IME, any breakage of that kind on master is left
> unfixed until yours truly does it.

I think it's clear we don't want to merge  of those two branches if they
introduce regressions in the "normal" case (i.e. the case where we
build without pgtk and without native-comp support).

> If I'm unable to build the native-comp code (and Andy just said
> the --with-wide-int configuration, which is what I use, doesn't
> build),

I suspect that his report of breakage is not that the native-comp branch
can't build at all on Windows+wide-int but that it doesn't build *if* you
enable native-comp.


        Stefan

Re: Merging native-comp and pgtk

[Andy Moreton]

On Sat 13 Feb 2021, Stefan Monnier wrote:

>> We may indeed decide that good support for the MS-Windows build is not
>> a prerequisite, but then we'd need someone to volunteer to fix any
>> potential problems, including breaking the MS-Windows build, as result
>> of the merge.  IME, any breakage of that kind on master is left
>> unfixed until yours truly does it.
>
> I think it's clear we don't want to merge  of those two branches if they
> introduce regressions in the "normal" case (i.e. the case where we
> build without pgtk and without native-comp support).
>
>> If I'm unable to build the native-comp code (and Andy just said
>> the --with-wide-int configuration, which is what I use, doesn't
>> build),
>
> I suspect that his report of breakage is not that the native-comp branch
> can't build at all on Windows+wide-int but that it doesn't build *if* you
> enable native-comp.

Yes. The problem is only when configured with both "--with-wide-int" and
"--with-nativecomp". Sorry if that was not clear.

   AndyM

Re: Merging native-comp and pgtk

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Sat, 13 Feb 2021 14:10:45 +0000
> 
> > I suspect that his report of breakage is not that the native-comp branch
> > can't build at all on Windows+wide-int but that it doesn't build *if* you
> > enable native-comp.
> 
> Yes. The problem is only when configured with both "--with-wide-int" and
> "--with-nativecomp". Sorry if that was not clear.

It was clear to me.

Re: Merging native-comp and pgtk

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: andrewjmoreton@gmail.com,  emacs-devel@gnu.org
> Date: Sat, 13 Feb 2021 08:27:39 -0500
> 
> > If I'm unable to build the native-comp code (and Andy just said
> > the --with-wide-int configuration, which is what I use, doesn't
> > build),
> 
> I suspect that his report of breakage is not that the native-comp branch
> can't build at all on Windows+wide-int but that it doesn't build *if* you
> enable native-comp.

Sure, but I _want_ to enable it.

Re: Merging native-comp and pgtk

[martin rudalics]

 > I think it's clear we don't want to merge  of those two branches if they
 > introduce regressions in the "normal" case (i.e. the case where we
 > build without pgtk and without native-comp support).

Which probably means that we still have to keep around and maintain our
"impure" GTK3 build as well which implicitly means that the overhead for
frame maintenance will almost double: In my experience, GTK3 is
currently responsible for 90% of the overhead, NS for 9% and the entire
rest (including Windows) for less than 1%.

martin

Re: Merging native-comp and pgtk

[Eli Zaretskii]

> Cc: andrewjmoreton@gmail.com, emacs-devel@gnu.org
> From: martin rudalics <rudalics@gmx.at>
> Date: Sat, 13 Feb 2021 16:48:53 +0100
> 
>  > I think it's clear we don't want to merge  of those two branches if they
>  > introduce regressions in the "normal" case (i.e. the case where we
>  > build without pgtk and without native-comp support).
> 
> Which probably means that we still have to keep around and maintain our
> "impure" GTK3 build as well which implicitly means that the overhead for
> frame maintenance will almost double: In my experience, GTK3 is
> currently responsible for 90% of the overhead, NS for 9% and the entire
> rest (including Windows) for less than 1%.

The maintenance overhead will double if we assume that pgtk will
impose the same burden as GTK3 -- is the a reason to make that
assumption?

But yes, we will need to keep the GTK3 code at least in Emacs 28.  If
pgtk turns out to be relatively problem-free and similar enough in UX
to GTK3, we could look into removing GTK3 from Emacs 29.

Re: Merging native-comp and pgtk

[martin rudalics]

 > The maintenance overhead will double if we assume that pgtk will
 > impose the same burden as GTK3 -- is the a reason to make that
 > assumption?

It's not very clear to me how many people tested the pgtk branch under X
so far.  Here I get occasional crashes and strange artifacts but no
simple, presentable scenario yet.  So it's too early to tell, I suppose.

 > But yes, we will need to keep the GTK3 code at least in Emacs 28.  If
 > pgtk turns out to be relatively problem-free and similar enough in UX
 > to GTK3, we could look into removing GTK3 from Emacs 29.

If we intend to make it Wayland-only, we have to keep the impure GTK3
branch anyway.  So all depends on how well pgtk integrates into the
existing X installations and whether we want to remove duplicate code.

martin

Re: Merging native-comp and pgtk

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Fri, 12 Feb 2021 22:29:31 +0000
> 
>  - The async background compile and intrusive warnings are intrusive and
>    make emacs unresponsive to user input. The async background compile
>    is difficult to control (other than disabling completely), or to see
>    which files are still in the queue to be compiled.

Did you report the problems with the warnings?  If not, please report
them to the bug tracker.

>  - The dependencies for building .eln files do not work. If the .elc
>    file for a .el file exists, the .eln will not be built (even if the
>    .eln does not exist).

Was this reported as a bug?

> On mingw64 32bit configured with "--wide":
> 
>  - the build fails: bootstrap-emacs.exe crashes when compiling lisp).
>    gdb does not produce anything informative in the backtrace.
> 
>  - the "--wide" configure flag affects ABI, so should be included in the
>    native-comp ABI hash identifier in .eln filenames.

Andrea, did you try a 32-bit build --with-wide-int?  Did it work for
you?

> As a pre-existing problem (present in master), the .elc files are built
> in the source tree, rather than the build tree. That prevents doing
> out-of-tree builds for different platforms/ABIs from the same source
> tree.

Yes, we don't yet fully support out-of-tree builds, and that's
unrelated to the native-comp branch.

Thanks.

Re: Merging native-comp and pgtk

[Andy Moreton]

On Sat 13 Feb 2021, Eli Zaretskii wrote:

>> From: Andy Moreton <andrewjmoreton@gmail.com>
>> Date: Fri, 12 Feb 2021 22:29:31 +0000
>> 
>>  - The async background compile and intrusive warnings are intrusive and
>>    make emacs unresponsive to user input. The async background compile
>>    is difficult to control (other than disabling completely), or to see
>>    which files are still in the queue to be compiled.
>
> Did you report the problems with the warnings?  If not, please report
> them to the bug tracker.

I mentioned this on 2020-12-21 in bug #45303.

>>  - The dependencies for building .eln files do not work. If the .elc
>>    file for a .el file exists, the .eln will not be built (even if the
>>    .eln does not exist).
>
> Was this reported as a bug?

No. I thought I had mentioned it here or in #45303, but I cannot find it
in the archives.

>> As a pre-existing problem (present in master), the .elc files are built
>> in the source tree, rather than the build tree. That prevents doing
>> out-of-tree builds for different platforms/ABIs from the same source
>> tree.
>
> Yes, we don't yet fully support out-of-tree builds, and that's
> unrelated to the native-comp branch.

I did not claim otherwise. Building the .elc files in the build tree
would surely make the .el -> .elc -> .eln dependencies easier to manage
though.

    AndyM

Re: Merging native-comp and pgtk

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Sat, 13 Feb 2021 13:07:04 +0000
> 
> >>  - The async background compile and intrusive warnings are intrusive and
> >>    make emacs unresponsive to user input. The async background compile
> >>    is difficult to control (other than disabling completely), or to see
> >>    which files are still in the queue to be compiled.
> >
> > Did you report the problems with the warnings?  If not, please report
> > them to the bug tracker.
> 
> I mentioned this on 2020-12-21 in bug #45303.
> 
> >>  - The dependencies for building .eln files do not work. If the .elc
> >>    file for a .el file exists, the .eln will not be built (even if the
> >>    .eln does not exist).
> >
> > Was this reported as a bug?
> 
> No. I thought I had mentioned it here or in #45303, but I cannot find it
> in the archives.

I suggest to report these two.  That will go a long way towards
ensuring they will be fixed.

Thanks.

Re: Merging native-comp and pgtk

[Andy Moreton]

On Sat 13 Feb 2021, Eli Zaretskii wrote:

>> From: Andy Moreton <andrewjmoreton@gmail.com>
>> Date: Sat, 13 Feb 2021 13:07:04 +0000
>> 
>> >>  - The async background compile and intrusive warnings are intrusive and
>> >>    make emacs unresponsive to user input. The async background compile
>> >>    is difficult to control (other than disabling completely), or to see
>> >>    which files are still in the queue to be compiled.
>> >
>> > Did you report the problems with the warnings?  If not, please report
>> > them to the bug tracker.
>> 
>> I mentioned this on 2020-12-21 in bug #45303.
>> 
>> >>  - The dependencies for building .eln files do not work. If the .elc
>> >>    file for a .el file exists, the .eln will not be built (even if the
>> >>    .eln does not exist).
>> >
>> > Was this reported as a bug?
>> 
>> No. I thought I had mentioned it here or in #45303, but I cannot find it
>> in the archives.
>
> I suggest to report these two.  That will go a long way towards
> ensuring they will be fixed.

Done:

  bug#46494 [native-comp] Problems with async background compile
  bug#46495 [native-comp] Build fails for 32bit --with-wide-int

I suspect that the wide int problem is not Windows-specific and should
be reproducable on Linux (but I have not tried that).

    AndyM

Re: Merging native-comp and pgtk

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Sat, 13 Feb 2021 18:01:24 +0000
> 
> > I suggest to report these two.  That will go a long way towards
> > ensuring they will be fixed.
> 
> Done:
> 
>   bug#46494 [native-comp] Problems with async background compile
>   bug#46495 [native-comp] Build fails for 32bit --with-wide-int

Thanks.

Re: Merging native-comp and pgtk

[Andrea Corallo via Emacs development discussions.]

Eli Zaretskii <eliz@gnu.org> writes:

[...]

> Andrea, did you try a 32-bit build --with-wide-int?  Did it work for
> you?

Yes, it's supposed to work, it was certainly working, but as I don't test
it on a regular base it might be broken now.  I'll verify and come back
on this in the dedicated bug report.

  Andrea

PS  This weekend I'm mostly off the game, I'll be back on top of things
just after.

Re: Merging native-comp and pgtk

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: Andy Moreton <andrewjmoreton@gmail.com>, emacs-devel@gnu.org
> Date: Sun, 14 Feb 2021 07:33:29 +0000
> 
> Yes, it's supposed to work, it was certainly working, but as I don't test
> it on a regular base it might be broken now.  I'll verify and come back
> on this in the dedicated bug report.

Thanks.  If this works on Posix platforms, it's probably
Windows-specific, perhaps something with 'long' being 32-bit data
type.

> PS  This weekend I'm mostly off the game, I'll be back on top of things
> just after.

There's no rush.

Re: Merging native-comp and pgtk

[Andreas Schwab]

On Feb 12 2021, Andy Moreton wrote:

> As a pre-existing problem (present in master), the .elc files are built
> in the source tree, rather than the build tree. That prevents doing
> out-of-tree builds for different platforms/ABIs from the same source
> tree.

.elc files are platform independent.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: Merging native-comp and pgtk

[Andy Moreton]

On Sat 13 Feb 2021, Andreas Schwab wrote:

> On Feb 12 2021, Andy Moreton wrote:
>
>> As a pre-existing problem (present in master), the .elc files are built
>> in the source tree, rather than the build tree. That prevents doing
>> out-of-tree builds for different platforms/ABIs from the same source
>> tree.
>
> .elc files are platform independent.

Yes, but that's not a good reason to prevent concurrent builds in
different build trees from working properly.

    AndyM

Re: Merging native-comp and pgtk

[Phillip Lord]

Andy Moreton <andrewjmoreton@gmail.com> writes:

> On mingw64 32bit configured with "--wide":
>
>  - the build fails: bootstrap-emacs.exe crashes when compiling lisp).
>    gdb does not produce anything informative in the backtrace.
>
>  - the "--wide" configure flag affects ABI, so should be included in the
>    native-comp ABI hash identifier in .eln filenames.

While this doesn't stop anyone else from doing it, I have just removed
support for 32bit builds from the windows packaging process:

https://lists.gnu.org/archive/html/emacs-devel/2021-01/msg00661.html


> As a pre-existing problem (present in master), the .elc files are built
> in the source tree, rather than the build tree. That prevents doing
> out-of-tree builds for different platforms/ABIs from the same source
> tree.

I am still in two minds on this. When I was building Emacs for x86_64
and i686, this prevented me from clean building both architectures at
the same time, so the one build with catch up with the other and then
they would fight.

At the same token, it did mean that I only had to build the elc files
once; so even with forcing me to build one, then the other, it was
probably still quicker.

Phil

Re: Merging native-comp and pgtk

[Eli Zaretskii]

> From: Phillip Lord <phillip.lord@russet.org.uk>
> Date: Tue, 16 Feb 2021 23:22:40 +0000
> Cc: emacs-devel@gnu.org
> 
> Andy Moreton <andrewjmoreton@gmail.com> writes:
> 
> > On mingw64 32bit configured with "--wide":
> >
> >  - the build fails: bootstrap-emacs.exe crashes when compiling lisp).
> >    gdb does not produce anything informative in the backtrace.
> >
> >  - the "--wide" configure flag affects ABI, so should be included in the
> >    native-comp ABI hash identifier in .eln filenames.
> 
> While this doesn't stop anyone else from doing it, I have just removed
> support for 32bit builds from the windows packaging process:
> 
> https://lists.gnu.org/archive/html/emacs-devel/2021-01/msg00661.html

Because it was unreasonable to expect you to keep supporting it when
MSYS2 folks dropped it.  And since MinGW64 dropped support of old
Windows versions, which are the main audience for the 32-bit build,
such a MinGW64 build is no longer useful anyway.

But that doesn't mean we want to drop support for 32-bit build on
MS-Windows.  As long as mingw.org's MinGW supports Windows 9X, we
should not deliberately break such builds.

Re: Merging native-comp and pgtk

[Andy Moreton]

On Wed 17 Feb 2021, Eli Zaretskii wrote:

>> From: Phillip Lord <phillip.lord@russet.org.uk>
>> Date: Tue, 16 Feb 2021 23:22:40 +0000
>> Cc: emacs-devel@gnu.org
>> 
>> Andy Moreton <andrewjmoreton@gmail.com> writes:
>> 
>> > On mingw64 32bit configured with "--wide":
>> >
>> >  - the build fails: bootstrap-emacs.exe crashes when compiling lisp).
>> >    gdb does not produce anything informative in the backtrace.
>> >
>> >  - the "--wide" configure flag affects ABI, so should be included in the
>> >    native-comp ABI hash identifier in .eln filenames.
>> 
>> While this doesn't stop anyone else from doing it, I have just removed
>> support for 32bit builds from the windows packaging process:
>> 
>> https://lists.gnu.org/archive/html/emacs-devel/2021-01/msg00661.html
>
> Because it was unreasonable to expect you to keep supporting it when
> MSYS2 folks dropped it.  And since MinGW64 dropped support of old
> Windows versions, which are the main audience for the 32-bit build,
> such a MinGW64 build is no longer useful anyway.

Agreed: when I wrote the above, I was unaware that the MSYS2 developers
had dropped 32bit support.

> But that doesn't mean we want to drop support for 32-bit build on
> MS-Windows.  As long as mingw.org's MinGW supports Windows 9X, we
> should not deliberately break such builds.

Does anyone test regularly on Win9x ? Inadvertant breakage seems likely
otherwise.

    AndyM

Re: Merging native-comp and pgtk

[Óscar Fuentes]

Andy Moreton <andrewjmoreton@gmail.com> writes:

>> Because it was unreasonable to expect you to keep supporting it when
>> MSYS2 folks dropped it.  And since MinGW64 dropped support of old
>> Windows versions, which are the main audience for the 32-bit build,
>> such a MinGW64 build is no longer useful anyway.
>
> Agreed: when I wrote the above, I was unaware that the MSYS2 developers
> had dropped 32bit support.

Just to clarify: MSYS2 dropped 32bit support for the POSIX packages,
because Cygwin (which is its upstream project) did so. But MSYS2 still
builds and distributes MinGW-w64 32 bit packages, and AFAIK the plan is
to keep doing that for the foreseeable future.

Re: Merging native-comp and pgtk

[Andy Moreton]

On Wed 17 Feb 2021, Óscar Fuentes wrote:

> Andy Moreton <andrewjmoreton@gmail.com> writes:
>
>>> Because it was unreasonable to expect you to keep supporting it when
>>> MSYS2 folks dropped it.  And since MinGW64 dropped support of old
>>> Windows versions, which are the main audience for the 32-bit build,
>>> such a MinGW64 build is no longer useful anyway.
>>
>> Agreed: when I wrote the above, I was unaware that the MSYS2 developers
>> had dropped 32bit support.
>
> Just to clarify: MSYS2 dropped 32bit support for the POSIX packages,
> because Cygwin (which is its upstream project) did so. But MSYS2 still
> builds and distributes MinGW-w64 32 bit packages, and AFAIK the plan is
> to keep doing that for the foreseeable future.

Thanks for clarifying. However given that the build system needs the
POSIX tools, the end result is similar.

    AndyM

Re: Merging native-comp and pgtk

[Óscar Fuentes]

Andy Moreton <andrewjmoreton@gmail.com> writes:

>> Just to clarify: MSYS2 dropped 32bit support for the POSIX packages,
>> because Cygwin (which is its upstream project) did so. But MSYS2 still
>> builds and distributes MinGW-w64 32 bit packages, and AFAIK the plan is
>> to keep doing that for the foreseeable future.
>
> Thanks for clarifying. However given that the build system needs the
> POSIX tools, the end result is similar.

Not from the point of view of the packager or anyone who has a 64 bit
machine. MSYS2 builds 32 bit executables with the same effort as the 64
bit ones, it only depends on which shorcut you use to start the shell or
the contents of an environment variable. Then you just install the
resulting package on the target machine.

Re: Merging native-comp and pgtk

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Wed, 17 Feb 2021 17:32:30 +0000
> 
> Does anyone test regularly on Win9x ?

Not lately, not that I know of.

> Inadvertant breakage seems likely otherwise.

Indeed.  But I'd like at least to avoid breaking it deliberately.

Re: Merging native-comp and pgtk

[Stefan Monnier]

>> Where are we w.r.t merging those two branches into `master`?
>> IIUC the plan is to include those features as "experimental" in
>> Emacs-28.1, right?  If so, I think it's becoming urgent that we merge
>> them into `master`.
> I don't thin it's ready yet, see my other message.

By "it" you mean both features?

I've seen some reference to the pgtk branch in another message but that
seemed to be a minor issue when using pgtk.

So now I'm wondering: what do you think should be the criteria for
inclusion into `master`?

As written above, I thought the plan was to include those as
experimental features for Emacs-28.1, so I thought the criteria were
going to be something like:
- Code is clean enough: doesn't risk introducing regressions into the rest
  of the code.
- It's very likely that the feature will reach maturity (i.e. lose
  the "experimental" label) in some not too distant future.
- It's already usable enough that most people who're looking forward to
  this feature will be fairly satisfied if they try it (it might still
  have some rough edges, but by and large it works).

> (And why is it urgent?  Emacs 27.2 is not out yet, and Emacs 28 is
> supposed to have native-comp included.  So we still have quite a long
> way to go.)

Leaving a feature waiting on a branch for extended period of time
imposes a lot of extra work to keep it up to date (and it can very
discouraging to have to do that if there's no clear set of "things
missing").


        Stefan

Re: Merging native-comp and pgtk

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: emacs-devel@gnu.org
> Date: Fri, 12 Feb 2021 18:04:10 -0500
> 
> >> Where are we w.r.t merging those two branches into `master`?
> >> IIUC the plan is to include those features as "experimental" in
> >> Emacs-28.1, right?  If so, I think it's becoming urgent that we merge
> >> them into `master`.
> > I don't thin it's ready yet, see my other message.
> 
> By "it" you mean both features?

No, I meant pgtk.

> So now I'm wondering: what do you think should be the criteria for
> inclusion into `master`?

In general?  I'm not sure I want to entertain such a discussion, as it
runs a high risk of triggering yet another long dispute.

> As written above, I thought the plan was to include those as
> experimental features for Emacs-28.1

IMO, it makes little sense having these features as experimental in
Emacs 28.1, because that would mean their promotion to mainstream
would be delayed till Emacs 29, and that's too far away for such
important features.  I think we should release them in Emacs 28.1 as
fully supported ones.  Yes, that could delay Emacs 28.1 a bit, but I
think users will want these two features badly enough to justify such
a delay (if it indeed happens).

> so I thought the criteria were
> going to be something like:
> - Code is clean enough: doesn't risk introducing regressions into the rest
>   of the code.
> - It's very likely that the feature will reach maturity (i.e. lose
>   the "experimental" label) in some not too distant future.
> - It's already usable enough that most people who're looking forward to
>   this feature will be fairly satisfied if they try it (it might still
>   have some rough edges, but by and large it works).

I'm not sure we want to codify the criteria, not in general, anyway.
We managed to do without any formal criteria until now, including when
you were the head maintainer.

> > (And why is it urgent?  Emacs 27.2 is not out yet, and Emacs 28 is
> > supposed to have native-comp included.  So we still have quite a long
> > way to go.)
> 
> Leaving a feature waiting on a branch for extended period of time
> imposes a lot of extra work to keep it up to date (and it can very
> discouraging to have to do that if there's no clear set of "things
> missing").

I agree with the principle, but its short summary is "as soon as
possible", not "urgently".  We should, of course, merge each of these
branches as soon as they are ready.

Re: Merging native-comp and pgtk

[Stefan Monnier]

>> So now I'm wondering: what do you think should be the criteria for
>> inclusion into `master`?
> In general?

No, I meant specifically (tho it's of course based on a general
principles, but these can change from case to case).

>> As written above, I thought the plan was to include those as
>> experimental features for Emacs-28.1
> IMO, it makes little sense having these features as experimental in
> Emacs 28.1, because that would mean their promotion to mainstream
> would be delayed till Emacs 29, and that's too far away for such
> important features.  I think we should release them in Emacs 28.1 as
> fully supported ones.  Yes, that could delay Emacs 28.1 a bit, but I
> think users will want these two features badly enough to justify such
> a delay (if it indeed happens).

Great, that clarifies part of the plan.  It also means it's that much
more important to merge them soon into `master`.

>> so I thought the criteria were
>> going to be something like:
>> - Code is clean enough: doesn't risk introducing regressions into the rest
>>   of the code.
>> - It's very likely that the feature will reach maturity (i.e. lose
>>   the "experimental" label) in some not too distant future.
>> - It's already usable enough that most people who're looking forward to
>>   this feature will be fairly satisfied if they try it (it might still
>>   have some rough edges, but by and large it works).
>
> I'm not sure we want to codify the criteria, not in general, anyway.

I'm sure we don't.
I was just drawing general lines that might apply to these two cases, to
clarify what I'm trying to find out.

>> Leaving a feature waiting on a branch for extended period of time
>> imposes a lot of extra work to keep it up to date (and it can very
>> discouraging to have to do that if there's no clear set of "things
>> missing").
> I agree with the principle, but its short summary is "as soon as
> possible", not "urgently".  We should, of course, merge each of these
> branches as soon as they are ready.

What is still unclear for me is what it is that's still "missing".


        Stefan

Re: Merging native-comp and pgtk

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: emacs-devel@gnu.org
> Date: Sat, 13 Feb 2021 08:24:19 -0500
> 
> > IMO, it makes little sense having these features as experimental in
> > Emacs 28.1, because that would mean their promotion to mainstream
> > would be delayed till Emacs 29, and that's too far away for such
> > important features.  I think we should release them in Emacs 28.1 as
> > fully supported ones.  Yes, that could delay Emacs 28.1 a bit, but I
> > think users will want these two features badly enough to justify such
> > a delay (if it indeed happens).
> 
> Great, that clarifies part of the plan.  It also means it's that much
> more important to merge them soon into `master`.

Yes, of course.  There's no argument here.

> What is still unclear for me is what it is that's still "missing".

That was answered in other messages, I believe.

Re: Merging native-comp and pgtk

[Andrea Corallo via Emacs development discussions.]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> BTW,
>
> Where are we w.r.t merging those two branches into `master`?
>
> IIUC the plan is to include those features as "experimental" in
> Emacs-28.1, right?  If so, I think it's becoming urgent that we merge
> them into `master`.

Hi Stefan,

my take on the native-comp branch status and some retrospective:

  - This is under development in the emacs.git since more than a year
    (~2y of total dev time, most of my holidays evenings and weekends,
    ~1350 commits).

  - Is, at present, used by literally hundreds of users (see Emacs
    survey).  I believe this should be prove of its stability usability
    and maturity.  I think we can say (seeing the quantity and content
    of feed-backs on the web) that at least on GNU/Linux the experience
    is typically quite slick.

  - Runs on a variety of OSes other than GNU/Linux including: MacOS,
    Windows, various BSDs.

  - A surprising number of packages, configuration kits, distros or other
    members of the ecosystem already provide some kind of integration
    with it.

  - No miscompilation bugs has been found since a while.  IOW
    native-comp is not reported to fail running or compiling any
    specific package.

  - ~6 months ago I was asked by Eli to open a bug [1] to handle code
    review and merge process.  As of today ~3 months more passed since
    my last try to ping this, unfortunately no progress was made.

    Each day the code is not on master we lose some verification
    opportunity.

  - I'm regularly hit by the burden of maintaining an out of tree
    feature branch, not only by conflicts in merge commits (that I do on
    a regular basis) but also by functional breakages.

    Ex: this week my hobby development time budget was wiped-out by
    having to manually bisect the commit that once merged from master
    was breaking the build plus writing the fix.  At least the first
    step was avoidable with the code on master and the emba CI.

This is indeed very demotivational for me and if I can say IMHO probably
not even completely respectful for all the effort that was spent into.

Aside from my personal feelings I'm sorry to report this my other
opinion: I'm convinced that if a Free Software project cannot review new
features is just *not* showing a good health status.

With all the respect, I'm convinced that if the maintainer has no time
for reviewing just means more maintainers are needed.  Not that
development should stop, slow down or the burden should be paid by
contributors.  This is just not an efficient development model.

Best Regards

  Andrea

[1] <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=43725>

Re: Merging native-comp and pgtk

[Eli Zaretskii]

> Cc: emacs-devel <emacs-devel@gnu.org>
> Date: Fri, 12 Feb 2021 21:47:15 +0000
> From:  Andrea Corallo via "Emacs development discussions." <emacs-devel@gnu.org>
> 
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
> 
> This is indeed very demotivational for me and if I can say IMHO probably
> not even completely respectful for all the effort that was spent into.
> 
> Aside from my personal feelings I'm sorry to report this my other
> opinion: I'm convinced that if a Free Software project cannot review new
> features is just *not* showing a good health status.
> 
> With all the respect, I'm convinced that if the maintainer has no time
> for reviewing just means more maintainers are needed.  Not that
> development should stop, slow down or the burden should be paid by
> contributors.  This is just not an efficient development model.

I apologize for not being able to find more time to devote to the
native-comp branch.  (I did work on it silently by preparing my main
development system to be ready for building the branch, which turned
out to be a non-trivial project, for reasons that will remain unsaid,
as they aren't interesting to anyone but myself.  I also try to speak
up when issues emerge on which I have something useful to say.)

Thank you for your perseverance and all the hard work you invest in
this important feature.  Please don't give up.  We _will_ merge this
branch soon, I hope.

Re: Merging native-comp and pgtk

[Lars Ingebrigtsen]

Andrea Corallo via "Emacs development discussions."
<emacs-devel@gnu.org> writes:

>   - ~6 months ago I was asked by Eli to open a bug [1] to handle code
>     review and merge process.  As of today ~3 months more passed since
>     my last try to ping this, unfortunately no progress was made.
>
>     Each day the code is not on master we lose some verification
>     opportunity.

I've got one question about how things are compiled that I meant to ask
but forgot: Why is AOT not the default, and why does the native-comp
branch do compilation on-the-fly in the background?

Neither seems natural for Emacs, I think?  I mean, we've never done
opportunistic .elc compilation -- we build Emacs, and then we use
whatever has been compiled (or not).  So I'm not sure how introducing
native code should make Emacs start compiling stuff in the background
when running Emacs normally.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Merging native-comp and pgtk

[Tassilo Horn]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Andrea Corallo via "Emacs development discussions."
> <emacs-devel@gnu.org> writes:
>
>>   - ~6 months ago I was asked by Eli to open a bug [1] to handle code
>>     review and merge process.  As of today ~3 months more passed since
>>     my last try to ping this, unfortunately no progress was made.
>>
>>     Each day the code is not on master we lose some verification
>>     opportunity.
>
> I've got one question about how things are compiled that I meant to
> ask but forgot: Why is AOT not the default, and why does the
> native-comp branch do compilation on-the-fly in the background?

Given that the ELN compilation takes quite some time, I think that's one
major reason.  Another thing is that the on-the-fly compilation also
compiles all ELPA packages and even packages just residing somewhere on
my customized load-path which aren't accessible at emacs build time.
And it seems like they are compiled not until they are actually used,
e.g., auctex compilation was not triggered until I found a tex file.  So
one only pays the price of compilation for the things one actually uses.

Bye,
Tassilo

Re: Merging native-comp and pgtk

[Lars Ingebrigtsen]

Tassilo Horn <tsdh@gnu.org> writes:

> Given that the ELN compilation takes quite some time, I think that's one
> major reason.  Another thing is that the on-the-fly compilation also
> compiles all ELPA packages and even packages just residing somewhere on
> my customized load-path which aren't accessible at emacs build time.

That's nice, but it's unusual for Emacs to do something like that.

When installing an ELPA package, Emacs byte-compiles the files, right?
So it should do .eln compilation at the same time, in my opinion.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Merging native-comp and pgtk

[Dmitry Gutov]

On 13.02.2021 13:57, Lars Ingebrigtsen wrote:
> When installing an ELPA package, Emacs byte-compiles the files, right?
> So it should do .eln compilation at the same time, in my opinion.

For ELPA packages, this might make sense (even though, with your 
proposed scheme, one would have to reinstall all packages on an existing 
system to take advantage of native compilation, right?).

But when we're talking about built-in packages, that might increase the 
build time with no good reason. E.g. if I never use Org or Gnus, ideally 
I shouldn't have to wait for them to be native-compiled.

Native compilation differs from the normal byte-compilation in speed, I 
guess.

Re: Merging native-comp and pgtk

[Lars Ingebrigtsen]

Dmitry Gutov <dgutov@yandex.ru> writes:

> For ELPA packages, this might make sense (even though, with your
> proposed scheme, one would have to reinstall all packages on an
> existing system to take advantage of native compilation, right?).

Yeah, I guess...

> But when we're talking about built-in packages, that might increase
> the build time with no good reason. E.g. if I never use Org or Gnus,
> ideally I shouldn't have to wait for them to be native-compiled.
>
> Native compilation differs from the normal byte-compilation in speed,
> I guess.

The primary way Emacs is distributed is already-compiled, and these will
presumably have .eln files pre-installed.  I think it makes some sense
in having the development version of Emacs behave more like what users
see normally, which means defaulting to compiling .eln files when saying
"make".

But it would indeed make sense to be able to inhibit it if your build
machine isn't super-spiffy.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Merging native-comp and pgtk

[Dmitry Gutov]

On 13.02.2021 16:53, Lars Ingebrigtsen wrote:
> The primary way Emacs is distributed is already-compiled, and these will
> presumably have .eln files pre-installed.  I think it makes some sense
> in having the development version of Emacs behave more like what users
> see normally, which means defaulting to compiling .eln files when saying
> "make".

But the actual people who compile Emacs for distribution are the 
minority, aren't they? So it might make sense to just document the 
advice to do full AOT when compiling for distribution, but default to 
lazy compilation.

> But it would indeed make sense to be able to inhibit it if your build
> machine isn't super-spiffy.

Or, you know, if you update the code often enough and don't want to 
waste time on native-recompiling the world every time.

Also, the lazy compilation approach seems to be the trickiest to get 
right, so I'd keep it the default even just to expose it to as many 
users as possible, to iron out whatever kinds could hide there.

Re: Merging native-comp and pgtk

[Lars Ingebrigtsen]

Dmitry Gutov <dgutov@yandex.ru> writes:

> But the actual people who compile Emacs for distribution are the
> minority, aren't they? So it might make sense to just document the
> advice to do full AOT when compiling for distribution, but default to
> lazy compilation.

Then developers would (commonly) be running a different thing than most
users...

>> But it would indeed make sense to be able to inhibit it if your build
>> machine isn't super-spiffy.
>
> Or, you know, if you update the code often enough and don't want to
> waste time on native-recompiling the world every time.

Does AOT need really need to recompile everything every time temacs is
built, though (as somebody said)?

> Also, the lazy compilation approach seems to be the trickiest to get
> right, so I'd keep it the default even just to expose it to as many
> users as possible, to iron out whatever kinds could hide there.

Sure, I have no problem with that.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Merging native-comp and pgtk

[Dmitry Gutov]

On 13.02.2021 23:03, Lars Ingebrigtsen wrote:
> Dmitry Gutov <dgutov@yandex.ru> writes:
> 
>> But the actual people who compile Emacs for distribution are the
>> minority, aren't they? So it might make sense to just document the
>> advice to do full AOT when compiling for distribution, but default to
>> lazy compilation.
> 
> Then developers would (commonly) be running a different thing than most
> users...

There is a certain subset of our users who routinely compile from 
master, but don't really contribute to the development.

For instance, see the Homebrew stats here for 'emacs --HEAD':

https://formulae.brew.sh/formula/emacs

(I don't really have any numbers across platforms, this is just the 
first thing I managed to dig up).

>>> But it would indeed make sense to be able to inhibit it if your build
>>> machine isn't super-spiffy.
>>
>> Or, you know, if you update the code often enough and don't want to
>> waste time on native-recompiling the world every time.
> 
> Does AOT need really need to recompile everything every time temacs is
> built, though (as somebody said)?

Hmm, maybe I spoke too strongly. I haven't tried a full AOT build myself 
yet, actually (Stefan showed some numbers on that), but even the current 
"lazy" approach feels quite slow to me.

So if you have ever been annoyed with the current recompilation speed 
after 'git pull' (even just of the files that do need to be recompiled), 
you can multiply that time by X.

Re: Merging native-comp and pgtk

[Lars Ingebrigtsen]

Dmitry Gutov <dgutov@yandex.ru> writes:

> So if you have ever been annoyed with the current recompilation speed
> after 'git pull' (even just of the files that do need to be
> recompiled), you can multiply that time by X.

I'm very seldom annoyed by that, because it's usually all the .c files
that are recompiled en masse.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Merging native-comp and pgtk

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Dmitry Gutov <dgutov@yandex.ru> writes:
>
>> So if you have ever been annoyed with the current recompilation speed
>> after 'git pull' (even just of the files that do need to be
>> recompiled), you can multiply that time by X.
>
> I'm very seldom annoyed by that, because it's usually all the .c files
> that are recompiled en masse.

Anyway, it's been some time since I've timed the builds...  on my build
machine (an 8 core AMD machine), trunk takes 2m52s to do a "make
bootstrap", while a full AOT build in the native-comp branch takes
10m15s.  So the native build takes a bit more than 3x as long there.

Which is less than I had assumed, really.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Merging native-comp and pgtk

[Tassilo Horn]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Anyway, it's been some time since I've timed the builds...  on my
> build machine (an 8 core AMD machine), trunk takes 2m52s to do a "make
> bootstrap", while a full AOT build in the native-comp branch takes
> 10m15s.

Impressive, I can make a pot of tea, let it steep, and drink it during
the non-native bootstrap.  For the native one, I can also bake some
cake. :-)

> So the native build takes a bit more than 3x as long there.  Which is
> less than I had assumed, really.

I think you can increase your tea & cake time by raising the
optimization level.

Bye,
Tassilo

Re: Merging native-comp and pgtk

[Stefan Kangas]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> I've got one question about how things are compiled that I meant to ask
> but forgot: Why is AOT not the default, and why does the native-comp
> branch do compilation on-the-fly in the background?

For me, it is a happy middle-ground: I do not need to make
NATIVE_FULL_AOT=1 (which is slower) but can still rely on getting the
performance increase automagically.

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Tassilo Horn <tsdh@gnu.org> writes:
>
>> Given that the ELN compilation takes quite some time, I think that's one
>> major reason.  Another thing is that the on-the-fly compilation also
>> compiles all ELPA packages and even packages just residing somewhere on
>> my customized load-path which aren't accessible at emacs build time.
>
> That's nice, but it's unusual for Emacs to do something like that.
>
> When installing an ELPA package, Emacs byte-compiles the files, right?
> So it should do .eln compilation at the same time, in my opinion.

It would have some benefits I guess, but it would slow down package
installation.  Whatever we do, it would be nice to avoid that.

On the nativecomp branch, package.el requests to do the .eln compilation
asynchronously in the background.  AFAIU, if Emacs is killed before the
package is compiled, it will be put back in the compilation queue after
the package is used in a subsequent session.

The drawback is that we get compilation warnings pop up every now and
then, which is arguably somewhat disruptive.  But this could be improved
upon, I think.  For example, I'm not sure users need to see all warnings
by default for installed packages even in the byte-compilation case.  We
could perhaps have an option to enable these warnings for the genuine
sticklers out there.  (I for one would definitely turn it on...)

Re: Merging native-comp and pgtk

[Andrea Corallo via Emacs development discussions.]

Stefan Kangas <stefankangas@gmail.com> writes:

> Lars Ingebrigtsen <larsi@gnus.org> writes:
>
>> I've got one question about how things are compiled that I meant to ask
>> but forgot: Why is AOT not the default, and why does the native-comp
>> branch do compilation on-the-fly in the background?
>
> For me, it is a happy middle-ground: I do not need to make
> NATIVE_FULL_AOT=1 (which is slower) but can still rely on getting the
> performance increase automagically.
>
> Lars Ingebrigtsen <larsi@gnus.org> writes:
>
>> Tassilo Horn <tsdh@gnu.org> writes:
>>
>>> Given that the ELN compilation takes quite some time, I think that's one
>>> major reason.  Another thing is that the on-the-fly compilation also
>>> compiles all ELPA packages and even packages just residing somewhere on
>>> my customized load-path which aren't accessible at emacs build time.
>>
>> That's nice, but it's unusual for Emacs to do something like that.
>>
>> When installing an ELPA package, Emacs byte-compiles the files, right?
>> So it should do .eln compilation at the same time, in my opinion.
>
> It would have some benefits I guess, but it would slow down package
> installation.  Whatever we do, it would be nice to avoid that.
>
> On the nativecomp branch, package.el requests to do the .eln compilation
> asynchronously in the background.  AFAIU, if Emacs is killed before the
> package is compiled, it will be put back in the compilation queue after
> the package is used in a subsequent session.
>
> The drawback is that we get compilation warnings pop up every now and
> then, which is arguably somewhat disruptive.  But this could be improved
> upon, I think.  For example, I'm not sure users need to see all warnings
> by default for installed packages even in the byte-compilation case.  We
> could perhaps have an option to enable these warnings for the genuine
> sticklers out there.  (I for one would definitely turn it on...)

Hi Stefan,

we have the `comp-async-report-warnings-errors' customize to control
this.

  Andrea

Re: Merging native-comp and pgtk

[Stefan Kangas]

Andrea Corallo <akrl@sdf.org> writes:

>> The drawback is that we get compilation warnings pop up every now and
>> then, which is arguably somewhat disruptive.  But this could be improved
>> upon, I think.  For example, I'm not sure users need to see all warnings
>> by default for installed packages even in the byte-compilation case.  We
>> could perhaps have an option to enable these warnings for the genuine
>> sticklers out there.  (I for one would definitely turn it on...)
>
> Hi Stefan,
>
> we have the `comp-async-report-warnings-errors' customize to control
> this.

Should that perhaps default to nil?  I'm not sure users typically need
to see these warnings -- developers do.

Re: Merging native-comp and pgtk

[Andrea Corallo via Emacs development discussions.]

Stefan Kangas <stefankangas@gmail.com> writes:

> Andrea Corallo <akrl@sdf.org> writes:
>
>>> The drawback is that we get compilation warnings pop up every now and
>>> then, which is arguably somewhat disruptive.  But this could be improved
>>> upon, I think.  For example, I'm not sure users need to see all warnings
>>> by default for installed packages even in the byte-compilation case.  We
>>> could perhaps have an option to enable these warnings for the genuine
>>> sticklers out there.  (I for one would definitely turn it on...)
>>
>> Hi Stefan,
>>
>> we have the `comp-async-report-warnings-errors' customize to control
>> this.
>
> Should that perhaps default to nil?  I'm not sure users typically need
> to see these warnings -- developers do.

In the past a lot of developers not seing these warnings on native-comp
got hit by this.

I think we touched on the reasons why hiding warnings on native-comp is
dangerous here <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=44746#8>.

So yeah I agree with you normal users are not interested, but developers
should learn that this customize exists and set it to t, this is not
100% trivial.

Anyway I guess as every default the best value is very debatable :)

  Andrea

Re: Merging native-comp and pgtk

[Óscar Fuentes]

Andrea Corallo via "Emacs development discussions."
<emacs-devel@gnu.org> writes:

> Stefan Kangas <stefankangas@gmail.com> writes:
>
>> Andrea Corallo <akrl@sdf.org> writes:
>>
>>>> The drawback is that we get compilation warnings pop up every now and
>>>> then, which is arguably somewhat disruptive.  But this could be improved
>>>> upon, I think.  For example, I'm not sure users need to see all warnings
>>>> by default for installed packages even in the byte-compilation case.  We
>>>> could perhaps have an option to enable these warnings for the genuine
>>>> sticklers out there.  (I for one would definitely turn it on...)
>>>
>>> Hi Stefan,
>>>
>>> we have the `comp-async-report-warnings-errors' customize to control
>>> this.

Oh, some weeks ago I put

(setq warning-minimum-level :error)

on my .emacs, but it turns out that there is something to address the
real culprit :-)

>> Should that perhaps default to nil?  I'm not sure users typically need
>> to see these warnings -- developers do.
>
> In the past a lot of developers not seing these warnings on native-comp
> got hit by this.
>
> I think we touched on the reasons why hiding warnings on native-comp is
> dangerous here <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=44746#8>.
>
> So yeah I agree with you normal users are not interested, but developers
> should learn that this customize exists and set it to t, this is not
> 100% trivial.
>
> Anyway I guess as every default the best value is very debatable :)

The problem is that you execute a command and some time later a window
pops out showing some stuff which is uninsteresting or even
unintelligible for the user. And that happens too often, in my
experience. It's quite annoying.

At this stage having comp-async-report-warnings-errors t is probably the
correct thing, but on the release it should be nil, no doubt about it
:-)

Re: Merging native-comp and pgtk

[Andrea Corallo via Emacs development discussions.]

Óscar Fuentes <ofv@wanadoo.es> writes:

[...]

>> In the past a lot of developers not seing these warnings on native-comp
>> got hit by this.
>>
>> I think we touched on the reasons why hiding warnings on native-comp is
>> dangerous here <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=44746#8>.
>>
>> So yeah I agree with you normal users are not interested, but developers
>> should learn that this customize exists and set it to t, this is not
>> 100% trivial.
>>
>> Anyway I guess as every default the best value is very debatable :)
>
> The problem is that you execute a command and some time later a window
> pops out showing some stuff which is uninsteresting or even
> unintelligible for the user. And that happens too often, in my
> experience. It's quite annoying.
>
> At this stage having comp-async-report-warnings-errors t is probably the
> correct thing, but on the release it should be nil, no doubt about it
> :-)

Fair :)

Re: Merging native-comp and pgtk

[Stefan Monnier]

> I've got one question about how things are compiled that I meant to ask
> but forgot: Why is AOT not the default, and why does the native-comp
> branch do compilation on-the-fly in the background?

There are some important differences between compiling to byte-code and
compiling to native-code:

- Compiling to byte-code is not transparent.  The resulting code may
  behave differently in the two cases, there are questions like
  `load-prefer-newer`, the compilation itself is useful to get feedback
  via warnings (or even failures), the compilation can also have all
  kinds of side-effects (like an `eval-when-compile` which triggers
  compilation of something else...), the `load-history` will tell you
  which of .el or .elc you loaded, ...
- Compilation of byte-code is fast.
- Byte-code enjoys backward compatibility.

None of those apply to native-code compilation.  On an under-powered
machine like a BananaPi byte-code compilation of the whole of Emacs
takes quite a while (order of magnitude around 1h).  It's a pain but
it's bearable.  For native-code compilation the order of magnitude is
a whole day, which I don't find bearable, especially since all those
.eln files need to be rebuilt as soon as I have recompiled a new
`temacs`!

Lazy compilation is not a natural by-product of the way native-comp
works: it's an extra feature that was added manually because it proved
very useful.  It can be turned off of course.


        Stefan

Re: Merging native-comp and pgtk

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Sat, 13 Feb 2021 08:41:45 -0500
> Cc: Andrea Corallo <akrl@sdf.org>,
>  "Andrea Corallo via Emacs development discussions." <emacs-devel@gnu.org>
> 
> None of those apply to native-code compilation.  On an under-powered
> machine like a BananaPi byte-code compilation of the whole of Emacs
> takes quite a while (order of magnitude around 1h).  It's a pain but
> it's bearable.  For native-code compilation the order of magnitude is
> a whole day, which I don't find bearable, especially since all those
> .eln files need to be rebuilt as soon as I have recompiled a new
> `temacs`!
> 
> Lazy compilation is not a natural by-product of the way native-comp
> works: it's an extra feature that was added manually because it proved
> very useful.  It can be turned off of course.

The question was whether it should be ON by default.  If the problem
with that happens on slow machines, we should consider an
alternative: have it OFF by default and ask users of slow machines to
turn this ON.

Re: Merging native-comp and pgtk

[Dmitry Gutov]

On 13.02.2021 16:18, Eli Zaretskii wrote:
> The question was whether it should be ON by default.  If the problem
> with that happens on slow machines, we should consider an
> alternative: have it OFF by default and ask users of slow machines to
> turn this ON.

I have a fast machine, and it feels pretty slow to me.

Everything is relative.

Re: Merging native-comp and pgtk

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 13 Feb 2021 22:32:09 +0200
> Cc: larsi@gnus.org, akrl@sdf.org, emacs-devel@gnu.org
> 
> On 13.02.2021 16:18, Eli Zaretskii wrote:
> > The question was whether it should be ON by default.  If the problem
> > with that happens on slow machines, we should consider an
> > alternative: have it OFF by default and ask users of slow machines to
> > turn this ON.
> 
> I have a fast machine, and it feels pretty slow to me.
> 
> Everything is relative.

We are not talking about personal settings, those are always possible,
no matter what are the defaults.

Re: Merging native-comp and pgtk

[Stefan Monnier]

> The question was whether it should be ON by default.  If the problem
> with that happens on slow machines, we should consider an
> alternative: have it OFF by default and ask users of slow machines to
> turn this ON.

I don't think we'll be able to make such decisions before the code is on
`master` and we have feedback from a lot more people (and not just
those interested in the feature).


        Stefan

Re: Merging native-comp and pgtk

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> There are some important differences between compiling to byte-code and
> compiling to native-code:
>
> - Compiling to byte-code is not transparent.  The resulting code may
>   behave differently in the two cases, there are questions like
>   `load-prefer-newer`, the compilation itself is useful to get feedback
>   via warnings (or even failures), the compilation can also have all
>   kinds of side-effects (like an `eval-when-compile` which triggers
>   compilation of something else...), the `load-history` will tell you
>   which of .el or .elc you loaded, ...

.eln isn't fully transparent either -- you get different backtraces etc
from compiled code than byte-compiled code, for instance.

> - Compilation of byte-code is fast.

That's the main point for not switching it on, I guess.

> - Byte-code enjoys backward compatibility.

Very limited backwards compatibility -- we've introduced new byte codes
in Emacs 28, for instance.  It's not a goal, but when it happens it's
nice.

Byte-code enjoys sideways compatibility, though -- one can use the same
.elc files on both ARM and AMD systems, but that's definitely not true
for .eln files.  But...  I'm not sure that's much of a consideration
either -- the rest of Emacs has to be built for a specific architecture,
so...

> None of those apply to native-code compilation.  On an under-powered
> machine like a BananaPi byte-code compilation of the whole of Emacs
> takes quite a while (order of magnitude around 1h).  It's a pain but
> it's bearable.  For native-code compilation the order of magnitude is
> a whole day, which I don't find bearable, especially since all those
> .eln files need to be rebuilt as soon as I have recompiled a new
> `temacs`!

Oh, they have to be?  I didn't know that.  Why?  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Merging native-comp and pgtk

[Andrea Corallo via Emacs development discussions.]

Lars Ingebrigtsen <larsi@gnus.org> writes:

>> None of those apply to native-code compilation.  On an under-powered
>> machine like a BananaPi byte-code compilation of the whole of Emacs
>> takes quite a while (order of magnitude around 1h).  It's a pain but
>> it's bearable.  For native-code compilation the order of magnitude is
>> a whole day, which I don't find bearable, especially since all those
>> .eln files need to be rebuilt as soon as I have recompiled a new
>> `temacs`!
>
> Oh, they have to be?  I didn't know that.  Why?  :-)

AFAIK they don't have to, I might be missing something here.

  Andrea

Re: Merging native-comp and pgtk

[Andrea Corallo via Emacs development discussions.]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Andrea Corallo via "Emacs development discussions."
> <emacs-devel@gnu.org> writes:
>
>>   - ~6 months ago I was asked by Eli to open a bug [1] to handle code
>>     review and merge process.  As of today ~3 months more passed since
>>     my last try to ping this, unfortunately no progress was made.
>>
>>     Each day the code is not on master we lose some verification
>>     opportunity.
>
> I've got one question about how things are compiled that I meant to ask
> but forgot: Why is AOT not the default, and why does the native-comp
> branch do compilation on-the-fly in the background?

So I was asked to set it as default in order to limit the build time, I
guess the main point was about thinking to people building from source.

IMO I think too is a good default, distros can easily learn how to build
AOT.

Regards

  Andrea

Re: Cleaning out old X11 toolkits?

[Eli Zaretskii]

> Cc: chad <yandros@gmail.com>, Yuan Fu <casouri@gmail.com>,
>  Alan Third <alan@idiocy.org>, emacs-devel <emacs-devel@gnu.org>,
>  "Basil L. Contovounesios" <contovob@tcd.ie>,
>  Stefan Monnier <monnier@iro.umontreal.ca>, Lars Ingebrigtsen
>  <larsi@gnus.org>, Eli Zaretskii <eliz@gnu.org>
> From: martin rudalics <rudalics@gmx.at>
> Date: Fri, 12 Feb 2021 18:56:05 +0100
> 
>  > If thatʼs what we want, then we should merge pgtk to master and ask
>  > people to test it.
> 
> Why should people "test" it more when it's been merged?

Because presumably building a pgtk port will need some non-default
switch to the configure script?  And if no one uses that switch, the
merged code will remain unused?

>  > Works fine for me here with a GTK build, although my frame ends up
>  > with a height of 30. pgtk generally has issues with frame sizing and
>  > positioning.
> 
> We might be in for some rude awakening here.

I think those positioning issues need to be fixed before we will agree
to merge the branch.

Re: Cleaning out old X11 toolkits?

[martin rudalics]

 >> Why should people "test" it more when it's been merged?
 >
 > Because presumably building a pgtk port will need some non-default
 > switch to the configure script?  And if no one uses that switch, the
 > merged code will remain unused?

I'm confused.  How would that switch be related to the question whether
people test anything more when it's been merged in than when it's on a
separate branch?  They need the switch in either case.

 > I think those positioning issues need to be fixed before we will agree
 > to merge the branch.

If no one tests them how will they get fixed?  Isn't that a dog chasing
its tail?  Moreover, I suppose that most people currently testing pgtk
are on Wayland, so user experiences might vary widely.

martin

Re: Cleaning out old X11 toolkits?

[Eli Zaretskii]

> Cc: casouri@gmail.com, rpluim@gmail.com, alan@idiocy.org,
>  emacs-devel@gnu.org, contovob@tcd.ie, monnier@iro.umontreal.ca,
>  yandros@gmail.com, larsi@gnus.org
> From: martin rudalics <rudalics@gmx.at>
> Date: Sat, 13 Feb 2021 10:01:14 +0100
> 
>  >> Why should people "test" it more when it's been merged?
>  >
>  > Because presumably building a pgtk port will need some non-default
>  > switch to the configure script?  And if no one uses that switch, the
>  > merged code will remain unused?
> 
> I'm confused.  How would that switch be related to the question whether
> people test anything more when it's been merged in than when it's on a
> separate branch?  They need the switch in either case.

But using the switch is much easier than checking out a separate
branch, is it not?

>  > I think those positioning issues need to be fixed before we will agree
>  > to merge the branch.
> 
> If no one tests them how will they get fixed?

They were already reported, so we don't need any more testing to know
these problems exist.

Re: Cleaning out old X11 toolkits?

[martin rudalics]

 >> I'm confused.  How would that switch be related to the question whether
 >> people test anything more when it's been merged in than when it's on a
 >> separate branch?  They need the switch in either case.
 >
 > But using the switch is much easier than checking out a separate
 > branch, is it not?

Not for me.  And if checking out a separate branch were difficult, where
would we get the "hundreds of users" of the native-comp branch from?

 >> If no one tests them how will they get fixed?
 >
 > They were already reported,

.. with diverging symptoms ...

 > so we don't need any more testing to know
 > these problems exist.

We need a strategy for how to proceed.  Any such strategy needs as much
data as it can get.

martin

Re: Cleaning out old X11 toolkits?

[Robert Pluim]

>>>>> On Fri, 12 Feb 2021 20:32:31 +0200, Eli Zaretskii <eliz@gnu.org> said:

    >> Cc: chad <yandros@gmail.com>, Yuan Fu <casouri@gmail.com>,
    >> Alan Third <alan@idiocy.org>, emacs-devel <emacs-devel@gnu.org>,
    >> "Basil L. Contovounesios" <contovob@tcd.ie>,
    >> Stefan Monnier <monnier@iro.umontreal.ca>, Lars Ingebrigtsen
    >> <larsi@gnus.org>, Eli Zaretskii <eliz@gnu.org>
    >> From: martin rudalics <rudalics@gmx.at>
    >> Date: Fri, 12 Feb 2021 18:56:05 +0100
    >> 
    >> > If thatʼs what we want, then we should merge pgtk to master and ask
    >> > people to test it.
    >> 
    >> Why should people "test" it more when it's been merged?

    Eli> Because presumably building a pgtk port will need some non-default
    Eli> switch to the configure script?  And if no one uses that switch, the
    Eli> merged code will remain unused?

right. And not everyone is going to want to go to the trouble of
switching branches in git or creating a second workspace, so having a
configure options just makes their lives easier. In fact, perhaps it
should be

--with-x-toolkit=pgtk

(even though it works for Wayland as well)

    >> > Works fine for me here with a GTK build, although my frame ends up
    >> > with a height of 30. pgtk generally has issues with frame sizing and
    >> > positioning.
    >> 
    >> We might be in for some rude awakening here.

    Eli> I think those positioning issues need to be fixed before we will agree
    Eli> to merge the branch.

Iʼm not sure what we can do: under Wayland the window manager I have
ignores programmatic requests to change frame positions, and this is
apparently by design.

Hmm, specifying *initial* frame sizes and positions seems to work, but
only if done early enough, such as by setting 'default-frame-alist' in
early-init.el. I guess worst case we can destroy frames and recreate
them instead of moving them. Someone (Martin?) suggested creating an
invisible max-size top-level frame that we could use as the parent for
all other frames on pgtk, since pgtk does allow us to move and resize
child-frames. I donʼt know if thatʼs actually possible.

Robert

Re: Cleaning out old X11 toolkits?

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: martin rudalics <rudalics@gmx.at>,  yandros@gmail.com,
>   casouri@gmail.com,  alan@idiocy.org,  emacs-devel@gnu.org,
>   contovob@tcd.ie,  monnier@iro.umontreal.ca,  larsi@gnus.org
> Date: Sun, 14 Feb 2021 13:50:23 +0100
> 
>     Eli> I think those positioning issues need to be fixed before we will agree
>     Eli> to merge the branch.
> 
> Iʼm not sure what we can do: under Wayland the window manager I have
> ignores programmatic requests to change frame positions, and this is
> apparently by design.
> 
> Hmm, specifying *initial* frame sizes and positions seems to work, but
> only if done early enough, such as by setting 'default-frame-alist' in
> early-init.el. I guess worst case we can destroy frames and recreate
> them instead of moving them. Someone (Martin?) suggested creating an
> invisible max-size top-level frame that we could use as the parent for
> all other frames on pgtk, since pgtk does allow us to move and resize
> child-frames. I donʼt know if thatʼs actually possible.

If it's feasible to fix these issues, we should try doing that, so
that Emacs behaves the same as with other toolkits; controlling the
position and geometry of a frame is pretty important, I think.

But if it turns out infeasible to do that on Wayland, we should
document that fact and move on.

Re: Cleaning out old X11 toolkits?

[Robert Pluim]

>>>>> On Sun, 14 Feb 2021 17:53:29 +0200, Eli Zaretskii <eliz@gnu.org> said:

    >> Hmm, specifying *initial* frame sizes and positions seems to work, but
    >> only if done early enough, such as by setting 'default-frame-alist' in
    >> early-init.el. I guess worst case we can destroy frames and recreate
    >> them instead of moving them. Someone (Martin?) suggested creating an
    >> invisible max-size top-level frame that we could use as the parent for
    >> all other frames on pgtk, since pgtk does allow us to move and resize
    >> child-frames. I donʼt know if thatʼs actually possible.

    Eli> If it's feasible to fix these issues, we should try doing that, so
    Eli> that Emacs behaves the same as with other toolkits; controlling the
    Eli> position and geometry of a frame is pretty important, I think.

    Eli> But if it turns out infeasible to do that on Wayland, we should
    Eli> document that fact and move on.

It turns out to be worse than I thought: the pgtk build has problems
with 'width and 'height on the initial frame whether the backend is
XWayland or Wayland (I haven't tested with a pure X11 window
manager). Specifying the position/size of a new frame seems to work ok
though.

More generally, fixing this is not going to be trivial, as
gtk_window_move under Wayland is designed to cause us grief:

<https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=940916>

so one way to get eg desktop.el to work is to ensure that we know the
required size and position of all the frames before we create them.

Robert

Re: Cleaning out old X11 toolkits?

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: casouri@gmail.com,  alan@idiocy.org,  emacs-devel@gnu.org,
>   contovob@tcd.ie,  rudalics@gmx.at,  monnier@iro.umontreal.ca,
>   yandros@gmail.com,  larsi@gnus.org
> Date: Mon, 15 Feb 2021 10:48:13 +0100
> 
> It turns out to be worse than I thought: the pgtk build has problems
> with 'width and 'height on the initial frame whether the backend is
> XWayland or Wayland (I haven't tested with a pure X11 window
> manager). Specifying the position/size of a new frame seems to work ok
> though.
> 
> More generally, fixing this is not going to be trivial, as
> gtk_window_move under Wayland is designed to cause us grief:
> 
> <https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=940916>

If this is the Wayland protocol, then users of Wayland will expect
that, and we don't have to solve this problem.  Right?

Re: Cleaning out old X11 toolkits?

[Robert Pluim]

>>>>> On Mon, 15 Feb 2021 17:23:44 +0200, Eli Zaretskii <eliz@gnu.org> said:

    >>Specifying the position/size of a new frame seems to work ok
    >> though.

...when using the XWayland backend.

    >> More generally, fixing this is not going to be trivial, as
    >> gtk_window_move under Wayland is designed to cause us grief:
    >> 
    >> <https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=940916>

    Eli> If this is the Wayland protocol, then users of Wayland will expect
    Eli> that, and we don't have to solve this problem.  Right?

I guess, although be prepared for a bunch of "desktop.el is broken"
bug reports, along with "my workflow with (make-frame '((top . 50)
(left . 50))) or (set-frame-position) doesnʼt work any more"
complaints as Wayland becomes more widespread.

(there is a 'session-manager' in GTK that you can allegedly use to
save and restore frame positions and sizes, but I canʼt find much
documentation about it. I suspect it would still fall foul of
Wayland's "there is no global coordinate system" mantra)

Robert

Re: Cleaning out old X11 toolkits?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1234 bytes --]

On Mon, Feb 15, 2021 at 05:02:22PM +0100, Robert Pluim wrote:
> >>>>> On Mon, 15 Feb 2021 17:23:44 +0200, Eli Zaretskii <eliz@gnu.org> said:
> 
>     >>Specifying the position/size of a new frame seems to work ok
>     >> though.
> 
> ...when using the XWayland backend.
> 
>     >> More generally, fixing this is not going to be trivial, as
>     >> gtk_window_move under Wayland is designed to cause us grief:
>     >> 
>     >> <https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=940916>
> 
>     Eli> If this is the Wayland protocol, then users of Wayland will expect
>     Eli> that, and we don't have to solve this problem.  Right?
> 
> I guess, although be prepared for a bunch of "desktop.el is broken"
> bug reports, along with "my workflow with (make-frame '((top . 50)
> (left . 50))) or (set-frame-position) doesnʼt work any more"
> complaints as Wayland becomes more widespread.

FWIW, on X the window manager also has the last word on top-level
window positioning.

The difference is that a user picking a WM making strong use of
that "last word" knows what she's doing (tiling WMs come to mind).

Wayland users, OTOH, haven't that choice, possibly they haven't
the insight.

Cheers
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Cleaning out old X11 toolkits?

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: casouri@gmail.com,  alan@idiocy.org,  emacs-devel@gnu.org,
>   contovob@tcd.ie,  rudalics@gmx.at,  monnier@iro.umontreal.ca,
>   yandros@gmail.com,  larsi@gnus.org
> Date: Mon, 15 Feb 2021 17:02:22 +0100
> 
>     Eli> If this is the Wayland protocol, then users of Wayland will expect
>     Eli> that, and we don't have to solve this problem.  Right?
> 
> I guess, although be prepared for a bunch of "desktop.el is broken"
> bug reports, along with "my workflow with (make-frame '((top . 50)
> (left . 50))) or (set-frame-position) doesnʼt work any more"
> complaints as Wayland becomes more widespread.
> 
> (there is a 'session-manager' in GTK that you can allegedly use to
> save and restore frame positions and sizes, but I canʼt find much
> documentation about it. I suspect it would still fall foul of
> Wayland's "there is no global coordinate system" mantra)

Is there a way for the application such as Emacs to ask the
"compositor" to locate the application's window?

Re: Cleaning out old X11 toolkits?

[Robert Pluim]

>>>>> On Mon, 15 Feb 2021 19:03:57 +0200, Eli Zaretskii <eliz@gnu.org> said:

    >> (there is a 'session-manager' in GTK that you can allegedly use to
    >> save and restore frame positions and sizes, but I canʼt find much
    >> documentation about it. I suspect it would still fall foul of
    >> Wayland's "there is no global coordinate system" mantra)

    Eli> Is there a way for the application such as Emacs to ask the
    Eli> "compositor" to locate the application's window?

From all Iʼve read, the attitude on the Wayland side is "weʼre going
to give you random frame positions when you create a frame, because
we've decided that itʼs not the application's business where they end
up. Oh, and you canʼt move them either. Global coordinates for the
display are anathema, deal with it."

I suspect I would not get on well with them :-)

Robert

Re: Cleaning out old X11 toolkits?

[Dmitry Gutov]

On 12.02.2021 12:00, martin rudalics wrote:
> I regularly build with Motif.  Here it's just as reliable as Lucid.

Is there are reason to keep Motif, if Lucid is available?

Any particular advantages to it?

Re: Cleaning out old X11 toolkits?

[martin rudalics]

 > Is there are reason to keep Motif, if Lucid is available?
 >
 > Any particular advantages to it?

On GNU/Linux it's the only build with a wrapping menu bar (like that of
Windows).  All other build either truncate the menu bar (Lucid) or may
refuse to shrink the frame when it would get less wide than the menu bar
(GTK3).

martin

Re: Cleaning out old X11 toolkits?

[Eli Zaretskii]

> Cc: Yuan Fu <casouri@gmail.com>, Alan Third <alan@idiocy.org>,
>  emacs-devel <emacs-devel@gnu.org>, "Basil L. Contovounesios"
>  <contovob@tcd.ie>, Stefan Monnier <monnier@iro.umontreal.ca>,
>  Lars Ingebrigtsen <larsi@gnus.org>, Eli Zaretskii <eliz@gnu.org>
> From: martin rudalics <rudalics@gmx.at>
> Date: Fri, 12 Feb 2021 18:56:20 +0100
> 
>  > Is there are reason to keep Motif, if Lucid is available?
>  >
>  > Any particular advantages to it?
> 
> On GNU/Linux it's the only build with a wrapping menu bar (like that of
> Windows).

Why is that important?  Wrapping menu bar is generally a sign of a
misconfigured frame.

Re: Cleaning out old X11 toolkits?

[martin rudalics]

 > Why is that important?  Wrapping menu bar is generally a sign of a
 > misconfigured frame.

Maybe it is.  But then why do we wrap our tool and tab bars?

martin

Re: Cleaning out old X11 toolkits?

[Eli Zaretskii]

> Cc: casouri@gmail.com, rpluim@gmail.com, alan@idiocy.org,
>  emacs-devel@gnu.org, contovob@tcd.ie, monnier@iro.umontreal.ca,
>  dgutov@yandex.ru, yandros@gmail.com, larsi@gnus.org
> From: martin rudalics <rudalics@gmx.at>
> Date: Fri, 12 Feb 2021 21:09:20 +0100
> 
>  > Why is that important?  Wrapping menu bar is generally a sign of a
>  > misconfigured frame.
> 
> Maybe it is.  But then why do we wrap our tool and tab bars?

Because it's a natural behavior of the Emacs display engine, I guess.

But my question actually meant to say why the fact that Motif can wrap
the menu is important enough to keep its support, if no one uses it?

Re: Cleaning out old X11 toolkits?

[martin rudalics]

 > But my question actually meant to say why the fact that Motif can wrap
 > the menu is important enough to keep its support,

Because on GNU/Linux this provides a means for testing resizing of the
inner frame under the most heavy environmental conditions: It has to
deal with a wrapping menu, tool and tab bar all in one, something I can
test only on Windows otherwise.

 > if no one uses it?

IIUC no one uses --with-x-toolkit=no or --without-x builds either, so
this argument doesn't count.  But why even care about removing things
that are not broken as long as

- there exist builds that are severely broken in many regards like our
   default GNU/Linux build,

- there exist largely untested builds like pgtk whose merging-in would
   in no way interfere with any of the toolkits whose support people
   despise,

- removing special Motif code might break other parts as well (we would
   have to continue to work with its WM hints, for example)?

martin

Re: Cleaning out old X11 toolkits?

[Eli Zaretskii]

> Cc: casouri@gmail.com, rpluim@gmail.com, alan@idiocy.org,
>  emacs-devel@gnu.org, contovob@tcd.ie, monnier@iro.umontreal.ca,
>  dgutov@yandex.ru, yandros@gmail.com, larsi@gnus.org
> From: martin rudalics <rudalics@gmx.at>
> Date: Sat, 13 Feb 2021 10:02:10 +0100
> 
>  > But my question actually meant to say why the fact that Motif can wrap
>  > the menu is important enough to keep its support,
> 
> Because on GNU/Linux this provides a means for testing resizing of the
> inner frame under the most heavy environmental conditions: It has to
> deal with a wrapping menu, tool and tab bar all in one, something I can
> test only on Windows otherwise.

If no configuration on GNU/Linux supports wrapping of the menu, then
this wrapping doesn't need to be tested on GNU/Linux, and is not an
argument for keeping the Motif support.

>  > if no one uses it?
> 
> IIUC no one uses --with-x-toolkit=no or --without-x builds either, so
> this argument doesn't count.  But why even care about removing things
> that are not broken as long as
> 
> - there exist builds that are severely broken in many regards like our
>    default GNU/Linux build,
> 
> - there exist largely untested builds like pgtk whose merging-in would
>    in no way interfere with any of the toolkits whose support people
>    despise,
> 
> - removing special Motif code might break other parts as well (we would
>    have to continue to work with its WM hints, for example)?

Good questions.

Re: Cleaning out old X11 toolkits?

[Colin Baxter]

>>>>> martin rudalics <rudalics@gmx.at> writes:

    >> But my question actually meant to say why the fact that Motif can
    >> wrap the menu is important enough to keep its support,

    > Because on GNU/Linux this provides a means for testing resizing of
    > the inner frame under the most heavy environmental conditions: It
    > has to deal with a wrapping menu, tool and tab bar all in one,
    > something I can test only on Windows otherwise.

    >> if no one uses it?

    > IIUC no one uses --with-x-toolkit=no 

Not true :-( I use it to build emacs specifically for terminal use.

Re: Cleaning out old X11 toolkits?

[martin rudalics]

 >      > IIUC no one uses --with-x-toolkit=no
 >
 > Not true :-( I use it to build emacs specifically for terminal use.

For what specific purpose?  Here a --with-x-toolkit=no build with
debugging symbols mounts up to 23.5 MB, a Motif build to 24.4 MB.  A
--without-x build has 18.5 MB.  Even that nowadays does not justify the
extra maintenance burden.  Motif builds OTOH never incur any additional
maintenance burden when fixing something in the GUI parts of our code.

martin

Re: Cleaning out old X11 toolkits?

[Ulrich Mueller]

>>>>> On Sat, 13 Feb 2021, martin rudalics wrote:

>> > IIUC no one uses --with-x-toolkit=no
>> 
>> Not true :-( I use it to build emacs specifically for terminal use.

> For what specific purpose?  Here a --with-x-toolkit=no build with
> debugging symbols mounts up to 23.5 MB, a Motif build to 24.4 MB.  A
> --without-x build has 18.5 MB.  Even that nowadays does not justify the
> extra maintenance burden.

It will require X11 to be present on the system at compile time and at
run time. There are systems where that isn't a feasible requirement.

Re: Cleaning out old X11 toolkits?

[martin rudalics]

 > It will require X11 to be present on the system at compile time and at
 > run time. There are systems where that isn't a feasible requirement.

Are there?  In practical use?

martin

Re: Cleaning out old X11 toolkits?

[Jean Louis]

* martin rudalics <rudalics@gmx.at> [2021-02-13 18:50]:
> > It will require X11 to be present on the system at compile time and at
> > run time. There are systems where that isn't a feasible requirement.
> 
> Are there?  In practical use?

I am also Termux user. We use Emacs in terminal. I do not know if it
is compiled for graphics, and is maybe not feasible compiling oneself,
but some people will do that.

Reference: https://termux.com/

One of my mobile devices runs Android and I use Emacs within Termux
when I am on the go. On other devices there is Lineage OS installed, I
use Emacs on Motorola E and E LTE versions to send emails and do
common Emacs stuff like writing. Often I access devices by using SSH.

Then majority of Virtual Private Server instances do not use X, they
need just terminal version. On Debian GNU/Linux runned VPS-es I will
install often `emacs-nox' package version without X, but too often I
need development version, in that case I am compiling it myself with
`./configure --without-all' but somebody could choose `no' to the
toolkit as well.

I guess that there are many ways why X may not be requires on
Raspberry Pi devices.

Without X, I believe that ./configure alone will work and will not
compile X anyway. But why compile X on those operating systems where
there are no X libraries by default, like VPS-es, small computers,
Android and Replicant and Lineage OS systems, no need for it.

Re: Cleaning out old X11 toolkits?

[martin rudalics]

 > Then majority of Virtual Private Server instances do not use X, they
 > need just terminal version. On Debian GNU/Linux runned VPS-es I will
 > install often `emacs-nox' package version without X, but too often I
 > need development version, in that case I am compiling it myself with
 > `./configure --without-all' but somebody could choose `no' to the
 > toolkit as well.

Honestly, I didn't expect anyone to do that.

Thanks for the information, martin

Re: Cleaning out old X11 toolkits?

[Ulrich Mueller]

>>>>> On Sun, 14 Feb 2021, martin rudalics wrote:

>> Then majority of Virtual Private Server instances do not use X, they
>> need just terminal version. On Debian GNU/Linux runned VPS-es I will
>> install often `emacs-nox' package version without X, but too often I
>> need development version, in that case I am compiling it myself with
>> `./configure --without-all' but somebody could choose `no' to the
>> toolkit as well.

> Honestly, I didn't expect anyone to do that.

In distros, it is all about dependencies. :)

Emacs can be configured (e.g., --without-all --without-x --without-json
--without-libgmp) so that it will need only glibc and ncurses:

$ objdump -p emacs | grep NEEDED
  NEEDED               librt.so.1
  NEEDED               libtinfo.so.6
  NEEDED               libpthread.so.0
  NEEDED               libanl.so.1
  NEEDED               libm.so.6
  NEEDED               libc.so.6

(BTW, why doesn't --without-all imply the --without-json and
--without-libgmp options? That's what I would have expected.)

Re: Cleaning out old X11 toolkits?

[martin rudalics]

 >> Honestly, I didn't expect anyone to do that.
 >
 > In distros, it is all about dependencies. :)

What I meant to say is that in my experience it's often more hassle to
sort out the dependencies than to download the necessary libraries.  And
the --without-all issue seems to confirm that.

 > Emacs can be configured (e.g., --without-all --without-x --without-json
 > --without-libgmp) so that it will need only glibc and ncurses:
 >
 > $ objdump -p emacs | grep NEEDED
 >    NEEDED               librt.so.1
 >    NEEDED               libtinfo.so.6
 >    NEEDED               libpthread.so.0
 >    NEEDED               libanl.so.1
 >    NEEDED               libm.so.6
 >    NEEDED               libc.so.6
 >
 > (BTW, why doesn't --without-all imply the --without-json and
 > --without-libgmp options? That's what I would have expected.)

Could you fix it?

martin

Re: Cleaning out old X11 toolkits?

[Eli Zaretskii]

> From: Ulrich Mueller <ulm@gentoo.org>
> Date: Sun, 14 Feb 2021 10:33:32 +0100
> Cc: Colin Baxter <m43cap@yandex.com>, emacs-devel@gnu.org
> 
> (BTW, why doesn't --without-all imply the --without-json and
> --without-libgmp options? That's what I would have expected.)

GMP cannot be removed, since we now support bignums, and that's not an
optional feature.

Native support for JSON could be disabled, but it's a small addition
that gives a significant gain.  E.g., without it LSP and similar stuff
will work much slower.

The description of --without-all says "almost all features", which is
accurate enough, I think.

Re: Cleaning out old X11 toolkits?

[Andy Moreton]

On Sun 14 Feb 2021, Eli Zaretskii wrote:

>> From: Ulrich Mueller <ulm@gentoo.org>
>> Date: Sun, 14 Feb 2021 10:33:32 +0100
>> Cc: Colin Baxter <m43cap@yandex.com>, emacs-devel@gnu.org
>> 
>> (BTW, why doesn't --without-all imply the --without-json and
>> --without-libgmp options? That's what I would have expected.)
>
> GMP cannot be removed, since we now support bignums, and that's not an
> optional feature.

However if libgmp is not found, emacs should be using lib/mini-gmp* as
the GMP implementation (slower than libgmp but aways available).

    AndyM

Re: Cleaning out old X11 toolkits?

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Sun, 14 Feb 2021 18:18:11 +0000
> 
> > GMP cannot be removed, since we now support bignums, and that's not an
> > optional feature.
> 
> However if libgmp is not found, emacs should be using lib/mini-gmp* as
> the GMP implementation (slower than libgmp but aways available).

So how is that a win? you get the same functionality with worse
performance.  --without-all is supposed to remove functionality.

Re: Cleaning out old X11 toolkits?

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> > GMP cannot be removed, since we now support bignums, and that's not an
>> > optional feature.
>> 
>> However if libgmp is not found, emacs should be using lib/mini-gmp* as
>> the GMP implementation (slower than libgmp but aways available).
>
> So how is that a win?

Building on a machine for installing on another one. It would be quite
puzzling if Emacs works fine on the build machine but fails outright on
the install machine.

> you get the same functionality with worse
> performance.  --without-all is supposed to remove functionality.

configure --help says:

--without-all           omit almost all features and build small executable
                        with minimal dependencies

If Emacs works without libgmp and libjson, they can't be considered as
part of its minimal dependencies, IMO.

Re: Cleaning out old X11 toolkits?

[martin rudalics]

 >  --without-all is supposed to remove functionality.

I would have thought it's supposed to remove dependencies.  Other than
that, I'd always want maximum functionality.

martin

Re: Cleaning out old X11 toolkits?

[Andy Moreton]

On Sun 14 Feb 2021, Eli Zaretskii wrote:

>> From: Andy Moreton <andrewjmoreton@gmail.com>
>> Date: Sun, 14 Feb 2021 18:18:11 +0000
>> 
>> > GMP cannot be removed, since we now support bignums, and that's not an
>> > optional feature.
>> 
>> However if libgmp is not found, emacs should be using lib/mini-gmp* as
>> the GMP implementation (slower than libgmp but aways available).
>
> So how is that a win? you get the same functionality with worse
> performance.  --without-all is supposed to remove functionality.

The configure help says:
  --without-all   omit almost all features and build small executable
                  with minimal dependencies

So --without-all is doing two things: removing optional features, and
removing dependencies. The GMP feature is not optional, but the
dependency on libgmp is optional.

If libgmp is not available for the target then it is still possible to
build and run emacs (but if libgmp is available for the target then it
should be installed and used).

    AndyM

Re: Cleaning out old X11 toolkits?

[Colin Baxter]

>>>>> martin rudalics <rudalics@gmx.at> writes:

    >> > IIUC no one uses --with-x-toolkit=no
    >> 
    >> Not true :-( I use it to build emacs specifically for terminal
    >> use.

    > For what specific purpose?

Running in an x-term, usually as part of tmux. My x-toolkit build
loads a third faster than my gtk build and I really don't need pretty
menus.

    > Here a --with-x-toolkit=no build with
    > debugging symbols mounts up to 23.5 MB, a Motif build to 24.4 MB.
    > A --without-x build has 18.5 MB.  Even that nowadays does not
    > justify the extra maintenance burden.

Ok, I'm sorry about that. I don't want to overburden the developers. I
was replying to a statement that nobody built a "no x-toolkit" emacs.

Best wishes,

Re: Cleaning out old X11 toolkits?

[Jean Louis]

* Colin Baxter <m43cap@yandex.com> [2021-02-13 17:40]:
> >>>>> martin rudalics <rudalics@gmx.at> writes:
> 
>     >> > IIUC no one uses --with-x-toolkit=no
>     >> 
>     >> Not true :-( I use it to build emacs specifically for terminal
>     >> use.
> 
>     > For what specific purpose?
> 
> Running in an x-term, usually as part of tmux. My x-toolkit build
> loads a third faster than my gtk build and I really don't need pretty
> menus.

On remote machines, when I compile Emacs I often invoke:

./configure --without-all

That makes it maybe even faster.

Re: Cleaning out old X11 toolkits?

[martin rudalics]

 > Running in an x-term, usually as part of tmux. My x-toolkit build
 > loads a third faster than my gtk build and I really don't need pretty
 > menus.

OK.  But then the --without-x build should load even faster.

 > Ok, I'm sorry about that. I don't want to overburden the developers.

As I said elsewhere, we spend 99% of our time for mending GTK and NS so
"overburden" is slightly exaggerated.

 > I was replying to a statement that nobody built a "no x-toolkit" emacs.

And I hope you will be able to do so for the coming decades.

martin

Re: Cleaning out old X11 toolkits?

[Colin Baxter]

>>>>> martin rudalics <rudalics@gmx.at> writes:

    >> Running in an x-term, usually as part of tmux. My x-toolkit build
    >> loads a third faster than my gtk build and I really don't need
    >> pretty menus.

    > OK.  But then the --without-x build should load even faster.

I've never tried that. Now's the time perhaps.

    >> I was replying to a statement that nobody built a "no x-toolkit"
    >> emacs.

    > And I hope you will be able to do so for the coming decades.

Thanks :-)

Best wishes,

Re: Cleaning out old X11 toolkits?

[Stefan Monnier]

>> if no one uses it?
> IIUC no one uses --with-x-toolkit=no

I don't know if noone does, but I do think it's one of the targets we
could throw out.  I suspect it would simplify more of the code than
removign Motif, since the no-toolkit is somewhat atypical (has special
events and code for scrollbars, and treats menus differently than the
rest of the code, AFAIK).

> or --without-x builds either,

This is in use, OTOH, typically for headless machines where you don't
want to bother installing the X11 libs, for Termux, and things like that.

There's an `emacs-nox` package in Debian for that, for example.
I don't know how its popularity compares to the `emacs-lucid` package.


        Stefan

Re: Cleaning out old X11 toolkits?

[Robert Pluim]

>>>>> On Sat, 13 Feb 2021 16:42:36 -0500, Stefan Monnier <monnier@iro.umontreal.ca> said:

    >>> if no one uses it?
    >> IIUC no one uses --with-x-toolkit=no

    Stefan> I don't know if noone does, but I do think it's one of the targets we
    Stefan> could throw out.  I suspect it would simplify more of the code than
    Stefan> removign Motif, since the no-toolkit is somewhat atypical (has special
    Stefan> events and code for scrollbars, and treats menus differently than the
    Stefan> rest of the code, AFAIK).

Itʼs also more likely to break in future as the world moves to
Wayland, and direct X11 access becomes unsupported.

    >> or --without-x builds either,

    Stefan> This is in use, OTOH, typically for headless machines where you don't
    Stefan> want to bother installing the X11 libs, for Termux, and things like that.

    Stefan> There's an `emacs-nox` package in Debian for that, for example.
    Stefan> I don't know how its popularity compares to the `emacs-lucid` package.

I donʼt know, but a no-gui emacs build is definitely worth preserving.

Robert

Re: Cleaning out old X11 toolkits?

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: martin rudalics <rudalics@gmx.at>,  Eli Zaretskii <eliz@gnu.org>,
>   casouri@gmail.com,  alan@idiocy.org,  emacs-devel@gnu.org,
>   contovob@tcd.ie,  dgutov@yandex.ru,  yandros@gmail.com,  larsi@gnus.org
> Date: Sun, 14 Feb 2021 13:41:36 +0100
> 
> >>>>> On Sat, 13 Feb 2021 16:42:36 -0500, Stefan Monnier <monnier@iro.umontreal.ca> said:
> 
>     >>> if no one uses it?
>     >> IIUC no one uses --with-x-toolkit=no
> 
>     Stefan> I don't know if noone does, but I do think it's one of the targets we
>     Stefan> could throw out.  I suspect it would simplify more of the code than
>     Stefan> removign Motif, since the no-toolkit is somewhat atypical (has special
>     Stefan> events and code for scrollbars, and treats menus differently than the
>     Stefan> rest of the code, AFAIK).
> 
> Itʼs also more likely to break in future as the world moves to
> Wayland, and direct X11 access becomes unsupported.

OTOH, if no other toolkit exists on a system, or if no other toolkit
builds with Emacs, the no-toolkit is the only way to have a GUI Emacs.
So maybe it is worth preserving, especially as the amount of
maintenance it requires is strictly zero.

> I donʼt know, but a no-gui emacs build is definitely worth preserving.

Right.

Re: Cleaning out old X11 toolkits?

[Stefan Monnier]

> OTOH, if no other toolkit exists on a system, or if no other toolkit
> builds with Emacs, the no-toolkit is the only way to have a GUI Emacs.
> So maybe it is worth preserving, especially as the amount of
> maintenance it requires is strictly zero.

Is there a system where --with-toolkit=no can build
yet --with-toolkit=lucid can't?


        Stefan

Re: Cleaning out old X11 toolkits?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Robert Pluim <rpluim@gmail.com>,  rudalics@gmx.at,  casouri@gmail.com,
>   alan@idiocy.org,  emacs-devel@gnu.org,  contovob@tcd.ie,
>   dgutov@yandex.ru,  yandros@gmail.com,  larsi@gnus.org
> Date: Sun, 14 Feb 2021 13:18:21 -0500
> 
> > OTOH, if no other toolkit exists on a system, or if no other toolkit
> > builds with Emacs, the no-toolkit is the only way to have a GUI Emacs.
> > So maybe it is worth preserving, especially as the amount of
> > maintenance it requires is strictly zero.
> 
> Is there a system where --with-toolkit=no can build
> yet --with-toolkit=lucid can't?

How can we possibly know? I'm talking about systems that are rare and
obscure, not something we see every day.

Re: Cleaning out old X11 toolkits?

[Ulrich Mueller]

>>>>> On Thu, 11 Feb 2021, chad  wrote:

>> Actually, looking through configure, we also have motif and athena and
>> 'none', so itʼs 6, not 3.

> I started a reply mentioning motif and athena/Xaw3d for hysterical
> reasons, but then peeked ahead and saw that they had already been
> mentioned.

> In more pragmatic terms, I would guess that it's entirely possible to
> excise motif/lesstif, athena, and Xaw3d from main without anyone
> noticing.

Your guess would be wrong.

> Whether this is worth the effort in a world leaning ever so slowly
> towards Cairo and pgtk is a little hard to tell, but a quick grep
> through src suggests that it would at least clear up a bunch of #ifdef
> spaghetti.

> Would the maintainers be interested in a branch that tried this? Would
> it be better to wait for pgtk to settle first? Is there a big use-case
> for those toolkits of which I'm unaware?

Has the GTK+ "closing displays" bug [1,2] been fixed? If not, I think
it's a showstopper for removal of the old toolkits. I use Emacs with
Athena here because of that bug.

[1] https://gitlab.gnome.org/GNOME/gtk/-/issues/221 
[2] https://gitlab.gnome.org/GNOME/gtk/-/issues/2315

Re: Cleaning out old X11 toolkits?

[James Cloos]

>>>>> "c" == chad  <yandros@gmail.com> writes:

c> In more pragmatic terms, I would guess that it's entirely possible to
c> excise motif/lesstif, athena, and Xaw3d from main without anyone noticing.

no it would not be.

i only use xaw3d. (like athena/lucid, but with depth and therefore
easier to glance.)

i have no idea whether anyone still uses motif.

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 0x997A9F17ED7DAEA6

Re: Cleaning out old X11 toolkits?

[Robert Pluim]

>>>>> On Fri, 12 Feb 2021 08:33:45 -0500, James Cloos <cloos@jhcloos.com> said:

>>>>> "c" == chad  <yandros@gmail.com> writes:
    c> In more pragmatic terms, I would guess that it's entirely possible to
    c> excise motif/lesstif, athena, and Xaw3d from main without anyone noticing.

    James> no it would not be.

    James> i only use xaw3d. (like athena/lucid, but with depth and therefore
    James> easier to glance.)

Yes, lucid in its various incarnations is still useful in a lot of
cases.

    James> i have no idea whether anyone still uses motif.

Weʼre about to find out, as Iʼve just removed it from master.

Robert

Re: "Fix" sag scaling for hidpi

[Stefan Monnier]

> Eli, Lars, when can we realistically envisage deprecating lucid
> support?  Followed a release later by the current gtk support.

I suspect that the "current gtk" support will disappear before the Lucid
support, since the Lucid support is still AFAIK the only one that
supports having frames on multiple X servers.

I also suspect that the "current gtk" support can disappear as soon as
the "pgtk" port is marked non-experimental, hence soon, e.g. it could be
gone for Emacs-29.


        Stefan

Re: "Fix" sag scaling for hidpi

[Robert Pluim]

>>>>> On Wed, 10 Feb 2021 14:19:00 -0500, Stefan Monnier <monnier@iro.umontreal.ca> said:

    >> Eli, Lars, when can we realistically envisage deprecating lucid
    >> support?  Followed a release later by the current gtk support.

    Stefan> I suspect that the "current gtk" support will disappear before the Lucid
    Stefan> support, since the Lucid support is still AFAIK the only one that
    Stefan> supports having frames on multiple X servers.

? both pure and impure gtk let you do that. Are you talking about
the 'crash when the display goes away' problem? Thatʼs an annoyance,
but it doesnʼt stop you using multiple frames.

    Stefan> I also suspect that the "current gtk" support can disappear as soon as
    Stefan> the "pgtk" port is marked non-experimental, hence soon, e.g. it could be
    Stefan> gone for Emacs-29.

Iʼm sharpening my delete key :-)

Robert

Re: "Fix" sag scaling for hidpi

[Stefan Monnier]

>     >> Eli, Lars, when can we realistically envisage deprecating lucid
>     >> support?  Followed a release later by the current gtk support.
>     Stefan> I suspect that the "current gtk" support will disappear before the Lucid
>     Stefan> support, since the Lucid support is still AFAIK the only one that
>     Stefan> supports having frames on multiple X servers.
> ? both pure and impure gtk let you do that.

I don't know the status of the pgtk port with respect to
multiple-displays, but for the "impure gtk" port, it's at best flaky.

> Are you talking about the 'crash when the display goes away' problem?

Yes.

> Thatʼs an annoyance, but it doesnʼt stop you using multiple frames.

I'm not talking about multiple frames, but about multiple displays.


        Stefan

Re: "Fix" sag scaling for hidpi

[Robert Pluim]

>>>>> On Thu, 11 Feb 2021 09:59:27 -0500, Stefan Monnier <monnier@iro.umontreal.ca> said:

    >> >> Eli, Lars, when can we realistically envisage deprecating lucid
    >> >> support?  Followed a release later by the current gtk support.
    Stefan> I suspect that the "current gtk" support will disappear before the Lucid
    Stefan> support, since the Lucid support is still AFAIK the only one that
    Stefan> supports having frames on multiple X servers.
    >> ? both pure and impure gtk let you do that.

    Stefan> I don't know the status of the pgtk port with respect to
    Stefan> multiple-displays, but for the "impure gtk" port, it's at best flaky.

Your network connections must be flakier than mine. (hmm, can mosh
forward X displays....)

    >> Are you talking about the 'crash when the display goes away' problem?

    Stefan> Yes.

    >> Thatʼs an annoyance, but it doesnʼt stop you using multiple frames.

    Stefan> I'm not talking about multiple frames, but about multiple displays.

multiple frames, with each on a different display, is what I meant.

Robert

Re: "Fix" sag scaling for hidpi

[Alan Third]

[-- Attachment #1: Type: text/plain, Size: 3220 bytes --]

On Wed, Feb 10, 2021 at 01:00:52PM -0500, Yuan Fu wrote:
> I agrees that we should keep the logical size, i.e., keep the size
> comparing against text. If we expose correct physical size, packages
> that generate bitmaps for display can generate crisp bitmaps with
> correct pixel size.
> 
> IIUC, a high-res image with :scale 0.5 should work across high and
> low-res displays. So ideally any package that wants to generate
> crisp bitmap can get the physical size and pixel-ratio from Emacs,
> generate the image and set :scale to 1/pixel-ratio. And this image
> works across different displays.

Yes, until the users drags the frame from a scale factor 1 screen to a
scale factor 2 screen, then it's blurry again. But I really don't
think there is a satisfactory solution to that.

> > We'll probably have to do more fiddling with SVGs though, since they
> > can define sizes in real-world units, like cm or inches, so the dpi
> > has to match the physical pixels, and if we move to a different screen
> > and regenerate the SVG the DPI will be different but the scale likely
> > won't be recalculated and the image will change size.
> 
> To simplify things, maybe we can assume DPI is 96. I.e., assume 1
> inch = 96 logical pixels = 96 * pixel-ratio physical pixels.
> 
> Obviously, if we can get DPI information from all terminals, then we
> could use that information. But from your previous message it
> doesn’t seem easy.

After messing about a bit today, I think the DPI thing is a red
herring. If we calculate the logical size of the SVG, then just
multiply it by the scale factor, then when drawing scale it down by
the scale factor everything is the right size and we can forget about
modifying the dpi.

> For SVGs I think we should automatically handle the pixel ratio and
> dpi so the image is always crisp and lisp doesn’t need to do
> anything (don’t need to add the :scale attribute or anything).
> 
> For bitmap images I think we display them in their physical size and
> let lisp alter the size by the :scale attribute.
> 
> For :width and :height attributes, I think they should be in logical
> pixels.Because the ratio of logical pixels and other text in a
> buffer doesn’t change when you drag a frame across different
> displays. So if the user set an image to have certain :width and
> drag the frame to a different display, the image doesn’t change its
> size comparing to everything else in the buffer.

Attached is a first go at this. I can't work out image.el and
image-mode.el, so I'll either need a lot more time or someone more
familiar with them can have a look.

The basic problem is that image-size returns physical pixels, and we
want to calculate logical pixels. I tried a few things and failed
miserably. I tried changing image-size to return logical pixels, but
that just seemed to make everything worse, and I don't think it's a
good idea to lie about the image size.

Anyway, SVGs will display nicely on macOS with the attached patch, and
frame-scale-factor returns a useful value. I didn't write the code for
GTK yet, but I guess it's probably not too hard.

I guess we also want to change the ghostscript code to do the same
thing as the SVG code too?
-- 
Alan Third

[-- Attachment #2: 0001-Scale-SVGs-to-match-frame-s-scale-factor.patch --]
[-- Type: text/plain, Size: 4536 bytes --]

From 6bff2bf4b22d4b018a8b13aef02b25185754f167 Mon Sep 17 00:00:00 2001
From: Alan Third <alan@idiocy.org>
Date: Wed, 10 Feb 2021 22:12:16 +0000
Subject: [PATCH] Scale SVGs to match frame's scale factor

---
 src/image.c  | 39 +++++++++++++++++++++++++++++++++------
 src/nsterm.h |  1 +
 src/nsterm.m | 11 +++++++++++
 3 files changed, 45 insertions(+), 6 deletions(-)

diff --git a/src/image.c b/src/image.c
index a124cf91ba..f8cd0d140e 100644
--- a/src/image.c
+++ b/src/image.c
@@ -135,6 +135,12 @@ #define PIX_MASK_DRAW	1
 # define COLOR_TABLE_SUPPORT 1
 #endif
 
+#ifdef HAVE_NS
+# define FRAME_SCALE_FACTOR(f) ns_frame_scale_factor (f)
+#else
+# define FRAME_SCALE_FACTOR(f) 1;
+#endif
+
 static void image_disable_image (struct frame *, struct image *);
 static void image_edge_detection (struct frame *, struct image *, Lisp_Object,
                                   Lisp_Object);
@@ -2207,8 +2213,8 @@ image_set_transform (struct frame *f, struct image *img)
   /* SVGs are pre-scaled to the correct size.  */
   if (EQ (image_spec_value (img->spec, QCtype, NULL), Qsvg))
     {
-      width = img->width;
-      height = img->height;
+      width = img->width / FRAME_SCALE_FACTOR (f);
+      height = img->height / FRAME_SCALE_FACTOR (f);
     }
   else
 #endif
@@ -2238,9 +2244,9 @@ image_set_transform (struct frame *f, struct image *img)
 		  : img->height / (double) height),
 # elif defined HAVE_NTGUI || defined HAVE_NS
 	[0][0] = (!IEEE_FLOATING_POINT && img->width == 0 ? DBL_MAX
-		  : width / (double) img->width),
+		  : width / (double) (img->width)),
 	[1][1] = (!IEEE_FLOATING_POINT && img->height == 0 ? DBL_MAX
-		  : height / (double) img->height),
+		  : height / (double) (img->height)),
 # else
 	[0][0] = 1, [1][1] = 1,
 # endif
@@ -10008,6 +10014,9 @@ svg_load_image (struct frame *f, struct image *img, char *contents,
   compute_image_size (viewbox_width, viewbox_height, img->spec,
                       &width, &height);
 
+  width *= FRAME_SCALE_FACTOR (f);
+  height *= FRAME_SCALE_FACTOR (f);
+
   if (! check_image_size (f, width, height))
     {
       image_size_error ();
@@ -10499,9 +10508,8 @@ DEFUN ("lookup-image", Flookup_image, Slookup_image, 1, 1, 0,
 
 #endif /* GLYPH_DEBUG */
 
-
 /***********************************************************************
-			    Initialization
+			    Image Scaling
  ***********************************************************************/
 
 DEFUN ("image-transforms-p", Fimage_transforms_p, Simage_transforms_p, 0, 1, 0,
@@ -10538,6 +10546,24 @@ DEFUN ("image-transforms-p", Fimage_transforms_p, Simage_transforms_p, 0, 1, 0,
   return Qnil;
 }
 
+DEFUN ("frame-scale-factor", Fframe_scale_factor, Sframe_scale_factor, 0, 1, 0,
+       doc: /* Return the image scale factor for FRAME.  */)
+       (Lisp_Object frame)
+{
+  struct frame *f = decode_live_frame (frame);
+  if (FRAME_WINDOW_P (f))
+    {
+      return make_float (FRAME_SCALE_FACTOR (f));
+    }
+
+  return make_float (1);
+}
+
+
+/***********************************************************************
+			    Initialization
+ ***********************************************************************/
+
 DEFUN ("init-image-library", Finit_image_library, Sinit_image_library, 1, 1, 0,
        doc: /* Initialize image library implementing image type TYPE.
 Return t if TYPE is a supported image type.
@@ -10820,6 +10846,7 @@ syms_of_image (void)
 #endif
 
   defsubr (&Simage_transforms_p);
+  defsubr (&Sframe_scale_factor);
 
   DEFVAR_BOOL ("cross-disabled-images", cross_disabled_images,
     doc: /* Non-nil means always draw a cross over disabled images.
diff --git a/src/nsterm.h b/src/nsterm.h
index eae1d0725e..017c2394ef 100644
--- a/src/nsterm.h
+++ b/src/nsterm.h
@@ -1252,6 +1252,7 @@ #define NSAPP_DATA2_RUNFILEDIALOG 11
 extern void ns_init_events (struct input_event *);
 extern void ns_finish_events (void);
 
+extern double ns_frame_scale_factor (struct frame *);
 
 #ifdef NS_IMPL_GNUSTEP
 extern char gnustep_base_version[];  /* version tracking */
diff --git a/src/nsterm.m b/src/nsterm.m
index 8f2b61a165..c15355c84d 100644
--- a/src/nsterm.m
+++ b/src/nsterm.m
@@ -857,6 +857,17 @@ Free a pool and temporary objects it refers to (callable from C)
 }
 
 
+double
+ns_frame_scale_factor (struct frame *f)
+{
+#ifdef NS_IMPL_COCOA
+  return [[FRAME_NS_VIEW (f) window] backingScaleFactor];
+#else
+  return 1;
+#endif
+}
+
+
 /* ==========================================================================
 
     Focus (clipping) and screen update
-- 
2.29.2

Re: "Fix" sag scaling for hidpi

[Yuan Fu]

> On Feb 10, 2021, at 6:55 PM, Alan Third <alan@idiocy.org> wrote:
> 
> On Wed, Feb 10, 2021 at 01:00:52PM -0500, Yuan Fu wrote:
>> I agrees that we should keep the logical size, i.e., keep the size
>> comparing against text. If we expose correct physical size, packages
>> that generate bitmaps for display can generate crisp bitmaps with
>> correct pixel size.
>> 
>> IIUC, a high-res image with :scale 0.5 should work across high and
>> low-res displays. So ideally any package that wants to generate
>> crisp bitmap can get the physical size and pixel-ratio from Emacs,
>> generate the image and set :scale to 1/pixel-ratio. And this image
>> works across different displays.
> 
> Yes, until the users drags the frame from a scale factor 1 screen to a
> scale factor 2 screen, then it's blurry again. But I really don't
> think there is a satisfactory solution to that.

I think that’s fine. If the user don’t want his image to be blurry, he should provide a high-enough-resolution image.

> 
>>> We'll probably have to do more fiddling with SVGs though, since they
>>> can define sizes in real-world units, like cm or inches, so the dpi
>>> has to match the physical pixels, and if we move to a different screen
>>> and regenerate the SVG the DPI will be different but the scale likely
>>> won't be recalculated and the image will change size.
>> 
>> To simplify things, maybe we can assume DPI is 96. I.e., assume 1
>> inch = 96 logical pixels = 96 * pixel-ratio physical pixels.
>> 
>> Obviously, if we can get DPI information from all terminals, then we
>> could use that information. But from your previous message it
>> doesn’t seem easy.
> 
> After messing about a bit today, I think the DPI thing is a red
> herring. If we calculate the logical size of the SVG, then just
> multiply it by the scale factor, then when drawing scale it down by
> the scale factor everything is the right size and we can forget about
> modifying the dpi.
> 
>> For SVGs I think we should automatically handle the pixel ratio and
>> dpi so the image is always crisp and lisp doesn’t need to do
>> anything (don’t need to add the :scale attribute or anything).
>> 
>> For bitmap images I think we display them in their physical size and
>> let lisp alter the size by the :scale attribute.
>> 
>> For :width and :height attributes, I think they should be in logical
>> pixels.Because the ratio of logical pixels and other text in a
>> buffer doesn’t change when you drag a frame across different
>> displays. So if the user set an image to have certain :width and
>> drag the frame to a different display, the image doesn’t change its
>> size comparing to everything else in the buffer.
> 
> Attached is a first go at this. I can't work out image.el and
> image-mode.el, so I'll either need a lot more time or someone more
> familiar with them can have a look.
> 
> The basic problem is that image-size returns physical pixels, and we
> want to calculate logical pixels. I tried a few things and failed
> miserably. I tried changing image-size to return logical pixels, but
> that just seemed to make everything worse, and I don't think it's a
> good idea to lie about the image size.

I’m not sure what do you want to achieve here. Are you trying to automatically size bitmap images when opening them, so that it is displayed 1:1 to physical pixels? If can do something like this:

(insert-image (create-image "~/d/Screen Shot 2021-02-09 at 10.11.56 AM.png"
                            nil nil :scale (/ 1 (frame-scale-factor))))

And the image inserted is displayed 1:1 to physical pixels. But :scale is not really meant for controlling pixel ratio. Maybe we should add a :pixel-ratio (or :scale-factor) attribute to control the default display size of a bitmap image?

> Anyway, SVGs will display nicely on macOS with the attached patch, and
> frame-scale-factor returns a useful value. I didn't write the code for
> GTK yet, but I guess it's probably not too hard.

Yes, SVGs worked well!

> 
> I guess we also want to change the ghostscript code to do the same
> thing as the SVG code too?

I think so.

Another question is what unit should :width and :height attributes use, but I guess we can discuss them later.

Yuan

Re: "Fix" sag scaling for hidpi

[Yuan Fu]

[-- Attachment #1: Type: text/plain, Size: 1315 bytes --]

>> 
>> Attached is a first go at this. I can't work out image.el and
>> image-mode.el, so I'll either need a lot more time or someone more
>> familiar with them can have a look.
>> 
>> The basic problem is that image-size returns physical pixels, and we
>> want to calculate logical pixels. I tried a few things and failed
>> miserably. I tried changing image-size to return logical pixels, but
>> that just seemed to make everything worse, and I don't think it's a
>> good idea to lie about the image size.
> 
> I’m not sure what do you want to achieve here. Are you trying to automatically size bitmap images when opening them, so that it is displayed 1:1 to physical pixels? If can do something like this:
> 
> (insert-image (create-image "~/d/Screen Shot 2021-02-09 at 10.11.56 AM.png"
>                            nil nil :scale (/ 1 (frame-scale-factor))))
> 
> And the image inserted is displayed 1:1 to physical pixels. But :scale is not really meant for controlling pixel ratio. Maybe we should add a :pixel-ratio (or :scale-factor) attribute to control the default display size of a bitmap image?

Actually, :pixel-ratio probably isn’t a good idea. I guess we should just let the user decide what they want when inserting an image, and automatically add :scale in image-mode.

Yuan

[-- Attachment #2: Type: text/html, Size: 6197 bytes --]

Re: "Fix" sag scaling for hidpi

[Alan Third]

On Thu, Feb 11, 2021 at 12:34:50PM -0500, Yuan Fu wrote:
> >> 
> >> Attached is a first go at this. I can't work out image.el and
> >> image-mode.el, so I'll either need a lot more time or someone more
> >> familiar with them can have a look.
> >> 
> >> The basic problem is that image-size returns physical pixels, and we
> >> want to calculate logical pixels. I tried a few things and failed
> >> miserably. I tried changing image-size to return logical pixels, but
> >> that just seemed to make everything worse, and I don't think it's a
> >> good idea to lie about the image size.
> > 
> > I’m not sure what do you want to achieve here. Are you trying to automatically size bitmap images when opening them, so that it is displayed 1:1 to physical pixels? If can do something like this:
> > 
> > (insert-image (create-image "~/d/Screen Shot 2021-02-09 at
> >                            10.11.56 AM.png" nil nil :scale (/ 1
> >                            (frame-scale-factor))))
> > 
> > And the image inserted is displayed 1:1 to physical pixels. But
> > :scale is not really meant for controlling pixel ratio. Maybe we
> > should add a :pixel-ratio (or :scale-factor) attribute to control
> > the default display size of a bitmap image?
> 
> Actually, :pixel-ratio probably isn’t a good idea. I guess we should
> just let the user decide what they want when inserting an image, and
> automatically add :scale in image-mode.

Yes, I agree. That's really what I was struggling with, getting
image-mode to load the image at physical pixel size. I don't do much
lisp coding, so I'm quite slow to get to understand it.

I'll keep looking, though.
-- 
Alan Third

Re: "Fix" sag scaling for hidpi

[Alan Third]

[-- Attachment #1: Type: text/plain, Size: 911 bytes --]

On Thu, Feb 11, 2021 at 08:18:41PM +0000, Alan Third wrote:
> On Thu, Feb 11, 2021 at 12:34:50PM -0500, Yuan Fu wrote:
> > Actually, :pixel-ratio probably isn’t a good idea. I guess we should
> > just let the user decide what they want when inserting an image, and
> > automatically add :scale in image-mode.
> 
> Yes, I agree. That's really what I was struggling with, getting
> image-mode to load the image at physical pixel size. I don't do much
> lisp coding, so I'm quite slow to get to understand it.
> 
> I'll keep looking, though.

OK, I've got something that works here. It seemed a bit too easy given
how much code there is in image-mode.el to do with scaling, but as I
said, it seems good to me.

(And I've just realised when attaching this that the commit name is
nonsense, but I imagine it will require a NEWS entry that I've not
written, so this won't be the final patch anyway.)
-- 
Alan Third

[-- Attachment #2: 0001-Scale-SVGs-to-match-frame-s-scale-factor.patch --]
[-- Type: text/plain, Size: 6368 bytes --]

From 62cbe4ed2376deeb3a6fc5315b938018cb71ef78 Mon Sep 17 00:00:00 2001
From: Alan Third <alan@idiocy.org>
Date: Wed, 10 Feb 2021 22:12:16 +0000
Subject: [PATCH] Scale SVGs to match frame's scale factor

* lisp/image-mode.el (image-vector-type): New variable.
(image-default-scale): New function.
(image-toggle-display-image): Use image-default-scale.
* src/image.c (FRAME_SCALE_FACTOR): New #define for getting frame
scale factor.
(image_set_transform):
(svg_load_image): Use FRAME_SCALE_FACTOR.
(Fframe_scale_factor): Returns the frame's scale factor.
* src/nsterm.m (ns_frame_scale_factor): Get the scale factor for an NS
frame.
---
 lisp/image-mode.el | 19 +++++++++++++++++--
 src/image.c        | 35 +++++++++++++++++++++++++++++++----
 src/nsterm.h       |  1 +
 src/nsterm.m       | 11 +++++++++++
 4 files changed, 60 insertions(+), 6 deletions(-)

diff --git a/lisp/image-mode.el b/lisp/image-mode.el
index 9ed295e2aa..4b5e282328 100644
--- a/lisp/image-mode.el
+++ b/lisp/image-mode.el
@@ -100,6 +100,11 @@ image-transform-right-angle-fudge
 There's no deep theory behind the default value, it should just
 be somewhat larger than ImageMagick's MagickEpsilon.")
 
+(defvar image-vector-type '(svg)
+  "List of types that won't be rescaled by the frame's scale factor.
+Vector image types should not be scaled down to match the
+screen's physical pixel dimensions.")
+
 (defun image-mode-winprops (&optional window cleanup)
   "Return winprops of WINDOW.
 A winprops object has the shape (WINDOW . ALIST).
@@ -435,6 +440,14 @@ image-mode-fit-frame
 				     (cons width height)
 				     window-configuration)))))))
 
+(defun image-default-scale (type)
+  "Return the default scale amount for image type TYPE.
+Should be 1 / frame-scale-factor for all image types except
+vector image types listed in `image-vector-type' that don't
+require rescaling."
+    (if (member type image-vector-type) 1.0
+      (/ 1 (frame-scale-factor))))
+
 ;;; Image Mode setup
 
 (defvar-local image-type nil
@@ -868,9 +881,11 @@ image-toggle-display-image
     ;; :scale 1: If we do not set this, create-image will apply
     ;; default scaling based on font size.
     (setq image (if (not edges)
-		    (create-image file-or-data type data-p :scale 1
+		    (create-image file-or-data type data-p
+                                  :scale (image-default-scale type)
                                   :format (and filename data-p))
-		  (create-image file-or-data type data-p :scale 1
+		  (create-image file-or-data type data-p
+                                :scale (image-default-scale type)
 				:max-width max-width
 				:max-height max-height
                                 ;; Type hint.
diff --git a/src/image.c b/src/image.c
index a124cf91ba..3f06860794 100644
--- a/src/image.c
+++ b/src/image.c
@@ -135,6 +135,12 @@ #define PIX_MASK_DRAW	1
 # define COLOR_TABLE_SUPPORT 1
 #endif
 
+#ifdef HAVE_NS
+# define FRAME_SCALE_FACTOR(f) ns_frame_scale_factor (f)
+#else
+# define FRAME_SCALE_FACTOR(f) 1;
+#endif
+
 static void image_disable_image (struct frame *, struct image *);
 static void image_edge_detection (struct frame *, struct image *, Lisp_Object,
                                   Lisp_Object);
@@ -2207,8 +2213,8 @@ image_set_transform (struct frame *f, struct image *img)
   /* SVGs are pre-scaled to the correct size.  */
   if (EQ (image_spec_value (img->spec, QCtype, NULL), Qsvg))
     {
-      width = img->width;
-      height = img->height;
+      width = img->width / FRAME_SCALE_FACTOR (f);
+      height = img->height / FRAME_SCALE_FACTOR (f);
     }
   else
 #endif
@@ -10008,6 +10014,9 @@ svg_load_image (struct frame *f, struct image *img, char *contents,
   compute_image_size (viewbox_width, viewbox_height, img->spec,
                       &width, &height);
 
+  width *= FRAME_SCALE_FACTOR (f);
+  height *= FRAME_SCALE_FACTOR (f);
+
   if (! check_image_size (f, width, height))
     {
       image_size_error ();
@@ -10499,9 +10508,8 @@ DEFUN ("lookup-image", Flookup_image, Slookup_image, 1, 1, 0,
 
 #endif /* GLYPH_DEBUG */
 
-
 /***********************************************************************
-			    Initialization
+			    Image Scaling
  ***********************************************************************/
 
 DEFUN ("image-transforms-p", Fimage_transforms_p, Simage_transforms_p, 0, 1, 0,
@@ -10538,6 +10546,24 @@ DEFUN ("image-transforms-p", Fimage_transforms_p, Simage_transforms_p, 0, 1, 0,
   return Qnil;
 }
 
+DEFUN ("frame-scale-factor", Fframe_scale_factor, Sframe_scale_factor, 0, 1, 0,
+       doc: /* Return the image scale factor for FRAME.  */)
+       (Lisp_Object frame)
+{
+  struct frame *f = decode_live_frame (frame);
+  if (FRAME_WINDOW_P (f))
+    {
+      return make_float (FRAME_SCALE_FACTOR (f));
+    }
+
+  return make_float (1);
+}
+
+
+/***********************************************************************
+			    Initialization
+ ***********************************************************************/
+
 DEFUN ("init-image-library", Finit_image_library, Sinit_image_library, 1, 1, 0,
        doc: /* Initialize image library implementing image type TYPE.
 Return t if TYPE is a supported image type.
@@ -10820,6 +10846,7 @@ syms_of_image (void)
 #endif
 
   defsubr (&Simage_transforms_p);
+  defsubr (&Sframe_scale_factor);
 
   DEFVAR_BOOL ("cross-disabled-images", cross_disabled_images,
     doc: /* Non-nil means always draw a cross over disabled images.
diff --git a/src/nsterm.h b/src/nsterm.h
index eae1d0725e..017c2394ef 100644
--- a/src/nsterm.h
+++ b/src/nsterm.h
@@ -1252,6 +1252,7 @@ #define NSAPP_DATA2_RUNFILEDIALOG 11
 extern void ns_init_events (struct input_event *);
 extern void ns_finish_events (void);
 
+extern double ns_frame_scale_factor (struct frame *);
 
 #ifdef NS_IMPL_GNUSTEP
 extern char gnustep_base_version[];  /* version tracking */
diff --git a/src/nsterm.m b/src/nsterm.m
index 3aba76ff5e..c095212d3a 100644
--- a/src/nsterm.m
+++ b/src/nsterm.m
@@ -857,6 +857,17 @@ Free a pool and temporary objects it refers to (callable from C)
 }
 
 
+double
+ns_frame_scale_factor (struct frame *f)
+{
+#ifdef NS_IMPL_COCOA
+  return [[FRAME_NS_VIEW (f) window] backingScaleFactor];
+#else
+  return 1;
+#endif
+}
+
+
 /* ==========================================================================
 
     Focus (clipping) and screen update
-- 
2.29.2

Re: "Fix" sag scaling for hidpi

[Lars Ingebrigtsen]

Alan Third <alan@idiocy.org> writes:

> OK, I've got something that works here. It seemed a bit too easy given
> how much code there is in image-mode.el to do with scaling, but as I
> said, it seems good to me.

I've not tried the patch, but if I understand correctly, it won't affect
the displayed size of non-SVG images?  In which case it looks good to
me, I think...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: "Fix" sag scaling for hidpi

[Alan Third]

On Sat, Feb 13, 2021 at 12:13:04PM +0100, Lars Ingebrigtsen wrote:
> Alan Third <alan@idiocy.org> writes:
> 
> > OK, I've got something that works here. It seemed a bit too easy given
> > how much code there is in image-mode.el to do with scaling, but as I
> > said, it seems good to me.
> 
> I've not tried the patch, but if I understand correctly, it won't affect
> the displayed size of non-SVG images?  In which case it looks good to
> me, I think...

No, on macOS retina screens it will display raster images at
1/scale-factor size in image-mode, which is 1:1 with physical pixels
on the screen.

If you just insert an image using create-image or in some other
manner, then nothing has changed.

I'm going to try and code up the GTK side of this, and then hopefully
more people can see the behaviour and decide if it's what we want to
happen.
-- 
Alan Third

Re: "Fix" sag scaling for hidpi

[Lars Ingebrigtsen]

Alan Third <alan@idiocy.org> writes:

> No, on macOS retina screens it will display raster images at
> 1/scale-factor size in image-mode, which is 1:1 with physical pixels
> on the screen.

But that's not what image-scaling-factor is about -- it's about scaling
images up on HiDPI screens (to make them visually larger, i.e., the same
size as on a non-HiDPI screen).  So it sounds like this is a roundabout
way of setting image-scaling-factor back to 1?

Uhm...  but on Macos, Emacs doesn't know the physical pixel size, I
think you said earlier?  So...  er...  now I'm confused.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: "Fix" sag scaling for hidpi

[Alan Third]

[-- Attachment #1: Type: text/plain, Size: 2151 bytes --]

On Sat, Feb 13, 2021 at 12:52:23PM +0100, Lars Ingebrigtsen wrote:
> Alan Third <alan@idiocy.org> writes:
> 
> > No, on macOS retina screens it will display raster images at
> > 1/scale-factor size in image-mode, which is 1:1 with physical pixels
> > on the screen.
> 
> But that's not what image-scaling-factor is about -- it's about scaling
> images up on HiDPI screens (to make them visually larger, i.e., the same
> size as on a non-HiDPI screen).  So it sounds like this is a roundabout
> way of setting image-scaling-factor back to 1?

Ah, I've misunderstood how GTK deals with scaling. I could've sworn
someone said upthread that SVG images are shown at double size, the
same as on macOS, but apparently not.

So we have the situation where on macOS everything is shown at logical
size, unless you specifically ask for it not to be, and on GTK
everything is shown at physical size, unless you ask for it not to be.

And they both use "scale factor" for these same things.

What does the PGTK port do? The same as XGTK?

Nope, it's the same as on macOS.

So on macOS and PGTK we want to use the scale factor to scale down
images so they are displayed with physical pixels, and on XGTK we want
to scale up (UI?) images so they are displayed in logical pixels.

I'm somewhat inclined to ignore XGTKs desires, especially as it seems
it may be removed once PGTK is merged.

In case it's not clear what's going on, I've attached a screenshot of
Emacs on macOS with my patch installed (left), and on PGTK with
GDK_SCALE=2 and without my patch (right). Both displaying the same SVG
file.

> Uhm...  but on Macos, Emacs doesn't know the physical pixel size, I
> think you said earlier?  So...  er...  now I'm confused.  :-)

It's something you have to go looking for specifically, usually
through multiplying by the scale factor. The reason I said I wouldn't
want Emacs to convert everything to physical pixels is because every
single size would have to be multiplied or divided by the scale
factor, and then Emacs would appear to be half the size of every other
app (font sizes would be half what they are in the terminal, for
example).
-- 
Alan Third

[-- Attachment #2: Screenshot 2021-02-13 at 12.53.43.png --]
[-- Type: image/png, Size: 372450 bytes --]

Re: "Fix" sag scaling for hidpi

[Alan Third]

On Sat, Feb 13, 2021 at 12:54:50PM +0000, Alan Third wrote:
> In case it's not clear what's going on, I've attached a screenshot of
> Emacs on macOS with my patch installed (left), and on PGTK with
> GDK_SCALE=2 and without my patch (right). Both displaying the same SVG
> file.

Sorry Lars, I'm losing track of this conversation. You asked about
whether it only affects SVGs.

The patch fixes the issue with SVG files being rendered at the logical
DPI.

But I've added code to shrink raster images as on macOS (and PGTK)
they are also displayed at the logical size. This is probably less of
a problem than with SVGs as often you can't really tell, but I'm of
the opinion that if I open an image in an image viewer (I'm
considering image-mode as an image viewer) I'd expect it to be the
physical size. Perhaps this is wrong and most people expect the
logical size.

-- 
Alan Third

Re: "Fix" sag scaling for hidpi

[Lars Ingebrigtsen]

Alan Third <alan@idiocy.org> writes:

> But I've added code to shrink raster images as on macOS (and PGTK)
> they are also displayed at the logical size. This is probably less of
> a problem than with SVGs as often you can't really tell, but I'm of
> the opinion that if I open an image in an image viewer (I'm
> considering image-mode as an image viewer) I'd expect it to be the
> physical size. Perhaps this is wrong and most people expect the
> logical size.

I expect the logical size in an image viewer.  

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: "Fix" sag scaling for hidpi

[Alan Third]

On Sat, Feb 13, 2021 at 04:09:26PM +0100, Lars Ingebrigtsen wrote:
> Alan Third <alan@idiocy.org> writes:
> 
> > But I've added code to shrink raster images as on macOS (and PGTK)
> > they are also displayed at the logical size. This is probably less of
> > a problem than with SVGs as often you can't really tell, but I'm of
> > the opinion that if I open an image in an image viewer (I'm
> > considering image-mode as an image viewer) I'd expect it to be the
> > physical size. Perhaps this is wrong and most people expect the
> > logical size.
> 
> I expect the logical size in an image viewer.  

OK, I'll remove that code.
-- 
Alan Third

Re: "Fix" sag scaling for hidpi

[Alan Third]

On Sat, Feb 13, 2021 at 04:00:21PM +0000, Alan Third wrote:
> On Sat, Feb 13, 2021 at 04:09:26PM +0100, Lars Ingebrigtsen wrote:
> > Alan Third <alan@idiocy.org> writes:
> > 
> > > But I've added code to shrink raster images as on macOS (and PGTK)
> > > they are also displayed at the logical size. This is probably less of
> > > a problem than with SVGs as often you can't really tell, but I'm of
> > > the opinion that if I open an image in an image viewer (I'm
> > > considering image-mode as an image viewer) I'd expect it to be the
> > > physical size. Perhaps this is wrong and most people expect the
> > > logical size.
> > 
> > I expect the logical size in an image viewer.  
> 
> OK, I'll remove that code.

I've pushed the code that fixes aliasing on SVG images on NS. I've not
made any changes to raster images or the XGTK stuff as I think it must
be doing the right thing already (for some value of right that I find
strange, but whatever).

It should be simple to extend the PGTK port to do the same thing as
the NS port to get the same effect.

(Amazingly this code works under GNUstep! I didn't know GNUstep did
scaling, but it does. Alas there seems to be a bug where when you
enable scaling the frame contents are pushed upwards. I don't think
it's an Emacs bug, but I'd have to investigate more to find out for
sure.)
-- 
Alan Third

Re: "Fix" sag scaling for hidpi

[Yuan Fu]

> On Feb 13, 2021, at 10:09 AM, Lars Ingebrigtsen <larsi@gnus.org> wrote:
> 
> Alan Third <alan@idiocy.org> writes:
> 
>> But I've added code to shrink raster images as on macOS (and PGTK)
>> they are also displayed at the logical size. This is probably less of
>> a problem than with SVGs as often you can't really tell, but I'm of
>> the opinion that if I open an image in an image viewer (I'm
>> considering image-mode as an image viewer) I'd expect it to be the
>> physical size. Perhaps this is wrong and most people expect the
>> logical size.
> 
> I expect the logical size in an image viewer.  

Don’t image viewer and video players display media in physical size by default? (At least on macOS.) That gives you better image.

Yuan

Re: "Fix" sag scaling for hidpi

[Lars Ingebrigtsen]

Yuan Fu <casouri@gmail.com> writes:

> Don’t image viewer and video players display media in physical size by
> default? (At least on macOS.) That gives you better image.

"Better image" is debatable.  Do you prefer a postage-sized image or one
that's big enough that you can make out what's displayed?

Macos is special in that it upscales images in the worst possible way --
it goes out of its way to blur as much as possible, giving you horrible
images.  Upscaling on other systems don't suffer from this problem to
the same degree -- in X, for instance, there's no blurring.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: "Fix" sag scaling for hidpi

[Dmitry Gutov]

On 13.02.2021 15:09, Alan Third wrote:
> But I've added code to shrink raster images as on macOS (and PGTK)
> they are also displayed at the logical size. This is probably less of
> a problem than with SVGs as often you can't really tell, but I'm of
> the opinion that if I open an image in an image viewer (I'm
> considering image-mode as an image viewer) I'd expect it to be the
> physical size. Perhaps this is wrong and most people expect the
> logical size.

Sorry for probably sidetracking this conversation, but can this at all 
be relevant to rendering the fringe bitmaps as well?

I noticed the pgtk and the GTK3 builds render them in different size at 
2x scaling. Basically, pgtk scales (and renders them kind of fuzzily) 
and GTK3 does not. I wonder if their rendering could be standardized as 
well, and whether the same approach could be used to avoid fuzziness.

Re: "Fix" sag scaling for hidpi

[Alan Third]

On Sun, Feb 14, 2021 at 12:50:26AM +0200, Dmitry Gutov wrote:
> I noticed the pgtk and the GTK3 builds render them in different size at 2x
> scaling. Basically, pgtk scales (and renders them kind of fuzzily) and GTK3
> does not. I wonder if their rendering could be standardized as well, and
> whether the same approach could be used to avoid fuzziness.

Really I think we should be providing hi-res, or vector, versions of
the fringe bitmaps for hi-dpi screens.

I'm not sure if we can just swap in SVG files for the fringe bitmaps
on systems that support them... Fringe bitmaps are special in some
way.
-- 
Alan Third

Re: "Fix" sag scaling for hidpi

[Dmitry Gutov]

On 14.02.2021 01:14, Alan Third wrote:
> On Sun, Feb 14, 2021 at 12:50:26AM +0200, Dmitry Gutov wrote:
>> I noticed the pgtk and the GTK3 builds render them in different size at 2x
>> scaling. Basically, pgtk scales (and renders them kind of fuzzily) and GTK3
>> does not. I wonder if their rendering could be standardized as well, and
>> whether the same approach could be used to avoid fuzziness.
> 
> Really I think we should be providing hi-res, or vector, versions of
> the fringe bitmaps for hi-dpi screens.
> 
> I'm not sure if we can just swap in SVG files for the fringe bitmaps
> on systems that support them... Fringe bitmaps are special in some
> way.

But couldn't we use a non-fuzzy algorithm to scale them, at least as a 
fallback? If the bitmap is all 1s an 0s, scaling by 2x could just double 
the lattice instead of doing the linear scaling.

Or there might be other scaling algorithms as well, if people feel that 
that approach ends up looking too "blocky".

There are an infinite number of possible scaling factors, so providing a 
high-res version for all of them is probably out, but some vector format 
sounds like an interesting possibility. As long as it's feasible to 
generate programmatically, because that's what I'm doing with a bunch of 
fringe bitmaps in a certain popular minor mode.

Re: "Fix" sag scaling for hidpi

[Yuan Fu]

> On Feb 13, 2021, at 7:02 PM, Dmitry Gutov <dgutov@yandex.ru> wrote:
> 
> On 14.02.2021 01:14, Alan Third wrote:
>> On Sun, Feb 14, 2021 at 12:50:26AM +0200, Dmitry Gutov wrote:
>>> I noticed the pgtk and the GTK3 builds render them in different size at 2x
>>> scaling. Basically, pgtk scales (and renders them kind of fuzzily) and GTK3
>>> does not. I wonder if their rendering could be standardized as well, and
>>> whether the same approach could be used to avoid fuzziness.
>> Really I think we should be providing hi-res, or vector, versions of
>> the fringe bitmaps for hi-dpi screens.
>> I'm not sure if we can just swap in SVG files for the fringe bitmaps
>> on systems that support them... Fringe bitmaps are special in some
>> way.
> 
> But couldn't we use a non-fuzzy algorithm to scale them, at least as a fallback? If the bitmap is all 1s an 0s, scaling by 2x could just double the lattice instead of doing the linear scaling.
> 
> Or there might be other scaling algorithms as well, if people feel that that approach ends up looking too "blocky".
> 
> There are an infinite number of possible scaling factors, so providing a high-res version for all of them is probably out, but some vector format sounds like an interesting possibility. As long as it's feasible to generate programmatically, because that's what I'm doing with a bunch of fringe bitmaps in a certain popular minor mode.

You can definitely generate SVG’s with sag-mode. For example, to generate a downward arrow:

(let ((svg (svg-create 9 9)))
         (svg-polygon svg '((0 . 0) (9 . 0) (4.5 . 9)) :fill "black")
         (svg-image svg))

Yuan

Re: "Fix" sag scaling for hidpi

[Yuan Fu]

> 
> You can definitely generate SVG’s with sag-mode.

I mean svg.el.

Re: "Fix" sag scaling for hidpi

[Dmitry Gutov]

On 14.02.2021 04:06, Yuan Fu wrote:
> You can definitely generate SVG’s with sag-mode. For example, to generate a downward arrow:
> 
> (let ((svg (svg-create 9 9)))
>           (svg-polygon svg '((0 . 0) (9 . 0) (4.5 . 9)) :fill "black")
>           (svg-image svg))

All right. Looking forward to being able to use svg in fringes, then.

Re: "Fix" sag scaling for hidpi

[Alan Third]

On Sun, Feb 14, 2021 at 02:02:52AM +0200, Dmitry Gutov wrote:
> On 14.02.2021 01:14, Alan Third wrote:
> > On Sun, Feb 14, 2021 at 12:50:26AM +0200, Dmitry Gutov wrote:
> > > I noticed the pgtk and the GTK3 builds render them in different size at 2x
> > > scaling. Basically, pgtk scales (and renders them kind of fuzzily) and GTK3
> > > does not. I wonder if their rendering could be standardized as well, and
> > > whether the same approach could be used to avoid fuzziness.
> > 
> > Really I think we should be providing hi-res, or vector, versions of
> > the fringe bitmaps for hi-dpi screens.
> > 
> > I'm not sure if we can just swap in SVG files for the fringe bitmaps
> > on systems that support them... Fringe bitmaps are special in some
> > way.
> 
> But couldn't we use a non-fuzzy algorithm to scale them, at least as a
> fallback? If the bitmap is all 1s an 0s, scaling by 2x could just double the
> lattice instead of doing the linear scaling.

That depends on the implementation. They don't look fuzzy on NS at 2x
scale. I could probably change some settings so they do, but won't. I
imagine the PGTK port could be tweaked to draw them with a different
scaling algorithm.

-- 
Alan Third

Re: "Fix" sag scaling for hidpi

[Lars Ingebrigtsen]

Alan Third <alan@idiocy.org> writes:

> Ah, I've misunderstood how GTK deals with scaling. I could've sworn
> someone said upthread that SVG images are shown at double size, the
> same as on macOS, but apparently not.

I tried loading the splash.svg file, and it's displayed here the same
size as the splash.png file.  Debian bullseye, HiDPI, scaling factor of
2.3, so the 333-pixel-wide image is displayed as a 760-pixel-wide image:

(insert-image (create-image "~/src/emacs/trunk/etc/images/splash.png"))
(insert-image (create-image "~/src/emacs/trunk/etc/images/splash.svg"))

> So we have the situation where on macOS everything is shown at logical
> size, unless you specifically ask for it not to be, and on GTK
> everything is shown at physical size, unless you ask for it not to be.
>
> And they both use "scale factor" for these same things.
>
> What does the PGTK port do? The same as XGTK?

This is a Gtk Emacs, but not a pgkt Emacs -- I haven't tried the pgtk
branch...

> Nope, it's the same as on macOS.
>
> So on macOS and PGTK we want to use the scale factor to scale down
> images so they are displayed with physical pixels, and on XGTK we want
> to scale up (UI?) images so they are displayed in logical pixels.
>
> I'm somewhat inclined to ignore XGTKs desires, especially as it seems
> it may be removed once PGTK is merged.

It sounds like confusion born out of some architectures reporting sizes
as logical pixels, and some as physical pixels.  That has to be fixed,
or we'll just confuse ourselves even more.  :-)

>> Uhm...  but on Macos, Emacs doesn't know the physical pixel size, I
>> think you said earlier?  So...  er...  now I'm confused.  :-)
>
> It's something you have to go looking for specifically, usually
> through multiplying by the scale factor. The reason I said I wouldn't
> want Emacs to convert everything to physical pixels is because every
> single size would have to be multiplied or divided by the scale
> factor, and then Emacs would appear to be half the size of every other
> app (font sizes would be half what they are in the terminal, for
> example).

I don't follow.  This Gtk Emacs is exactly the size I want it to be, and
it computes everything in physical pixels.

If the OS expects logical pixels, then we scale before asking the OS to
do something -- this is how Gtk menus are computed, for instance.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: "Fix" sag scaling for hidpi

[Alan Third]

On Sat, Feb 13, 2021 at 04:08:34PM +0100, Lars Ingebrigtsen wrote:
> Alan Third <alan@idiocy.org> writes:
> 
> > Ah, I've misunderstood how GTK deals with scaling. I could've sworn
> > someone said upthread that SVG images are shown at double size, the
> > same as on macOS, but apparently not.
> 
> I tried loading the splash.svg file, and it's displayed here the same
> size as the splash.png file.  Debian bullseye, HiDPI, scaling factor of
> 2.3, so the 333-pixel-wide image is displayed as a 760-pixel-wide image:
> 
> (insert-image (create-image "~/src/emacs/trunk/etc/images/splash.png"))
> (insert-image (create-image "~/src/emacs/trunk/etc/images/splash.svg"))

Strange. When I do that with GDK_SCALE=2 I get both images as 333
pixels wide. The only thing that appears to have been scaled are the
menus and toolbar, everything else is the exact same size as if I use
GDK_SCALE=1.

I guess I'm doing something wrong.

> > Nope, it's the same as on macOS.
> >
> > So on macOS and PGTK we want to use the scale factor to scale down
> > images so they are displayed with physical pixels, and on XGTK we want
> > to scale up (UI?) images so they are displayed in logical pixels.
> >
> > I'm somewhat inclined to ignore XGTKs desires, especially as it seems
> > it may be removed once PGTK is merged.
> 
> It sounds like confusion born out of some architectures reporting sizes
> as logical pixels, and some as physical pixels.  That has to be fixed,
> or we'll just confuse ourselves even more.  :-)

It's not really a problem in practice unless you want images that
aren't blurry due to being scaled up.

The confusion I'm seeing is purely down to the XGTK emacs scaling only
some UI elements.

(It does look like Windows is confusing though, since the first page
I've landed on to explain how it works describes three completely
separate mechanisms, one of which appears to match what macOS and GTK
do, while two are completely different.)

> >> Uhm...  but on Macos, Emacs doesn't know the physical pixel size, I
> >> think you said earlier?  So...  er...  now I'm confused.  :-)
> >
> > It's something you have to go looking for specifically, usually
> > through multiplying by the scale factor. The reason I said I wouldn't
> > want Emacs to convert everything to physical pixels is because every
> > single size would have to be multiplied or divided by the scale
> > factor, and then Emacs would appear to be half the size of every other
> > app (font sizes would be half what they are in the terminal, for
> > example).
> 
> I don't follow.  This Gtk Emacs is exactly the size I want it to be, and
> it computes everything in physical pixels.
> 
> If the OS expects logical pixels, then we scale before asking the OS to
> do something -- this is how Gtk menus are computed, for instance.

I'd stick with the default behaviour and just work around the few
occasions where that results in unwanted behaviour (the previously
mentioned blurry images).

-- 
Alan Third

Re: "Fix" sag scaling for hidpi

[Lars Ingebrigtsen]

Alan Third <alan@idiocy.org> writes:

>> (insert-image (create-image "~/src/emacs/trunk/etc/images/splash.png"))
>> (insert-image (create-image "~/src/emacs/trunk/etc/images/splash.svg"))
>
> Strange. When I do that with GDK_SCALE=2 I get both images as 333
> pixels wide. The only thing that appears to have been scaled are the
> menus and toolbar, everything else is the exact same size as if I use
> GDK_SCALE=1.

What's your `(image-compute-scaling-factor image-scaling-factor)'?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: "Fix" sag scaling for hidpi

[Alan Third]

On Sat, Feb 13, 2021 at 10:08:52PM +0100, Lars Ingebrigtsen wrote:
> Alan Third <alan@idiocy.org> writes:
> 
> >> (insert-image (create-image "~/src/emacs/trunk/etc/images/splash.png"))
> >> (insert-image (create-image "~/src/emacs/trunk/etc/images/splash.svg"))
> >
> > Strange. When I do that with GDK_SCALE=2 I get both images as 333
> > pixels wide. The only thing that appears to have been scaled are the
> > menus and toolbar, everything else is the exact same size as if I use
> > GDK_SCALE=1.
> 
> What's your `(image-compute-scaling-factor image-scaling-factor)'?

1

Do I need a specific version of GTK? I'm on Debian buster with GTK3.

Anyway, I suppose I'll have to make sure SVGs display without aliasing
on XGTK builds too! :)

-- 
Alan Third

Re: "Fix" sag scaling for hidpi

[Lars Ingebrigtsen]

Alan Third <alan@idiocy.org> writes:

>> > Strange. When I do that with GDK_SCALE=2 I get both images as 333
>> > pixels wide. The only thing that appears to have been scaled are the
>> > menus and toolbar, everything else is the exact same size as if I use
>> > GDK_SCALE=1.
>> 
>> What's your `(image-compute-scaling-factor image-scaling-factor)'?
>
> 1

The scaling factor is heuristically determined by how big the default
face is.  So I'm guessing that your Emacs is returning the size in
"logical pixels", which means that the heuristics don't trigger?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: "Fix" sag scaling for hidpi

[Alan Third]

On Sat, Feb 13, 2021 at 10:57:06PM +0100, Lars Ingebrigtsen wrote:
> Alan Third <alan@idiocy.org> writes:
> 
> >> > Strange. When I do that with GDK_SCALE=2 I get both images as 333
> >> > pixels wide. The only thing that appears to have been scaled are the
> >> > menus and toolbar, everything else is the exact same size as if I use
> >> > GDK_SCALE=1.
> >> 
> >> What's your `(image-compute-scaling-factor image-scaling-factor)'?
> >
> > 1
> 
> The scaling factor is heuristically determined by how big the default
> face is.  So I'm guessing that your Emacs is returning the size in
> "logical pixels", which means that the heuristics don't trigger?

I already wrote a C function to provide the real value to lisp. Would
that be better? (I've already deleted the commit, but I'm getting real
friendly with the git reflog tonight...)
-- 
Alan Third

Re: "Fix" sag scaling for hidpi

[chad]

[-- Attachment #1.1: Type: text/plain, Size: 596 bytes --]

On Sat, Feb 13, 2021 at 7:09 AM Lars Ingebrigtsen <larsi@gnus.org> wrote:

> I tried loading the splash.svg file, and it's displayed here the same
> size as the splash.png file.  Debian bullseye, HiDPI, scaling factor of
> 2.3, so the 333-pixel-wide image is displayed as a 760-pixel-wide image:
>
> (insert-image (create-image "~/src/emacs/trunk/etc/images/splash.png"))
> (insert-image (create-image "~/src/emacs/trunk/etc/images/splash.svg"))
>
> This is a Gtk Emacs, but not a pgkt Emacs -- I haven't tried the pgtk
> branch...
>

 FWIW, I took this from a pgtk branch as of late last night.

[-- Attachment #1.2: Type: text/html, Size: 992 bytes --]

[-- Attachment #2: Screenshot 2021-02-13 at 10.27.28.png --]
[-- Type: image/png, Size: 107744 bytes --]

Re: "Fix" sag scaling for hidpi

[chad]

[-- Attachment #1.1: Type: text/plain, Size: 878 bytes --]

On Sat, Feb 13, 2021 at 10:29 AM chad <yandros@gmail.com> wrote:

>
> On Sat, Feb 13, 2021 at 7:09 AM Lars Ingebrigtsen <larsi@gnus.org> wrote:
>
>> I tried loading the splash.svg file, and it's displayed here the same
>> size as the splash.png file.  Debian bullseye, HiDPI, scaling factor of
>> 2.3, so the 333-pixel-wide image is displayed as a 760-pixel-wide image:
>>
>> (insert-image (create-image "~/src/emacs/trunk/etc/images/splash.png"))
>> (insert-image (create-image "~/src/emacs/trunk/etc/images/splash.svg"))
>>
>> This is a Gtk Emacs, but not a pgkt Emacs -- I haven't tried the pgtk
>> branch...
>>
>
>  FWIW, I took this from a pgtk branch as of late last night.
>

Scratch that; I just realized that I had accidentally added an error to my
build scripts that was impacting feature/pgtk. This is from a fresh emacs
-Q from that branch. Apologies for the noise.

[-- Attachment #1.2: Type: text/html, Size: 1596 bytes --]

[-- Attachment #2: Screenshot 2021-02-13 at 11.49.58.png --]
[-- Type: image/png, Size: 97356 bytes --]

Re: "Fix" sag scaling for hidpi

[Yuan Fu]

> On Feb 13, 2021, at 10:08 AM, Lars Ingebrigtsen <larsi@gnus.org> wrote:
> 
> Alan Third <alan@idiocy.org> writes:
> 
>> Ah, I've misunderstood how GTK deals with scaling. I could've sworn
>> someone said upthread that SVG images are shown at double size, the
>> same as on macOS, but apparently not.
> 
> I tried loading the splash.svg file, and it's displayed here the same
> size as the splash.png file.  Debian bullseye, HiDPI, scaling factor of
> 2.3, so the 333-pixel-wide image is displayed as a 760-pixel-wide image:

What does this scaling factor do? Is it some sort of global GTK setting that makes it work with hidpi screens?

> 
> (insert-image (create-image "~/src/emacs/trunk/etc/images/splash.png"))
> (insert-image (create-image "~/src/emacs/trunk/etc/images/splash.svg"))
> 
>> So we have the situation where on macOS everything is shown at logical
>> size, unless you specifically ask for it not to be, and on GTK
>> everything is shown at physical size, unless you ask for it not to be.
>> 
>> And they both use "scale factor" for these same things.
>> 
>> What does the PGTK port do? The same as XGTK?
> 
> This is a Gtk Emacs, but not a pgkt Emacs -- I haven't tried the pgtk
> branch...
> 
>> Nope, it's the same as on macOS.
>> 
>> So on macOS and PGTK we want to use the scale factor to scale down
>> images so they are displayed with physical pixels, and on XGTK we want
>> to scale up (UI?) images so they are displayed in logical pixels.
>> 
>> I'm somewhat inclined to ignore XGTKs desires, especially as it seems
>> it may be removed once PGTK is merged.
> 
> It sounds like confusion born out of some architectures reporting sizes
> as logical pixels, and some as physical pixels.  That has to be fixed,
> or we'll just confuse ourselves even more.  :-)

I agree. Currently macOS and PGTK reports logical pixel sizes and other terminals report physical ones. We should make all terminal report logical sizes, then we don’t need image-scaling-factor anymore.

> 
>>> Uhm...  but on Macos, Emacs doesn't know the physical pixel size, I
>>> think you said earlier?  So...  er...  now I'm confused.  :-)
>> 
>> It's something you have to go looking for specifically, usually
>> through multiplying by the scale factor. The reason I said I wouldn't
>> want Emacs to convert everything to physical pixels is because every
>> single size would have to be multiplied or divided by the scale
>> factor, and then Emacs would appear to be half the size of every other
>> app (font sizes would be half what they are in the terminal, for
>> example).
> 
> I don't follow.  This Gtk Emacs is exactly the size I want it to be, and
> it computes everything in physical pixels.

Using logical pixels works better, I think. One, it doesn’t require people with hidpi screens to use a huge font size and rely on image-scaling-factor to scale UI icons; two, Emacs frame size will not change drastically if you drag it between hidpi screen and normal screen; three, it seems Mac, windows and gtk all support logical pixels, for me it’s TRT to do.

> 
> If the OS expects logical pixels, then we scale before asking the OS to
> do something -- this is how Gtk menus are computed, for instance.
> 
> -- 
> (domestic pets only, the antidote for overdose, milk.)
>   bloggy blog: http://lars.ingebrigtsen.no

Yuan

Re: "Fix" sag scaling for hidpi

[Lars Ingebrigtsen]

Yuan Fu <casouri@gmail.com> writes:

> I agree. Currently macOS and PGTK reports logical pixel sizes and
> other terminals report physical ones. We should make all terminal
> report logical sizes, then we don’t need image-scaling-factor anymore.

As long as they all do report pixels, there's no problem either way,
really.

The question is whether all platforms can report logical pixels.  If
they can't, then all platforms should to physical pixels -- there's
really no other choice.

I don't know the answer to the question, though.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: "Fix" sag scaling for hidpi

[Alan Third]

On Sat, Feb 13, 2021 at 10:11:45PM +0100, Lars Ingebrigtsen wrote:
> Yuan Fu <casouri@gmail.com> writes:
> 
> > I agree. Currently macOS and PGTK reports logical pixel sizes and
> > other terminals report physical ones. We should make all terminal
> > report logical sizes, then we don’t need image-scaling-factor anymore.
> 
> As long as they all do report pixels, there's no problem either way,
> really.
> 
> The question is whether all platforms can report logical pixels.  If
> they can't, then all platforms should to physical pixels -- there's
> really no other choice.

Physical pixels are just logical pixels at 1x scaling. There's really
no great incompatibility between them.
-- 
Alan Third

Re: "Fix" sag scaling for hidpi

[Grzegorz Kowzan]

On pią 05 lut 2021 at 11:24, Robert Pluim <rpluim@gmail.com> wrote:
>>>>>> On Fri, 05 Feb 2021 10:02:39 +0100, Lars Ingebrigtsen <larsi@gnus.org> said:
>
>     Lars> Yuan Fu <casouri@gmail.com> writes:
>     >> Sorry, I should be more specific. This blurry problem only occurs on
>     >> nsterm with hidpi screens. (That’s because cocoa reports logical pixel
>     >> sizes instead of physical ones.)
>
>     Lars> Should this be fixed (somehow) by changing the cocoa dimensions to be
>     Lars> physical ones, so that Emacs is consistent across architectures?
>
> Thatʼs the wrong direction: pgtk is moving us towards logical pixel
> sizes (gtk already uses them, but there are some rough edges).
>
> Robert

Please see also the extended discussion here:
https://github.com/masm11/emacs/issues/90

I would really like if this were solved at the Emacs level, without
requiring modifications in external packages. As far as I understand, on
Mac OS logical pixels are reported because there is no other option. This
makes pdf-tools, for example, include Mac OS-specific code to not
display blurry PDFs. With pgtk, we get to choose what is reported and
the current choice of logical pixels will require introducing the same
kind of platform-specific code into external packages for pgtk as for
cocoa.

Grzegorz

Re: "Fix" sag scaling for hidpi

[Alan Third]

On Fri, Feb 05, 2021 at 10:02:39AM +0100, Lars Ingebrigtsen wrote:
> Yuan Fu <casouri@gmail.com> writes:
> 
> > Sorry, I should be more specific. This blurry problem only occurs on
> > nsterm with hidpi screens. (That’s because cocoa reports logical pixel
> > sizes instead of physical ones.)
> 
> Should this be fixed (somehow) by changing the cocoa dimensions to be
> physical ones, so that Emacs is consistent across architectures?

No, macOS completely hides the physical sizes unless you really go
looking for them, so it would likely result in Emacs having some very
odd behaviours.

Plus I suspect it would result in an explosion of #ifdefs since Cocoa
drawing would use different dimensions than GNUstep drawing.
-- 
Alan Third

Re: "Fix" sag scaling for hidpi

[Alan Third]

On Thu, Feb 04, 2021 at 06:10:15PM -0500, Yuan Fu wrote:
> 
> 
> > On Feb 4, 2021, at 5:07 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> > 
> >> And Emacs will display a crisp 100x100 image in the buffer. However, this doesn’t work for sag images. If I do
> >> 
> >> (progn
> >>  (require 'svg)
> >>  (let ((svg (svg-create 200 200)))
> >>    (svg-text svg "test"
> >>              :font-size 100
> >>              :y 150)
> >>    (insert-image (svg-image svg :scale 0.5))))
> >> 
> >> The image is still blurry:
> > 
> > I think it's a bug, and that even
> > 
> >    (progn
> >      (require 'svg)
> >      (let ((svg (svg-create 100 100)))
> >        (svg-text svg "test"
> >                  :font-size 100
> >                  :y 150)
> >        (insert-image (svg-image svg :scale 1.0))))
> > 
> > should insert a crisp image.
> 
> Sorry, I should be more specific. This blurry problem only occurs on
> nsterm with hidpi screens. (That’s because cocoa reports logical
> pixel sizes instead of physical ones.)

At one time I had a patch that fixed this for SVGs on NS, but I wasn't
sure if it was "available" on other platforms (GTK is the only place I
know of that lets you set default scaling) and it wasn't generalised
to all images.

The best solution I could find is to set the SVG dpi and size
according to the real screen DPI, then when it comes to generating the
transform matrix allow divide the sizes by the scale factor.

It also wasn't clear to me what we should do about dpi on macOS.
Apparently macOS pretends the dpi is 72 no matter what the real value
is, and that meant keeping track of both the fake dpi and the real
dpi.

I wish I could find that patch now, but I guess it was deleted in a
clean-up.
-- 
Alan Third