bug#67628: Emacs 29.1: (x-open-connection (getenv "DISPLAY")) gets "Display :0.0 does not exist" error

bug#67628: Emacs 29.1: (x-open-connection (getenv "DISPLAY")) gets "Display :0.0 does not exist" error

[Herman]

Hi !

'x-create-frame' succeeds, but 'x-open-connection' fails with same display argument.

This error "Display :0.0 does not exist" happens only, when variable 'debug-on-error' is non-nil. When 'debug-on-error' is nil, the error is not present. I.e. it works on the principle - you get what you are looking for.

Tested also on Emacs 28.2 and there error is not present regardless of the value of the variable 'debug-on-error'.

xfwm4 4.18.0
xorg-server 21.1.9
Emacs 29.1
Archlinux

How to reproduce:
#################

$ emacs --fg-daemon -Q
...
Starting Emacs daemon.
$ emacsclient --eval "(setq debug-on-error t)"
t
$ emacsclient --eval "(x-open-connection (getenv \"DISPLAY\"))"
nil
...
Debugger entered--Lisp error: (error "Display :0.0 does not exist")
  signal(error ("Display :0.0 does not exist"))
  error("Display %s does not exist" ":0.0")
  get-device-terminal(":0.0")
  frames-on-display-list(":0.0")
  font-setting-change-default-font(":0.0" nil)
  dynamic-setting-handle-config-changed-event((config-changed-event font-render ":0.0"))
  funcall-interactively(dynamic-setting-handle-config-changed-event (config-changed-event font-render ":0.0"))
  call-interactively(dynamic-setting-handle-config-changed-event nil [(config-changed-event font-render ":0.0")])
  command-execute(dynamic-setting-handle-config-changed-event nil [(config-changed-event font-render ":0.0")] t)

The full testing:
#################

$ emacs --fg-daemon -Q
...
Starting Emacs daemon.
$ emacsclient --eval "(setq debug-on-error t)"
t
$ emacsclient --eval "debug-ignored-errors"
(beginning-of-line beginning-of-buffer end-of-line end-of-buffer end-of-file buffer-read-only file-supersession mark-inactive user-error)
$ emacsclient --eval "debug-on-signal"
nil
$ emacsclient --eval "debug-on-event"
sigusr2
$ emacsclient --eval "(terminal-list)"
(#<terminal 0 on initial_terminal>)
$ emacsclient --eval "(get-device-terminal nil)"
#<terminal 0 on initial_terminal>
$ emacsclient --eval "(x-display-list)"
nil
$ emacsclient --eval "(x-open-connection (getenv \"DISPLAY\"))"
nil
...
Debugger entered--Lisp error: (error "Display :0.0 does not exist")
  signal(error ("Display :0.0 does not exist"))
  error("Display %s does not exist" ":0.0")
  get-device-terminal(":0.0")
  frames-on-display-list(":0.0")
  font-setting-change-default-font(":0.0" nil)
  dynamic-setting-handle-config-changed-event((config-changed-event font-render ":0.0"))
  funcall-interactively(dynamic-setting-handle-config-changed-event (config-changed-event font-render ":0.0"))
  call-interactively(dynamic-setting-handle-config-changed-event nil [(config-changed-event font-render ":0.0")])
  command-execute(dynamic-setting-handle-config-changed-event nil [(config-changed-event font-render ":0.0")] t)

$ emacs --fg-daemon -Q
...
Starting Emacs daemon.
$ emacsclient --eval "(setq debug-on-error t)"
t
$ emacsclient --eval "(x-create-frame \`((display . ,(getenv \"DISPLAY\"))))"
#<frame *scratch* - GNU Emacs at Elitebook 0x55f5f31eeba0>
$ emacsclient --eval "(terminal-list)"
(#<terminal 0 on initial_terminal> #<terminal 1 on :0.0>)
$ emacsclient --eval "(get-device-terminal nil)"
#<terminal 1 on :0.0>
$ emacsclient --eval "(x-display-list)"
(":0.0")
$ emacsclient --eval "(kill-emacs)"
$ emacs --fg-daemon -Q
...
Starting Emacs daemon.
$ emacsclient --eval "(x-open-connection (getenv \"DISPLAY\"))"
nil
$ emacsclient --eval "(x-display-list)"
(":0.0")
$ emacsclient --eval "(terminal-list)"
(#<terminal 0 on initial_terminal> #<terminal 1 on :0.0>)
$ emacsclient --eval "(get-device-terminal nil)"
#<terminal 0 on initial_terminal>

bug#67628: Emacs 29.1: (x-open-connection (getenv "DISPLAY")) gets "Display :0.0 does not exist" error

[Eli Zaretskii]

> Date: Mon, 4 Dec 2023 19:31:38 +0200
> From: Herman <mahlamytsike@gmail.com>
> 
> 'x-create-frame' succeeds, but 'x-open-connection' fails with same display argument.
> 
> This error "Display :0.0 does not exist" happens only, when variable 'debug-on-error' is non-nil. When 'debug-on-error' is nil, the error is not present. I.e. it works on the principle - you get what you are looking for.

It could be that the code catches the error and proceeds, but your
setting debug-on-error non-nil prevents that.

Why do you need to set debug-on-error in that case?

bug#67628: Emacs 29.1: (x-open-connection (getenv "DISPLAY")) gets "Display :0.0 does not exist" error

[Herman]

On Mon, 04 Dec 2023 22:11:43 +0200
Eli Zaretskii <eliz@gnu.org> wrote:

> > Date: Mon, 4 Dec 2023 19:31:38 +0200
> > From: Herman <mahlamytsike@gmail.com>
> > 
> > 'x-create-frame' succeeds, but 'x-open-connection' fails with same
> > display argument.
> > 
> > This error "Display :0.0 does not exist" happens only, when
> > variable 'debug-on-error' is non-nil. When 'debug-on-error' is nil,
> > the error is not present. I.e. it works on the principle - you get
> > what you are looking for.  
> 
> It could be that the code catches the error and proceeds, but your
> setting debug-on-error non-nil prevents that.
> 
> Why do you need to set debug-on-error in that case?

At first I also thought that it is actually normal to catch an error when debug is enabled. I had debug turned on to look for other errors.

One argument could be that in version 28.2 debug did not catch such an error when debug is enabled. Version 29.1 behaves differently.

Another argument is that in the Elisp manual, in the Multiple terminals chapter, it is written that: "Normally, Emacs Lisp programs need not call 'x-open-connection' function, as 'make-frame-on-display' calls it automatically. The only reason for calling it is to check whether communication can be established with a given X display." 
If in 29.1 now 
$ emacs --daemon -Q
$ emacsclient -e "(setq debug-on-error t)"
then 
$ emacsclient -e "(make-frame-on-display (getenv \"DISPLAY\"))"
does not give an error, i.e. the display is found

but if
$ emacs --daemon -Q
$ emacsclient -e "(setq debug-on-error t)"
then
$ emacsclient -e "(x-open-connection (getenv \"DISPLAY\"))"
gives an error, i.e. the display does not exist.

bug#67628: Emacs 29.1: (x-open-connection (getenv "DISPLAY")) gets "Display :0.0 does not exist" error

[Eli Zaretskii]

> Date: Tue, 5 Dec 2023 00:44:05 +0200
> From: Herman <mahlamytsike@gmail.com>
> Cc: 67628@debbugs.gnu.org
> 
> On Mon, 04 Dec 2023 22:11:43 +0200
> Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > > Date: Mon, 4 Dec 2023 19:31:38 +0200
> > > From: Herman <mahlamytsike@gmail.com>
> > > 
> > > 'x-create-frame' succeeds, but 'x-open-connection' fails with same
> > > display argument.
> > > 
> > > This error "Display :0.0 does not exist" happens only, when
> > > variable 'debug-on-error' is non-nil. When 'debug-on-error' is nil,
> > > the error is not present. I.e. it works on the principle - you get
> > > what you are looking for.  
> > 
> > It could be that the code catches the error and proceeds, but your
> > setting debug-on-error non-nil prevents that.
> > 
> > Why do you need to set debug-on-error in that case?
> 
> At first I also thought that it is actually normal to catch an error when debug is enabled. I had debug turned on to look for other errors.
> 
> One argument could be that in version 28.2 debug did not catch such an error when debug is enabled. Version 29.1 behaves differently.
> 
> Another argument is that in the Elisp manual, in the Multiple terminals chapter, it is written that: "Normally, Emacs Lisp programs need not call 'x-open-connection' function, as 'make-frame-on-display' calls it automatically. The only reason for calling it is to check whether communication can be established with a given X display." 
> If in 29.1 now 
> $ emacs --daemon -Q
> $ emacsclient -e "(setq debug-on-error t)"
> then 
> $ emacsclient -e "(make-frame-on-display (getenv \"DISPLAY\"))"
> does not give an error, i.e. the display is found
> 
> but if
> $ emacs --daemon -Q
> $ emacsclient -e "(setq debug-on-error t)"
> then
> $ emacsclient -e "(x-open-connection (getenv \"DISPLAY\"))"
> gives an error, i.e. the display does not exist.

Po Lu, any ideas or suggestions?

bug#67628: Emacs 29.1: (x-open-connection (getenv "DISPLAY")) gets "Display :0.0 does not exist" error

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Tue, 5 Dec 2023 00:44:05 +0200
>> From: Herman <mahlamytsike@gmail.com>
>> Cc: 67628@debbugs.gnu.org
>> 
>> On Mon, 04 Dec 2023 22:11:43 +0200
>> Eli Zaretskii <eliz@gnu.org> wrote:
>> 
>> > > Date: Mon, 4 Dec 2023 19:31:38 +0200
>> > > From: Herman <mahlamytsike@gmail.com>
>> > > 
>> > > 'x-create-frame' succeeds, but 'x-open-connection' fails with same
>> > > display argument.
>> > > 
>> > > This error "Display :0.0 does not exist" happens only, when
>> > > variable 'debug-on-error' is non-nil. When 'debug-on-error' is nil,
>> > > the error is not present. I.e. it works on the principle - you get
>> > > what you are looking for.  
>> > 
>> > It could be that the code catches the error and proceeds, but your
>> > setting debug-on-error non-nil prevents that.
>> > 
>> > Why do you need to set debug-on-error in that case?
>> 
>> At first I also thought that it is actually normal to catch an error when debug is enabled. I had debug turned on to look for other errors.
>> 
>> One argument could be that in version 28.2 debug did not catch such an error when debug is enabled. Version 29.1 behaves differently.
>> 
>> Another argument is that in the Elisp manual, in the Multiple
>> terminals chapter, it is written that: "Normally, Emacs Lisp
>> programs need not call 'x-open-connection' function, as
>> 'make-frame-on-display' calls it automatically. The only reason for
>> calling it is to check whether communication can be established with
>> a given X display."
>> If in 29.1 now 
>> $ emacs --daemon -Q
>> $ emacsclient -e "(setq debug-on-error t)"
>> then 
>> $ emacsclient -e "(make-frame-on-display (getenv \"DISPLAY\"))"
>> does not give an error, i.e. the display is found
>> 
>> but if
>> $ emacs --daemon -Q
>> $ emacsclient -e "(setq debug-on-error t)"
>> then
>> $ emacsclient -e "(x-open-connection (getenv \"DISPLAY\"))"
>> gives an error, i.e. the display does not exist.
>
> Po Lu, any ideas or suggestions?

The display is opened successfully, but the next event to arrive
activates the dynamic-setting machinery that calls get-device-terminal,
which signals an error in response to there being no frames on that
display at the time.  The sound fix is for get-device-terminal to search
for the device provided within x_display_list instead of the list of
live frames.

bug#67628: Emacs 29.1: (x-open-connection (getenv "DISPLAY")) gets "Display :0.0 does not exist" error

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: Herman <mahlamytsike@gmail.com>,  67628@debbugs.gnu.org
> Date: Tue, 05 Dec 2023 20:37:00 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> If in 29.1 now 
> >> $ emacs --daemon -Q
> >> $ emacsclient -e "(setq debug-on-error t)"
> >> then 
> >> $ emacsclient -e "(make-frame-on-display (getenv \"DISPLAY\"))"
> >> does not give an error, i.e. the display is found
> >> 
> >> but if
> >> $ emacs --daemon -Q
> >> $ emacsclient -e "(setq debug-on-error t)"
> >> then
> >> $ emacsclient -e "(x-open-connection (getenv \"DISPLAY\"))"
> >> gives an error, i.e. the display does not exist.
> >
> > Po Lu, any ideas or suggestions?
> 
> The display is opened successfully, but the next event to arrive
> activates the dynamic-setting machinery that calls get-device-terminal,
> which signals an error in response to there being no frames on that
> display at the time.

So you are saying that this is an issue with the fine timing of when
the event arrives?  And if it arrived a bit later, it would have been
processed correctly?

> The sound fix is for get-device-terminal to search for the device
> provided within x_display_list instead of the list of live frames.

I'm not sure this is the correct fix.  If a frame is not yet in the
list of live frames exposed to Lisp, it might mean the frame is not
yet fully set up, and some code running off the dynamic-setting
machinery could similarly fail, right?

Wouldn't it be better to delay the processing of this event until the
frame is fully set?

bug#67628: Emacs 29.1: (x-open-connection (getenv "DISPLAY")) gets "Display :0.0 does not exist" error

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Po Lu <luangruo@yahoo.com>
>> Cc: Herman <mahlamytsike@gmail.com>,  67628@debbugs.gnu.org
>> Date: Tue, 05 Dec 2023 20:37:00 +0800
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> If in 29.1 now 
>> >> $ emacs --daemon -Q
>> >> $ emacsclient -e "(setq debug-on-error t)"
>> >> then 
>> >> $ emacsclient -e "(make-frame-on-display (getenv \"DISPLAY\"))"
>> >> does not give an error, i.e. the display is found
>> >> 
>> >> but if
>> >> $ emacs --daemon -Q
>> >> $ emacsclient -e "(setq debug-on-error t)"
>> >> then
>> >> $ emacsclient -e "(x-open-connection (getenv \"DISPLAY\"))"
>> >> gives an error, i.e. the display does not exist.
>> >
>> > Po Lu, any ideas or suggestions?
>> 
>> The display is opened successfully, but the next event to arrive
>> activates the dynamic-setting machinery that calls get-device-terminal,
>> which signals an error in response to there being no frames on that
>> display at the time.
>
> So you are saying that this is an issue with the fine timing of when
> the event arrives?  And if it arrived a bit later, it would have been
> processed correctly?

No, see below.

>> The sound fix is for get-device-terminal to search for the device
>> provided within x_display_list instead of the list of live frames.
>
> I'm not sure this is the correct fix.  If a frame is not yet in the
> list of live frames exposed to Lisp, it might mean the frame is not
> yet fully set up, and some code running off the dynamic-setting
> machinery could similarly fail, right?
>
> Wouldn't it be better to delay the processing of this event until the
> frame is fully set?

There's no frame being created halfway in this scenario.  Rather, a
display connection is opened _without_ creating any frames, from which a
dynamic-setting event subsequently arrives.

Thanks.

bug#67628: Emacs 29.1: (x-open-connection (getenv "DISPLAY")) gets "Display :0.0 does not exist" error

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: mahlamytsike@gmail.com,  67628@debbugs.gnu.org
> Date: Wed, 06 Dec 2023 20:21:36 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> The display is opened successfully, but the next event to arrive
> >> activates the dynamic-setting machinery that calls get-device-terminal,
> >> which signals an error in response to there being no frames on that
> >> display at the time.
> >
> > So you are saying that this is an issue with the fine timing of when
> > the event arrives?  And if it arrived a bit later, it would have been
> > processed correctly?
> 
> No, see below.
> 
> >> The sound fix is for get-device-terminal to search for the device
> >> provided within x_display_list instead of the list of live frames.
> >
> > I'm not sure this is the correct fix.  If a frame is not yet in the
> > list of live frames exposed to Lisp, it might mean the frame is not
> > yet fully set up, and some code running off the dynamic-setting
> > machinery could similarly fail, right?
> >
> > Wouldn't it be better to delay the processing of this event until the
> > frame is fully set?
> 
> There's no frame being created halfway in this scenario.  Rather, a
> display connection is opened _without_ creating any frames, from which a
> dynamic-setting event subsequently arrives.

If the frame doesn't exist, why do we consider this anything but a
cockpit error?

Or why would the dynamic-setting machinery call get-device-terminal
for a display that has no frames?

I guess I'm still missing something, so please elaborate, and maybe
point me to the relevant code involved in this.

Thanks.

bug#67628: Emacs 29.1: (x-open-connection (getenv "DISPLAY")) gets "Display :0.0 does not exist" error

[Eli Zaretskii]

> Cc: mahlamytsike@gmail.com, 67628@debbugs.gnu.org
> Date: Wed, 06 Dec 2023 14:38:25 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> 
> I guess I'm still missing something, so please elaborate, and maybe
> point me to the relevant code involved in this.

Oh, and one more question: why doesn't this happen in Emacs 28?  IOW,
what caused this "regression" in Emacs 29?

bug#67628: Emacs 29.1: (x-open-connection (getenv "DISPLAY")) gets "Display :0.0 does not exist" error

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

> Or why would the dynamic-setting machinery call get-device-terminal
> for a display that has no frames?

Because events are sent when configuration changes take place on each
display, rather than on each frame.  Although there aren't any frames, a
configuration change is detected and duly reported, after which the
dynamic-setting command bound to the event generated is called, and...

bug#67628: Emacs 29.1: (x-open-connection (getenv "DISPLAY")) gets "Display :0.0 does not exist" error

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

> Oh, and one more question: why doesn't this happen in Emacs 28?  IOW,
> what caused this "regression" in Emacs 29?

The same change that's behind bug#64809 and bug#65191, among others.  In
retrospect, the Cairo dynamic-setting code in bug#43128 shouldn't have
been installed at all, but then hindsight is universally 20/20.

bug#67628: Emacs 29.1: (x-open-connection (getenv "DISPLAY")) gets "Display :0.0 does not exist" error

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: mahlamytsike@gmail.com,  67628@debbugs.gnu.org
> Date: Wed, 06 Dec 2023 20:41:34 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Or why would the dynamic-setting machinery call get-device-terminal
> > for a display that has no frames?
> 
> Because events are sent when configuration changes take place on each
> display, rather than on each frame.  Although there aren't any frames, a
> configuration change is detected and duly reported, after which the
> dynamic-setting command bound to the event generated is called, and...

So the bug is in the command bound to the event, in that it assumes
there are frames on each display?  IOW, I would expect such a command
to do nothing about a display where we have no Emacs frames.

bug#67628: Emacs 29.1: (x-open-connection (getenv "DISPLAY")) gets "Display :0.0 does not exist" error

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: mahlamytsike@gmail.com,  67628@debbugs.gnu.org
> Date: Wed, 06 Dec 2023 20:50:31 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Oh, and one more question: why doesn't this happen in Emacs 28?  IOW,
> > what caused this "regression" in Emacs 29?
> 
> The same change that's behind bug#64809 and bug#65191, among others.  In
> retrospect, the Cairo dynamic-setting code in bug#43128 shouldn't have
> been installed at all, but then hindsight is universally 20/20.

If you mean commit 52d4c98cec0901ef5cc1c55d5b3b33ac9d9c519f, then I
don't see how it could cause the error.  Can you explain?

bug#67628: Emacs 29.1: (x-open-connection (getenv "DISPLAY")) gets "Display :0.0 does not exist" error

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Po Lu <luangruo@yahoo.com>
>> Cc: mahlamytsike@gmail.com,  67628@debbugs.gnu.org
>> Date: Wed, 06 Dec 2023 20:50:31 +0800
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > Oh, and one more question: why doesn't this happen in Emacs 28?  IOW,
>> > what caused this "regression" in Emacs 29?
>> 
>> The same change that's behind bug#64809 and bug#65191, among others.  In
>> retrospect, the Cairo dynamic-setting code in bug#43128 shouldn't have
>> been installed at all, but then hindsight is universally 20/20.
>
> If you mean commit 52d4c98cec0901ef5cc1c55d5b3b33ac9d9c519f, then I
> don't see how it could cause the error.  Can you explain?

It can't extract the default settings when the display is opened, so
once the connection to the client providing display settings is
established, it always reports a change.

bug#67628: Emacs 29.1: (x-open-connection (getenv "DISPLAY")) gets "Display :0.0 does not exist" error

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: mahlamytsike@gmail.com,  67628@debbugs.gnu.org
> Date: Thu, 07 Dec 2023 09:01:15 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Po Lu <luangruo@yahoo.com>
> >> Cc: mahlamytsike@gmail.com,  67628@debbugs.gnu.org
> >> Date: Wed, 06 Dec 2023 20:50:31 +0800
> >> 
> >> Eli Zaretskii <eliz@gnu.org> writes:
> >> 
> >> > Oh, and one more question: why doesn't this happen in Emacs 28?  IOW,
> >> > what caused this "regression" in Emacs 29?
> >> 
> >> The same change that's behind bug#64809 and bug#65191, among others.  In
> >> retrospect, the Cairo dynamic-setting code in bug#43128 shouldn't have
> >> been installed at all, but then hindsight is universally 20/20.
> >
> > If you mean commit 52d4c98cec0901ef5cc1c55d5b3b33ac9d9c519f, then I
> > don't see how it could cause the error.  Can you explain?
> 
> It can't extract the default settings when the display is opened, so
> once the connection to the client providing display settings is
> established, it always reports a change.

I admit I'm confused (and somewhat frustrated) by these terse
responses which doesn't explain enough for me to understand what would
be the best way out of this.

But never mind me: what do you suggest we do with this issue, on both
master and the emacs-29 branches?