[BUG] org-open-file immediately termininates when calling xdg-open that calls emacsclient [9.5.3 (release_9.5.3-508-ge268e4 @ /home/yantar92/.emacs.d/straight/build/org/)]

[BUG] org-open-file immediately termininates when calling xdg-open that calls emacsclient [9.5.3 (release_9.5.3-508-ge268e4 @ /home/yantar92/.emacs.d/straight/build/org/)]

[Ihor Radchenko]

Hi,

I have a problem with the following commits:

869b7a21b org-open-file: Avoid make-process for Emacs 24 compatibility
5db61eb0f org.el: Avoid xdg-open silent failure

They both set process-connection-type to nil/'pipe. Somehow, this change
breaks org-open-file for me when org-open-file calls xdg-open and xdg is
confugured to open the file with emacsclient. I only see a flicker and
the file does not open. Sometimes, Emacs even crashes.

If I change the process type back to default tty value, things work
again as expected.

While I understand that the original commits intended to solve a similar
problem, they clearly caused the same problem with other apps.

Best,
Ihor

Emacs  : GNU Emacs 28.1.50 (build 1, x86_64-pc-linux-gnu, cairo version 1.16.0)
 of 2022-05-14
Package: Org mode version 9.5.3 (release_9.5.3-508-ge268e4 @ /home/yantar92/.emacs.d/straight/build/org/)
-- 

Re: [BUG] org-open-file immediately termininates when calling xdg-open that calls emacsclient

[Max Nikulin]

On 22/05/2022 11:37, Ihor Radchenko wrote:
> 
> I have a problem with the following commits:
> 
> 869b7a21b org-open-file: Avoid make-process for Emacs 24 compatibility
> 5db61eb0f org.el: Avoid xdg-open silent failure
> 
> They both set process-connection-type to nil/'pipe. Somehow, this change
> breaks org-open-file for me when org-open-file calls xdg-open and xdg is
> confugured to open the file with emacsclient. I only see a flicker and
> the file does not open. Sometimes, Emacs even crashes.

I can not reproduce a problem, however I tried to factor-out xdg-open. 
Emacs-27.1, a minimal LXC container, so "&" to simulate kde-open5 
starting process in background. On the other hand I consider behavior 
when emacsclient is called from emacs as confusing due to requirement of 
C-x # to notify emacs about completion.

---- >8 ----
#+begin_src elisp
   (push '("\\.txt\\'" . "emacsclient %s &") org-file-apps)
#+end_src

[[file:some.txt]]
---- 8< ----

Nothing happens if emacs server is not started, but error handling was 
sacrificed for compatibility with old emacs versions.

Do you see the same problem for a function from mailcap.el in the 
development emacs version?

Eli Zaretskii suspected some compatibility issue, but he successfully 
resisted my attempts to pull some details from him. I know how to create 
a MIME handler that will cause 100% CPU consumption by Emacs for pipe 
processes, but he does not consider it as a real problem.

> If I change the process type back to default tty value, things work
> again as expected.

You might try `call-process' with 0 as DESTINATION (attention! not nil!).

See #+begin_comment snippet from the following patch (this part was not 
committed since Robert convinced me that it is confusing):
Max Nikulin to emacs-orgmode. [PATCH v3] Fix FAQ entry about mailto 
links. Mon, 14 Feb 2022 20:22:14 +0700. 
https://list.orgmode.org/9437ade2-af18-f97e-8790-a2df27c9017c@gmail.com

I do not know a way to reliably launch external MIME handler from Emacs. 
All means I am aware of have some caveats.

Re: [BUG] org-open-file immediately termininates when calling xdg-open that calls emacsclient

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

>> They both set process-connection-type to nil/'pipe. Somehow, this change
>> breaks org-open-file for me when org-open-file calls xdg-open and xdg is
>> confugured to open the file with emacsclient. I only see a flicker and
>> the file does not open. Sometimes, Emacs even crashes.
>
> I can not reproduce a problem, however I tried to factor-out xdg-open. 
> Emacs-27.1, a minimal LXC container, so "&" to simulate kde-open5 
> starting process in background. On the other hand I consider behavior 
> when emacsclient is called from emacs as confusing due to requirement of 
> C-x # to notify emacs about completion.

Try the following:

(start-process-shell-command "1" nil "emacsclient -c ~/.bashrc")
(let ((process-connection-type nil))
  (start-process-shell-command "1" nil "emacsclient -c ~/.bashrc"))

The second command will cause flickering, though opens the file in the
same frame (not the new as one would expect from -c switch).

To reproduce what I am seeing, you may need to adjust xdg- database
adding the following .desktop file:

~/.local/share/applications/emacsclient-local.desktop
---------------
[Desktop Entry]
Type=Application
Version=1.0
Name=Emacs Client
Exec=emacsclient -c %f
Icon=emacs-icon
Terminal=false
MimeType=text/css;text/english;text/html;text/plain;text/x-c;text/x-chdr;text/x-csrc;text/x-c++;text/x-c++hdr;text/x-c++src;text/x-java;text/x-makefile;text/x-moc;text/x-pascal;text/x-tcl;text/x-tex;application/x-shellscript;application/pdf;inode/directory;inode/mount-point;image/png;image/tiff;
---------------

Then, make sure that xdg-open uses the desktop file for
application/x-shellscript (xdg-mime query default
application/x-shellscript)

Finally, run 

(start-process-shell-command "1" nil "xdg-open -c ~/.bashrc")
(let ((process-connection-type nil))
  (start-process-shell-command "1" nil "xdg-open -c ~/.bashrc"))

Observe flickering + Emacs not opening the file.

> Do you see the same problem for a function from mailcap.el in the 
> development emacs version?

Which function are you referring to? 

Best,
Ihor

Re: [BUG] org-open-file immediately termininates when calling xdg-open that calls emacsclient

[Max Nikulin]

On 26/05/2022 21:24, Ihor Radchenko wrote:
> Max Nikulin writes:
> 
> Try the following:
> 
> (start-process-shell-command "1" nil "emacsclient -c ~/.bashrc")
> (let ((process-connection-type nil))
>    (start-process-shell-command "1" nil "emacsclient -c ~/.bashrc"))
> 
> The second command will cause flickering, though opens the file in the
> same frame (not the new as one would expect from -c switch).

Both commands creates a new frame in Emacs-26 and Emacs-27. I have not 
tried 28 yet.

You may try if the following works for you (borrowed from 
`browse-url-xdg-open')

(let ((url "~/.bashrc"))
   (call-process "xdg-open" nil 0 nil url))

Such approach has an advantage: viewer remains running after exit from 
Emacs.

You may submit an emacs bug for `mailcap-view-file' that should have 
similar behavior.

P.S. I still believe that Emacs configuration for mailcap.el is broken 
if external handler is invoked from Emacs just to execute emacsclient. 
On the other hand it should not lead to Emacs crash.

Re: [BUG] org-open-file immediately termininates when calling xdg-open that calls emacsclient

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

> On 26/05/2022 21:24, Ihor Radchenko wrote:
>> Max Nikulin writes:
>> 
>> Try the following:
>> 
>> (start-process-shell-command "1" nil "emacsclient -c ~/.bashrc")
>> (let ((process-connection-type nil))
>>    (start-process-shell-command "1" nil "emacsclient -c ~/.bashrc"))
>> 
>> The second command will cause flickering, though opens the file in the
>> same frame (not the new as one would expect from -c switch).
>
> Both commands creates a new frame in Emacs-26 and Emacs-27. I have not 
> tried 28 yet.

Thanks for testing! It appears that the problem was triggered by my
config in some very unexpected way.

I managed to narrow down the reproducer with the following minimal
configuration:

~/.emacs.d/init.el
-----------------------
(require 'server)
(unless (server-running-p) (server-start))
(setenv "DISPLAY" ":0.0")
(define-key input-decode-map (kbd "C-[") [C-\[])
------------------------------

With this exact init.el, trying to run
(let ((process-connection-type nil))
  (start-process-shell-command "1" nil "xdg-open ~/.bashrc"))

triggers flickering/Emacs crash. I am pretty sure that it is some kind
of weird Emacs bug.

Best,
Ihor

Re: [BUG] org-open-file immediately termininates when calling xdg-open that calls emacsclient

[Max Nikulin]

On 27/05/2022 13:19, Ihor Radchenko wrote:
> 
> I managed to narrow down the reproducer with the following minimal
> configuration:
> 
> ~/.emacs.d/init.el
> -----------------------
> (require 'server)
> (unless (server-running-p) (server-start))
> (setenv "DISPLAY" ":0.0")
> (define-key input-decode-map (kbd "C-[") [C-\[])
> ------------------------------
> 
> With this exact init.el, trying to run
> (let ((process-connection-type nil))
>    (start-process-shell-command "1" nil "xdg-open ~/.bashrc"))

Does it mean that you can not reproduce it with emacsclient instead of 
xdg-open? Anyway I can not reproduce it with Emacs-27.1 from 
Ubuntu-20.04. You may try to add before emacsclient command something like

     strace -s64 -f -t -o /tmp/emacsclient.strace

or attach strace to the running emacs using -p PID option just before 
executing `start-process-shell-command' and kill strace just after.

My attempt:
- Minimal LXC container, so I had to install xdg-utils and file packages
- Notice
       file --mime-type ~/.bashrc
       /home/ubuntu/.bashrc: text/plain
   so I tried with ~/.bashrc and with a script that is really recognized
   as text/x-shellscript
- ~/.local/share/applications/emacsclient-local.desktop
   from your previous message
- ~/.config/mimeapps.list
      [Added Associations]
      text/x-shellscript=emacsclient-local.desktop
      [Default Applications]
      text/x-shellscript=emacsclient-local.desktop
    without such file xdg-open can not find a handler despite I believed
    that it scans ~/.local/share/applications/ for .desktop files.
    Maybe that part of code works only for x-scheme-handler/...
    MIME types.
- emacs &
- switch from some info buffer specific to Debian
- execute the command you provided.

New frame is open and it seems reliable.

P.S. Likely it is unrelated to your problem:
emacs -Q --daemon

Warning: due to a long standing Gtk+ bug
https://gitlab.gnome.org/GNOME/gtk/issues/221
Emacs might crash when run in daemon mode and the X11 connection is 
unexpectedly lost.
Using an Emacs configured with --with-x-toolkit=lucid does not have this 
problem.
Starting Emacs daemon.

Re: [BUG] org-open-file immediately termininates when calling xdg-open that calls emacsclient

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

>> With this exact init.el, trying to run
>> (let ((process-connection-type nil))
>>    (start-process-shell-command "1" nil "xdg-open ~/.bashrc"))
>
> Does it mean that you can not reproduce it with emacsclient instead of 
> xdg-open? Anyway I can not reproduce it with Emacs-27.1 from 
> Ubuntu-20.04.

I have reported this to Emacs. The more precise steps are in
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=55672 

> - emacs &
> - switch from some info buffer specific to Debian
> - execute the command you provided.
>
> New frame is open and it seems reliable.

From the discussing in the bug report, it appears that part of the
problem is that my Emacs uses cairo build.

Best,
Ihor

[PATCH] org.el: Use `call-process' to launch external viewers (was: Re: [BUG] org-open-file immediately termininates when calling xdg-open that calls emacsclient)

[Max Nikulin]

[-- Attachment #1: Type: text/plain, Size: 958 bytes --]

On 27/05/2022 22:18, Ihor Radchenko wrote:
> Max Nikulin writes:
> 
>>> With this exact init.el, trying to run
>>> (let ((process-connection-type nil))
>>>     (start-process-shell-command "1" nil "xdg-open ~/.bashrc"))
>>
>> Does it mean that you can not reproduce it with emacsclient instead of
>> xdg-open? Anyway I can not reproduce it with Emacs-27.1 from
>> Ubuntu-20.04.
> 
> I have reported this to Emacs. The more precise steps are in
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=55672

As expected, nobody wishes to dig into obscure X11 issue.

You may try the attached patch. I do not consider it as unambiguous 
improvement since almost all errors are swallowed due to requirement to 
interpret viewer command using shell. The same approach settled on 
`browse-url' after several fixes, but there at least wrong executable 
name may be reported.

I suspect that you faced the bug working on another issue, so you do not 
really suffer from it.

[-- Attachment #2: 0001-org.el-Use-call-process-to-launch-external-viewers.patch --]
[-- Type: text/x-patch, Size: 2282 bytes --]

From 01c5625d4c9b47573b1b3c8fb9d5672ff87b6696 Mon Sep 17 00:00:00 2001
From: Max Nikulin <manikulin@gmail.com>
Date: Thu, 2 Jun 2022 23:55:09 +0700
Subject: [PATCH] org.el: Use `call-process' to launch external viewers

lisp/org.el (org-open-file): Another attempt to make launching
of viewers more reliable by using approach from `browse-url'
with hope to alleviate (bug#55672).  Unfortunately any errors
are silently ignored.
---
 lisp/org.el | 23 +++++++++++++----------
 1 file changed, 13 insertions(+), 10 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 95dff27ad..2cc122f5f 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -8015,16 +8015,19 @@ If the file does not exist, throw an error."
 
       (save-window-excursion
 	(message "Running %s...done" cmd)
-        ;; Handlers such as "gio open" and kde-open5 start viewer in background
-        ;; and exit immediately.  Use pipe connection type instead of pty to
-        ;; avoid killing children processes with SIGHUP when temporary terminal
-        ;; session is finished.
-        ;;
-        ;; TODO: Once minimum Emacs version is 25.1 or above, consider using
-        ;; the `make-process' invocation from 5db61eb0f929 to get more helpful
-        ;; error messages.
-        (let ((process-connection-type nil))
-	  (start-process-shell-command cmd nil cmd))
+        ;; Handlers such as "gio open" and kde-open5 start viewer in
+        ;; background and exit immediately.  As a result
+        ;; `start-shell-process' with default pty
+        ;; `process-connection-type', children processes are killed
+        ;; with SIGHUP when temporary terminal session is finished.
+        ;; Pipe processes have other set of problems, see (bug#55672),
+        ;; (bug#12972) message 47.  Shoot and forget method borrowed
+        ;; from `browse-url-xdg-open' has an advantage that launched
+        ;; viewer may run after quitting from Emacs.  It is hard to
+        ;; avoid shell here since it is required by mailcap
+        ;; specification, but with shell all errors are silently
+        ;; ignored.
+        (call-process shell-file-name nil 0 nil shell-command-switch cmd)
 	(and (boundp 'org-wait) (numberp org-wait) (sit-for org-wait))))
      ((or (stringp cmd)
 	  (eq cmd 'emacs))
-- 
2.25.1

Re: [PATCH] org.el: Use `call-process' to launch external viewers (was: Re: [BUG] org-open-file immediately termininates when calling xdg-open that calls emacsclient)

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

>> I have reported this to Emacs. The more precise steps are in
>> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=55672
>
> As expected, nobody wishes to dig into obscure X11 issue.
>
> You may try the attached patch. I do not consider it as unambiguous 
> improvement since almost all errors are swallowed due to requirement to 
> interpret viewer command using shell. The same approach settled on 
> `browse-url' after several fixes, but there at least wrong executable 
> name may be reported.

For reference, I am no longer able to reproduce the problem using the
provided recipe. Since the report, I have upgraded my system libraries
and Emacs (to the latest commit on master).

Now, I am not really sure if this patch is going to be an improvement.
Unless you have objections, I am marking the patch cancelled - I no
longer see any clear benefit of the patch.

Cancelled.

Best,
Ihor