bug#44824: 27.1; Org export as pdf and open file does not open it

bug#44824: 27.1; Org export as pdf and open file does not open it

[Geraldo Biotti]

[-- Attachment #1: Type: text/plain, Size: 5181 bytes --]

Create a new org file and save it.
Try to export to LaTeX as PDF file and open (C-c C-e l o)

This is what *messages* says:
(New file)
Saving file /tmp/test2.tex...
Wrote /tmp/test2.tex
Processing LaTeX file test2.tex...
PDF file produced.
Running /usr/bin/xdg-open /tmp/test2.pdf...done

The default PDF program (okular) appears to open (i see the icon, but not
the window) and closes without showing anything.

Working on Manjaro Linux 20.1:
KDE Plasma Version: 5.20.3
KDE Frameworks Version: 5.76.0
Qt version: 5.15.1
Kernel Version: 5.8.18-1-MANJARO
OS Type: 64 bit



In GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.22,
cairo version 1.17.3)
 of 2020-08-28 built on juergen
Windowing system distributor 'The X.Org Foundation', version 11.0.12009000
System Description: Manjaro Linux

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
(New file)
Saving file /tmp/test.org...
Wrote /tmp/test.org
Saving file /tmp/test.tex...
Wrote /tmp/test.tex
Processing LaTeX file test.tex...
PDF file produced.
Running /usr/bin/xdg-open /tmp/test.pdf...done

Configured using:
 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --with-x-toolkit=gtk3 --with-xft --with-wide-int
 --with-modules --with-cairo --with-harfbuzz 'CFLAGS=-march=x86-64
 -mtune=generic -O2 -pipe -fno-plt' CPPFLAGS=-D_FORTIFY_SOURCE=2
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now'

Configured features:
XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB NOTIFY
INOTIFY ACL GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD JSON
PDUMPER LCMS2 GMP

Important settings:
  value of $LC_MONETARY: it_IT.UTF-8
  value of $LC_NUMERIC: it_IT.UTF-8
  value of $LC_TIME: it_IT.UTF-8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Info

Minor modes in effect:
  shell-dirtrack-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug sendmail mule-util info tex-mode compile
shell latexenc cl-extra help-mode ox-odt rng-loc rng-uri rng-parse
rng-match rng-dt rng-util rng-pttrn nxml-parse nxml-ns nxml-enc xmltok
nxml-util ox-latex ox-icalendar ox-html table ox-ascii ox-publish ox
org-element avl-tree generator ol-eww ol-rmail ol-mhe ol-irc ol-info
ol-gnus nnir gnus-sum url url-proxy url-privacy url-expand url-methods
url-history mailcap shr url-cookie url-domsuf url-util url-parse
auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs json map
url-vars svg xml dom browse-url gnus-group gnus-undo gnus-start
gnus-cloud nnimap nnmail mail-source utf7 netrc nnoo parse-time iso8601
gnus-spec gnus-int gnus-range message rmc puny rfc822 mml mml-sec
password-cache epa derived epg epg-config mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader gnus-win gnus
nnheader gnus-util rmail rmail-loaddefs rfc2047 rfc2045 ietf-drums
text-property-search seq byte-opt gv bytecomp byte-compile cconv
mail-utils mm-util mail-prsvr wid-edit ol-docview doc-view jka-compr
image-mode exif dired dired-loaddefs ol-bibtex bibtex ol-bbdb ol-w3m org
ob ob-tangle ob-ref ob-lob ob-table ob-exp org-macro org-footnote
org-src ob-comint org-pcomplete pcomplete comint ansi-color ring
org-list org-faces org-entities time-date subr-x noutline outline
easy-mmode org-version ob-emacs-lisp ob-core ob-eval org-table ol
org-keys org-compat advice org-macs org-loaddefs format-spec find-func
cal-menu easymenu calendar cal-loaddefs cl-loaddefs cl-lib tooltip eldoc
electric uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win
x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify lcms2 dynamic-setting
system-font-setting font-render-setting cairo move-toolbar gtk x-toolkit
x multi-tty make-network-process emacs)

Memory information:
((conses 16 192463 13711)
 (symbols 48 20487 1)
 (strings 32 76053 9166)
 (string-bytes 1 2456243)
 (vectors 16 32285)
 (vector-slots 8 365851 18818)
 (floats 8 220 130)
 (intervals 56 360 0)
 (buffers 1000 14))

[-- Attachment #2: Type: text/html, Size: 5740 bytes --]

bug#44824: More info

[gbiotti]

Changing default program does not work: using evince instead of okular 
-> same results.

Executing the command "xdg-open /path/to/file.pdf" in a terminal 
(Konsole) works.

Executing the same command in Emacs via eshell DOES NOT WORK.

bug#44824: 27.1; Org export as pdf and open file does not open it

[Lars Ingebrigtsen]

"gbiotti@gmail.com" <gbiotti@gmail.com> writes:

> Executing the command "xdg-open /path/to/file.pdf" in a terminal
> (Konsole) works.
>
> Executing the same command in Emacs via eshell DOES NOT WORK.

What happens if you execute that command in Emacs via eshell?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44824: 27.1; Org export as pdf and open file does not open it

[gbiotti]

Il 27/01/2021 04:36, Lars Ingebrigtsen writes:
> "gbiotti@gmail.com" <gbiotti@gmail.com> writes:
>
>> Executing the command "xdg-open /path/to/file.pdf" in a terminal
>> (Konsole) works.
>>
>> Executing the same command in Emacs via eshell DOES NOT WORK.
> What happens if you execute that command in Emacs via eshell?
>

The same as using C-c C-e l o
"The default PDF program (okular) appears to open (i see the icon, but not
the window) and closes without showing anything."

bug#44824: 27.1; Org export as pdf and open file does not open it

[Lars Ingebrigtsen]

"gbiotti@gmail.com" <gbiotti@gmail.com> writes:

> The same as using C-c C-e l o
> "The default PDF program (okular) appears to open (i see the icon, but not
> the window) and closes without showing anything."

If I do

$ xdg-open ./doc/lispintro/cons-2.pdf

after `M-x shell', "Document Viewer" is opened as normal.

You don't get any output from xdg-open or anything in the shell buffer?

Glenn Morris <rgm@gnu.org> writes:

> Ref eg https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25234#8
> https://bugzilla.gnome.org/show_bug.cgi?id=652262
> https://gitlab.gnome.org/GNOME/glib/-/issues/1208
> and others going back over a decade.
> I think Emacs should have a PROBLEMS entry about this.

Yeah, calling xdg-open (and expecting it not to exit) is a known
problem, but here it seems that xdg-open doesn't even work from *shell*,
which is very odd.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44824: 27.1; Org export as pdf and open file does not open it

[Lars Ingebrigtsen]

"gbiotti@gmail.com" <gbiotti@gmail.com> writes:

> The same as using C-c C-e l o
> "The default PDF program (okular) appears to open (i see the icon, but not
> the window) and closes without showing anything."

If I do

$ xdg-open ./doc/lispintro/cons-2.pdf

after `M-x shell', "Document Viewer" is opened as normal.

You don't get any output from xdg-open or anything in the shell buffer?

Glenn Morris <rgm@gnu.org> writes:

> Ref eg https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25234#8
> https://bugzilla.gnome.org/show_bug.cgi?id=652262
> https://gitlab.gnome.org/GNOME/glib/-/issues/1208
> and others going back over a decade.
> I think Emacs should have a PROBLEMS entry about this.

Yeah, calling xdg-open (and expecting it not to exit) is a known
problem, but here it seems that xdg-open doesn't even work from *shell*,
which is very odd.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44824: 27.1; Org export as pdf and open file does not open it

[gbiotti]

Il 28/01/2021 04:02, Lars Ingebrigtsen ha scritto:
> "gbiotti@gmail.com" <gbiotti@gmail.com> writes:
>
>> The same as using C-c C-e l o
>> "The default PDF program (okular) appears to open (i see the icon, but not
>> the window) and closes without showing anything."
> If I do
>
> $ xdg-open ./doc/lispintro/cons-2.pdf
>
> after `M-x shell', "Document Viewer" is opened as normal.
>
> You don't get any output from xdg-open or anything in the shell buffer?
>
> Glenn Morris <rgm@gnu.org> writes:
>
>> Ref eg https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25234#8
>> https://bugzilla.gnome.org/show_bug.cgi?id=652262
>> https://gitlab.gnome.org/GNOME/glib/-/issues/1208
>> and others going back over a decade.
>> I think Emacs should have a PROBLEMS entry about this.
> Yeah, calling xdg-open (and expecting it not to exit) is a known
> problem, but here it seems that xdg-open doesn't even work from *shell*,
> which is very odd.
>

I have no more ideas.
The problem arise even exporting and opening html, so it's not the file 
type.

Doing a better search I found this:
https://forum.manjaro.org/t/xdg-open-or-kde-open-doesnt-work-when-called-from-emacs/39918

Same problem, but without an answer.

If needed I can make a video of what happen (if so please explain where 
to upload it).

bug#44824: 27.1; Org export as pdf and open file does not open it

[gbiotti]

Il 28/01/2021 04:02, Lars Ingebrigtsen writes:
> "gbiotti@gmail.com" <gbiotti@gmail.com> writes:
>
>> The same as using C-c C-e l o
>> "The default PDF program (okular) appears to open (i see the icon, but not
>> the window) and closes without showing anything."
> If I do
>
> $ xdg-open ./doc/lispintro/cons-2.pdf
>
> after `M-x shell', "Document Viewer" is opened as normal.
>
> You don't get any output from xdg-open or anything in the shell buffer?
>
> Glenn Morris <rgm@gnu.org> writes:
>
>> Ref eg https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25234#8
>> https://bugzilla.gnome.org/show_bug.cgi?id=652262
>> https://gitlab.gnome.org/GNOME/glib/-/issues/1208
>> and others going back over a decade.
>> I think Emacs should have a PROBLEMS entry about this.
> Yeah, calling xdg-open (and expecting it not to exit) is a known
> problem, but here it seems that xdg-open doesn't even work from *shell*,
> which is very odd.
>

More info:

As per bug 25234, using 'M-! xdg-open /tmp/test.pdf', 'M-& xdg-open 
/tmp/test.pdf'
and 'M-& xdg-open /tmp/test.pdf && sleep 3' I get same results as reported.

If I try in eshell buffer 'xdg-open /tmp/test.pdf && sleep 3' my cursor 
blinks
with the Okular icon for a few seconds and then nothing happens.

bug#44824: 27.1; Org export as pdf and open file does not open it

[Lars Ingebrigtsen]

"gbiotti@gmail.com" <gbiotti@gmail.com> writes:

>> Yeah, calling xdg-open (and expecting it not to exit) is a known
>> problem, but here it seems that xdg-open doesn't even work from *shell*,
>> which is very odd.
>
> More info:
>
> As per bug 25234, using 'M-! xdg-open /tmp/test.pdf', 'M-& xdg-open
> /tmp/test.pdf'
> and 'M-& xdg-open /tmp/test.pdf && sleep 3' I get same results as reported.

I'm not quite sure what you mean here.  Do you mean that
`M-! xdg-open /tmp/test.pdf' works fine, or that it fails?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44824: 27.1; Org export as pdf and open file does not open it

[Geraldo Biotti]

[-- Attachment #1: Type: text/plain, Size: 875 bytes --]

Il ven 29 gen 2021, 05:51 Lars Ingebrigtsen <larsi@gnus.org> writes:

> "gbiotti@gmail.com" <gbiotti@gmail.com> writes:
>
> >> Yeah, calling xdg-open (and expecting it not to exit) is a known
> >> problem, but here it seems that xdg-open doesn't even work from *shell*,
> >> which is very odd.
> >
> > More info:
> >
> > As per bug 25234, using 'M-! xdg-open /tmp/test.pdf', 'M-& xdg-open
> > /tmp/test.pdf'
> > and 'M-& xdg-open /tmp/test.pdf && sleep 3' I get same results as
> reported.
>
> I'm not quite sure what you mean here.  Do you mean that
> `M-! xdg-open /tmp/test.pdf' works fine, or that it fails?
>
> --
> (domestic pets only, the antidote for overdose, milk.)
>    bloggy blog: http://lars.ingebrigtsen.no


I get the same results reported in bug 25234. 'M-! xdg-open /tmp/test.pdf'
works fine. I apologise for my english but it's not my mother language.


>

[-- Attachment #2: Type: text/html, Size: 1788 bytes --]

bug#44824: 27.1; Org export as pdf and open file does not open it

[Lars Ingebrigtsen]

Geraldo Biotti <gbiotti@gmail.com> writes:

> I get the same results reported in bug 25234. 'M-! xdg-open /tmp/test.pdf' works
> fine. I apologise for my english but it's not my mother language.

So:

This works:
M-! xdg-open /tmp/test.pdf RET

This doesn't work:
M-& xdg-open /tmp/test.pdf RET

This doesn't work:
M-x shell RET xdg-open /tmp/test.pdf RET

?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44824: 27.1; Org export as pdf and open file does not open it

[Geraldo Biotti]

[-- Attachment #1: Type: text/plain, Size: 683 bytes --]

Il sab 30 gen 2021, 07:09 Lars Ingebrigtsen <larsi@gnus.org> ha scritto:

> Geraldo Biotti <gbiotti@gmail.com> writes:
>
> > I get the same results reported in bug 25234. 'M-! xdg-open
> /tmp/test.pdf' works
> > fine. I apologise for my english but it's not my mother language.
>
> So:
>
> This works:
> M-! xdg-open /tmp/test.pdf RET
>
> This doesn't work:
> M-& xdg-open /tmp/test.pdf RET
>
> This doesn't work:
> M-x shell RET xdg-open /tmp/test.pdf RET
>
> ?
>
> --
> (domestic pets only, the antidote for overdose, milk.)
>    bloggy blog: http://lars.ingebrigtsen.no


Exactly. And it doesn't even "export as PDF and open file" in org-mode,
which I think is a related problem.

[-- Attachment #2: Type: text/html, Size: 1304 bytes --]

bug#44824: 27.1; Org export as pdf and open file does not open it

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Sat, 30 Jan 2021 07:09:50 +0100
> Cc: 44824@debbugs.gnu.org
> 
> This works:
> M-! xdg-open /tmp/test.pdf RET
> 
> This doesn't work:
> M-& xdg-open /tmp/test.pdf RET
> 
> This doesn't work:
> M-x shell RET xdg-open /tmp/test.pdf RET

How about asking the xdg-open developers to help us figure out the
reason?  Or, failing that, debug xdg-open in the problematic
situations to find out what fails there and why?  E.g., could it be
that it fails because stdin/stdout is a PTY? what happens if you bind
process-connection-type to nil when starting the async subprocess?

bug#44824: 27.1; Org export as pdf and open file does not open it

[Maxim Nikulin]

On 30/01/2021 15:42, Eli Zaretskii wrote:
>>
>> This works:
>> M-! xdg-open /tmp/test.pdf RET
>>
>> This doesn't work:
>> M-& xdg-open /tmp/test.pdf RET
>>
>> This doesn't work:
>> M-x shell RET xdg-open /tmp/test.pdf RET

Geraldo, "M-x shell" case is rather strange. Could you, please, confirm 
ones more that okular window with the file content does not appear if 
you call xdg-open from an *interactive* emacs shell buffer? The link to 
an emacs-orgmode list message, that I have posted earlier, explains why 
async-shell-command *may* fail while shell-command should work reliably. 
I am really surprised by failure when command is executed in a [e]shell 
buffer.

> How about asking the xdg-open developers to help us figure out the
> reason?  Or, failing that, debug xdg-open in the problematic
> situations to find out what fails there and why?  E.g., could it be
> that it fails because stdin/stdout is a PTY? what happens if you bind
> process-connection-type to nil when starting the async subprocess?

I do not think, it is xdg-open problem. It just calls kde-open5 that 
spawns actual handler and immediately exits.

bug#44824: 27.1; Org export as pdf and open file does not open it

[Eli Zaretskii]

> From: Maxim Nikulin <m.a.nikulin@gmail.com>
> Date: Sat, 30 Jan 2021 20:31:53 +0700
> Cc: gbiotti@gmail.com
> 
> > How about asking the xdg-open developers to help us figure out the
> > reason?  Or, failing that, debug xdg-open in the problematic
> > situations to find out what fails there and why?  E.g., could it be
> > that it fails because stdin/stdout is a PTY? what happens if you bind
> > process-connection-type to nil when starting the async subprocess?
> 
> I do not think, it is xdg-open problem. It just calls kde-open5 that 
> spawns actual handler and immediately exits.

I didn't say it was their problem, I suggested to ask them to help us
understand why xdg-open doesn't work in those cases, under the
assumption that they are familiar with their code better than us.

bug#44824: 27.1; Org export as pdf and open file does not open it

[Maxim Nikulin]

On 30/01/2021 20:49, Eli Zaretskii wrote:
>>
>>> How about asking the xdg-open developers to help us figure out the
>>> reason?
>>
>> I do not think, it is xdg-open problem. It just calls kde-open5 that
>> spawns actual handler and immediately exits.
> 
> I didn't say it was their problem, I suggested to ask them to help us
> understand why xdg-open doesn't work in those cases, under the
> assumption that they are familiar with their code better than us.

What kind of help do you expect from xdg-open developers? It is a shell 
script, you could easily inspect it. I have posted already a command how 
to trace its execution. However currently I am almost sure that it 
merely calls 'kde-open5 /tmp/file.pdf'. The problem is that emacs does 
not expect that kde-open5 and thus xdg-open exits instantly. The 
question could be addressed to KDE developers, but unlike the issue with 
temporary files, in my opinion, pty+SIGHUP problem should be fixed in 
org mode. Some convenience function in emacs core would be nice but org 
mode is compatible with older emacs releases. Thus the only option is to 
change the org-open-files function.

bug#44824: 27.1; Org export as pdf and open file does not open it

[Eli Zaretskii]

> From: Maxim Nikulin <m.a.nikulin@gmail.com>
> Date: Sat, 30 Jan 2021 22:58:06 +0700
> Cc: 44824@debbugs.gnu.org
> 
> The problem is that emacs does not expect that kde-open5 and thus
> xdg-open exits instantly.

Why is that a problem, and how does it cause the invocation to fail,
i.e. not show the file in question?

> The question could be addressed to KDE developers, but unlike the
> issue with temporary files, in my opinion, pty+SIGHUP problem should
> be fixed in org mode.

What do you mean by "pty+SIGHUP problem" in this case?  What exactly
is the problem?

bug#44824: 27.1; Org export as pdf and open file does not open it

[Maxim Nikulin]

Bhavin, thank you very much for your clear report. I have tried once 
more with eshell session and this time I was lucky enough to reproduce 
the problem in both gnome and kde sessions on Ubuntu-20.04 focal

On 30/01/2021 23:28, Eli Zaretskii wrote:
>> From: Maxim Nikulin
>> Date: Sat, 30 Jan 2021 22:58:06 +0700
>>
>> The problem is that emacs does not expect that kde-open5 and thus
>> xdg-open exits instantly.
> 
> Why is that a problem, and how does it cause the invocation to fail,
> i.e. not show the file in question?
> 
>> The question could be addressed to KDE developers, but unlike the
>> issue with temporary files, in my opinion, pty+SIGHUP problem should
>> be fixed in org mode.
> 
> What do you mean by "pty+SIGHUP problem" in this case?  What exactly
> is the problem?

In the https://debbugs.gnu.org/cgi/bugreport.cgi?bug=44824#22 message I 
have posted a link to another thread in emacs-orgmode mail list thread 
with my earlier strace results: 
https://lists.gnu.org/archive/html/emacs-orgmode/2021-01/msg00327.html

Now I see that the problem with eshell is the same. I am not familiar 
with eshell, but it creates new shell process for every executed 
command. Actual handler is killed when underlying handler (kde-open5, 
"gio open") and thus xdg-open and the main shell process exit.

Excerpts from strace obtained for a eshell buffer

2221  16:59:43.513366 execve("/bin/sh", ["/bin/sh", "/usr/bin/xdg-open", 
"/tmp/test.pdf"], 0x7fff74be7f10 /* 58 vars */ <unfinished ...>
2224  16:59:43.566865 execve("/usr/bin/gio", ["gio", "open", 
"/tmp/test.pdf"], 0x55ee8454ec18 /* 58 vars */) = 0
2229  16:59:43.711846 execve("/bin/sh", ["/bin/sh", "-e", "-u", "-c", 
"export GIO_LAUNCHED_DESKTOP_FILE_PID=$$; exec \"$@\"", "sh", "evince", 
"/tmp/test.pdf"], 0x55bb59e67bb0 /* 59 vars */ <unfinished ...>
2221  16:59:43.717489 +++ exited with 0 +++
2229  16:59:43.719228 +++ killed by SIGHUP +++

Functions dealing with asynchronous processes in emacs, namely 
(start-process ...) and its siblings for shell commands calls 
(make-process :connection-type 'pty ...) that creates a pseudoterminal. 
It is redundant for applications that do not require an interactive 
terminal. When process (xdg-open this case) exits, pty is closed, all 
processes from the same terminal group receives SIGHUP. So actual 
handler is killed unless it has set signal handler or has detached from 
terminal session.

To fix the problem it is better to use (make-process :connection-type 
'pipe ...) that unfortunately has no higher level wrappers. "Pipe" 
process does not creates a pseudoterminal thus its children do not get 
SIGHUP on the exit of the main process. I am unsure concerning best 
values for other arguments however. The complication is that some 
mailcap entries have needsterminal flag, on the other hand they are 
likely irrelevant for GUI.

There is no problem if okular or evince are called directly (without 
kde-open5 or "gio open" wrapper) since main process does not exit while 
window is open.

Maybe the following command executed in eshell (namely eshell, not just 
shell) buffer is the best to demonstrate the problem (for those whose 
desktop environment is affected)

     sh -c "xdg-open /tmp/test.pdf; sleep 5"

The window with file content appears for 5 seconds then the viewer is 
killed.

On 31/01/2021 16:09, tomas@tuxteam.de wrote:
> This chaotic behaviour gives me the impression that it's an
> environment thing: desktop environments have the tendency to prime
> the environment variables in "creative" ways, often different from
> what a login shell would do.

Certainly the behavior depends on the desktop environment. You could 
check which DE-specific handler is called (and factor-out xdg-open) with

     sh -x /usr/bin/xdg-open /tmp/test.pdf

As to other options, M-! executes the process synchronously and is not 
affected. M-& has the same pty+SIGHUP problem.

I am almost sure that I have tried eshell before, but I have no idea why 
I have not noticed the problem that time.

bug#44824: 27.1; Org export as pdf and open file does not open it

[Maxim Nikulin]

Bhavin, thank you very much for your clear report. I have tried once 
more with eshell session and this time I was lucky enough to reproduce 
the problem in both gnome and kde sessions on Ubuntu-20.04 focal

On 30/01/2021 23:28, Eli Zaretskii wrote:
>> From: Maxim Nikulin
>> Date: Sat, 30 Jan 2021 22:58:06 +0700
>>
>> The problem is that emacs does not expect that kde-open5 and thus
>> xdg-open exits instantly.
> 
> Why is that a problem, and how does it cause the invocation to fail,
> i.e. not show the file in question?
> 
>> The question could be addressed to KDE developers, but unlike the
>> issue with temporary files, in my opinion, pty+SIGHUP problem should
>> be fixed in org mode.
> 
> What do you mean by "pty+SIGHUP problem" in this case?  What exactly
> is the problem?

In the https://debbugs.gnu.org/cgi/bugreport.cgi?bug=44824#22 message I 
have posted a link to another thread in emacs-orgmode mail list thread 
with my earlier strace results: 
https://lists.gnu.org/archive/html/emacs-orgmode/2021-01/msg00327.html

Now I see that the problem with eshell is the same. I am not familiar 
with eshell, but it creates new shell process for every executed 
command. Actual handler is killed when underlying handler (kde-open5, 
"gio open") and thus xdg-open and the main shell process exit.

Excerpts from strace obtained for a eshell buffer

2221  16:59:43.513366 execve("/bin/sh", ["/bin/sh", "/usr/bin/xdg-open", 
"/tmp/test.pdf"], 0x7fff74be7f10 /* 58 vars */ <unfinished ...>
2224  16:59:43.566865 execve("/usr/bin/gio", ["gio", "open", 
"/tmp/test.pdf"], 0x55ee8454ec18 /* 58 vars */) = 0
2229  16:59:43.711846 execve("/bin/sh", ["/bin/sh", "-e", "-u", "-c", 
"export GIO_LAUNCHED_DESKTOP_FILE_PID=$$; exec \"$@\"", "sh", "evince", 
"/tmp/test.pdf"], 0x55bb59e67bb0 /* 59 vars */ <unfinished ...>
2221  16:59:43.717489 +++ exited with 0 +++
2229  16:59:43.719228 +++ killed by SIGHUP +++

Functions dealing with asynchronous processes in emacs, namely 
(start-process ...) and its siblings for shell commands calls 
(make-process :connection-type 'pty ...) that creates a pseudoterminal. 
It is redundant for applications that do not require an interactive 
terminal. When process (xdg-open this case) exits, pty is closed, all 
processes from the same terminal group receives SIGHUP. So actual 
handler is killed unless it has set signal handler or has detached from 
terminal session.

To fix the problem it is better to use (make-process :connection-type 
'pipe ...) that unfortunately has no higher level wrappers. "Pipe" 
process does not creates a pseudoterminal thus its children do not get 
SIGHUP on the exit of the main process. I am unsure concerning best 
values for other arguments however. The complication is that some 
mailcap entries have needsterminal flag, on the other hand they are 
likely irrelevant for GUI.

There is no problem if okular or evince are called directly (without 
kde-open5 or "gio open" wrapper) since main process does not exit while 
window is open.

Maybe the following command executed in eshell (namely eshell, not just 
shell) buffer is the best to demonstrate the problem (for those whose 
desktop environment is affected)

     sh -c "xdg-open /tmp/test.pdf; sleep 5"

The window with file content appears for 5 seconds then the viewer is 
killed.

On 31/01/2021 16:09, tomas@tuxteam.de wrote:
> This chaotic behaviour gives me the impression that it's an
> environment thing: desktop environments have the tendency to prime
> the environment variables in "creative" ways, often different from
> what a login shell would do.

Certainly the behavior depends on the desktop environment. You could 
check which DE-specific handler is called (and factor-out xdg-open) with

     sh -x /usr/bin/xdg-open /tmp/test.pdf

As to other options, M-! executes the process synchronously and is not 
affected. M-& has the same pty+SIGHUP problem.

I am almost sure that I have tried eshell before, but I have no idea why 
I have not noticed the problem that time.

bug#44824: 27.1; Org export as pdf and open file does not open it

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1095 bytes --]

On Sun, Jan 31, 2021 at 06:15:27PM +0700, Maxim Nikulin wrote:
> Bhavin, thank you very much for your clear report. I have tried once
> more with eshell session and this time I was lucky enough to
> reproduce the problem in both gnome and kde sessions on Ubuntu-20.04
> focal

[...]

> 2221  16:59:43.717489 +++ exited with 0 +++
> 2229  16:59:43.719228 +++ killed by SIGHUP +++

Good catch.

> On 31/01/2021 16:09, tomas@tuxteam.de wrote:
> >This chaotic behaviour gives me the impression that it's an
> >environment thing [...]

> Certainly the behavior depends on the desktop environment. You could
> check which DE-specific handler is called (and factor-out xdg-open)
> with
> 
>     sh -x /usr/bin/xdg-open /tmp/test.pdf
> 
> As to other options, M-! executes the process synchronously and is
> not affected. M-& has the same pty+SIGHUP problem.
> 
> I am almost sure that I have tried eshell before, but I have no idea
> why I have not noticed the problem that time.

Yes, but in this case it seems I was barking up the wrong tree :)

Cheers & thanks
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

bug#44824: 27.1; Org export as pdf and open file does not open it

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1095 bytes --]

On Sun, Jan 31, 2021 at 06:15:27PM +0700, Maxim Nikulin wrote:
> Bhavin, thank you very much for your clear report. I have tried once
> more with eshell session and this time I was lucky enough to
> reproduce the problem in both gnome and kde sessions on Ubuntu-20.04
> focal

[...]

> 2221  16:59:43.717489 +++ exited with 0 +++
> 2229  16:59:43.719228 +++ killed by SIGHUP +++

Good catch.

> On 31/01/2021 16:09, tomas@tuxteam.de wrote:
> >This chaotic behaviour gives me the impression that it's an
> >environment thing [...]

> Certainly the behavior depends on the desktop environment. You could
> check which DE-specific handler is called (and factor-out xdg-open)
> with
> 
>     sh -x /usr/bin/xdg-open /tmp/test.pdf
> 
> As to other options, M-! executes the process synchronously and is
> not affected. M-& has the same pty+SIGHUP problem.
> 
> I am almost sure that I have tried eshell before, but I have no idea
> why I have not noticed the problem that time.

Yes, but in this case it seems I was barking up the wrong tree :)

Cheers & thanks
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

bug#44824: 27.1; Org export as pdf and open file does not open it

[Eli Zaretskii]

> From: Maxim Nikulin <m.a.nikulin@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>, gbiotti@gmail.com
> Date: Sun, 31 Jan 2021 18:15:27 +0700
> 
> Now I see that the problem with eshell is the same. I am not familiar 
> with eshell, but it creates new shell process for every executed 
> command. Actual handler is killed when underlying handler (kde-open5, 
> "gio open") and thus xdg-open and the main shell process exit.

What do you mean here by "actual handler" and "underlying handler"?

> Functions dealing with asynchronous processes in emacs, namely 
> (start-process ...) and its siblings for shell commands calls 
> (make-process :connection-type 'pty ...) that creates a pseudoterminal. 
> It is redundant for applications that do not require an interactive 
> terminal. When process (xdg-open this case) exits, pty is closed, all 
> processes from the same terminal group receives SIGHUP. So actual 
> handler is killed unless it has set signal handler or has detached from 
> terminal session.
> 
> To fix the problem it is better to use (make-process :connection-type 
> 'pipe ...) that unfortunately has no higher level wrappers.

Wouldn't it work to let-bind process-connection-type to nil around the
function that starts the async subprocess?

And I still don't understand why some people (like Lars) cannot
reproduce the problem at all -- the issue sounds like something that
should fail deterministically on any GNU/Linux system.  What am I
missing?

bug#44824: 27.1; Org export as pdf and open file does not open it

[Andreas Schwab]

On Jan 31 2021, Eli Zaretskii wrote:

> And I still don't understand why some people (like Lars) cannot
> reproduce the problem at all -- the issue sounds like something that
> should fail deterministically on any GNU/Linux system.  What am I
> missing?

If xdg-open doesn't need to start the program itself, and sends the
request to an already running process instead, there won't be any
problem with the disappearing session.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

bug#44824: 27.1; Org export as pdf and open file does not open it

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: Maxim Nikulin <m.a.nikulin@gmail.com>,  44824@debbugs.gnu.org,
>   gbiotti@gmail.com
> Date: Sun, 31 Jan 2021 16:17:46 +0100
> 
> On Jan 31 2021, Eli Zaretskii wrote:
> 
> > And I still don't understand why some people (like Lars) cannot
> > reproduce the problem at all -- the issue sounds like something that
> > should fail deterministically on any GNU/Linux system.  What am I
> > missing?
> 
> If xdg-open doesn't need to start the program itself, and sends the
> request to an already running process instead, there won't be any
> problem with the disappearing session.

Ah, okay.  Lars, could this be what happens on your system?

bug#44824: 27.1; Org export as pdf and open file does not open it

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> And I still don't understand why some people (like Lars) cannot
> reproduce the problem at all -- the issue sounds like something that
> should fail deterministically on any GNU/Linux system.  What am I
> missing?

The recipe said to start with `M-x shell' -- I wasn't able to reproduce
the problem there.  But with `M-x eshell' I can repeat the problem here,
too.

Perhaps the recipe was wrong?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44824: 27.1; Org export as pdf and open file does not open it

[Maxim Nikulin]

On 31/01/2021 22:05, Eli Zaretskii wrote:
>> From: Maxim Nikulin
>> Date: Sun, 31 Jan 2021 18:15:27 +0700
>>
>> Now I see that the problem with eshell is the same. I am not familiar
>> with eshell, but it creates new shell process for every executed
>> command. Actual handler is killed when underlying handler (kde-open5,
>> "gio open") and thus xdg-open and the main shell process exit.
> 
> What do you mean here by "actual handler" and "underlying handler"?

- actual handler: okular, evince, etc.
- underlying handler is what xdg-open actually calls: kde-open5, "gio 
open", etc. and that maps file type to particular .desktop (or mailcap) 
handler.

>> To fix the problem it is better to use (make-process :connection-type
>> 'pipe ...) that unfortunately has no higher level wrappers.
> 
> Wouldn't it work to let-bind process-connection-type to nil around the
> function that starts the async subprocess?

Sorry, for me it easier to reason how to express it in terms of system 
calls and terminal process groups than if let-bind could override a 
variable when lexical-bind is set to true.

> And I still don't understand why some people (like Lars) cannot
> reproduce the problem at all -- the issue sounds like something that
> should fail deterministically on any GNU/Linux system.  What am I
> missing?

On 31/01/2021 22:17, Andreas Schwab wrote:
 >
 > If xdg-open doesn't need to start the program itself, and sends the
 > request to an already running process instead, there won't be any
 > problem with the disappearing session.

I have been tempting to say that it is a race (either request is 
completed before SIGHUP or not) since Christopher Miles posted a link to 
stackexchange and I have realized the actual effect of an 
antidaemonizing cast I noticed earlier in a package related to org mode. 
On the other hand, I am not familiar with kde and gnome internals. I 
guess they could use a kind of server processes but I have no idea how 
to arrange parts for a convincing demonstration.

bug#44824: 27.1; Org export as pdf and open file does not open it

[Eli Zaretskii]

> From: Maxim Nikulin <m.a.nikulin@gmail.com>
> Date: Sun, 31 Jan 2021 22:57:57 +0700
> Cc: 44824@debbugs.gnu.org
> 
> >> To fix the problem it is better to use (make-process :connection-type
> >> 'pipe ...) that unfortunately has no higher level wrappers.
> > 
> > Wouldn't it work to let-bind process-connection-type to nil around the
> > function that starts the async subprocess?
> 
> Sorry, for me it easier to reason how to express it in terms of system 
> calls and terminal process groups than if let-bind could override a 
> variable when lexical-bind is set to true.

Well, I think we should try this, because if it works, it will show us
a way to fix the problem.  (I don't see how lexical-binding could
interfere with let-binding.)

bug#44824: 27.1; Org export as pdf and open file does not open it

[Maxim Nikulin]

On 31/01/2021 23:33, Eli Zaretskii wrote:
>>
>>>> To fix the problem it is better to use (make-process :connection-type
>>>> 'pipe ...) that unfortunately has no higher level wrappers.
>>>
>>> Wouldn't it work to let-bind process-connection-type to nil around the
>>> function that starts the async subprocess?

>> ...

> Well, I think we should try this, because if it works, it will show us
> a way to fix the problem.  (I don't see how lexical-binding could
> interfere with let-binding.)

It is not a great problem to use make-process:

https://lists.gnu.org/archive/html/emacs-orgmode/2021-01/msg00364.html

I think that at least a :sentinel should be added to improve error 
reporting. The code just before these lines has another problem (I hope 
it is really rare): 
https://lists.gnu.org/archive/html/emacs-orgmode/2021-01/msg00353.html

I would avoid shell since direct execution of the handler should be more 
robust, e.g. quoting of arguments is not necessary. Unfortunately RFC on 
mailcap states that command is executed by shell. On the other hand only 
a few of viewer entries have commands that could not be handled without 
shell.

bug#44824: 27.1; Org export as pdf and open file does not open it

[Maxim Nikulin]

On 31/01/2021 23:33, Eli Zaretskii wrote:
>>
>>>> To fix the problem it is better to use (make-process :connection-type
>>>> 'pipe ...) that unfortunately has no higher level wrappers.
>>>
>>> Wouldn't it work to let-bind process-connection-type to nil around the
>>> function that starts the async subprocess?

>> ...

> Well, I think we should try this, because if it works, it will show us
> a way to fix the problem.  (I don't see how lexical-binding could
> interfere with let-binding.)

It is not a great problem to use make-process:

https://lists.gnu.org/archive/html/emacs-orgmode/2021-01/msg00364.html

I think that at least a :sentinel should be added to improve error 
reporting. The code just before these lines has another problem (I hope 
it is really rare): 
https://lists.gnu.org/archive/html/emacs-orgmode/2021-01/msg00353.html

I would avoid shell since direct execution of the handler should be more 
robust, e.g. quoting of arguments is not necessary. Unfortunately RFC on 
mailcap states that command is executed by shell. On the other hand only 
a few of viewer entries have commands that could not be handled without 
shell.

bug#44824: [PATCH] org.el: Avoid xdg-open silent failure

[Maxim Nikulin]

[-- Attachment #1: Type: text/plain, Size: 1276 bytes --]

On 31/01/2021 23:33, Eli Zaretskii wrote:
>> From: Maxim Nikulin <m.a.nikulin@gmail.com>
>> Date: Sun, 31 Jan 2021 22:57:57 +0700
>> Cc: 44824@debbugs.gnu.org
>>
>>>> To fix the problem it is better to use (make-process :connection-type
>>>> 'pipe ...) that unfortunately has no higher level wrappers.
>>>
>>> Wouldn't it work to let-bind process-connection-type to nil around the
>>> function that starts the async subprocess?
>>
>> Sorry, for me it easier to reason how to express it in terms of system
>> calls and terminal process groups than if let-bind could override a
>> variable when lexical-bind is set to true.
> 
> Well, I think we should try this, because if it works, it will show us
> a way to fix the problem.  (I don't see how lexical-binding could
> interfere with let-binding.)

I have prepared a patch that uses `make-process'. I hope, error 
reporting is improved a bit, but in the case of this particular problem 
failure is still quiet since xdg-open exits successfully.

I could not estimate effect of such change on windows, so pipe process
is used only on linux. I am unsure concerning mac however.

Another question is if failure message should be suppressed when 
`waiting-for-user-input-p' returns truth to keep user prompt and input 
unaffected.

[-- Attachment #2: open-file-pipe-process.patch --]
[-- Type: text/x-patch, Size: 2912 bytes --]

commit 0001bff24864ee16598b4701a05cf40e9abc83af
Author: Max Nikulin <manikulin@gmail.com>
Date:   Wed Feb 17 16:35:58 2021 +0000

    org.el: Avoid xdg-open silent failure
    
    * lisp/org.el (org-open-file): Use 'pipe :connection-type instead of
    'pty to prevent killing of background process on handler exit.
    
    Problem happens only in some desktop environments where configured
    through `org-file-apps' or mailcap handlers launches actual viewer
    (as defined in .desktop files and obtained from mimeapps.list)
    in background.  E.g. xdg-open invokes "gio open" or kde-open5 for Gnome
    or KDE accordingly and these handlers launches e.g. eog or okular in
    background.  As soon as main process exits, temporary terminal session
    created by `start-process-shell-command' is terminated.  As a result
    background processes receive SIGHUP.
    
    Previously command were executed with no buffer, so the change
    does not affect "needsterminal" and "copiousoutput" mailcap features,
    they are not supported as earlier.
    
    If handler main process fails then show a message with exit reason.
    Output (including error messages) is ignored as before.
    Gtk application tends to report significant amount of failed asserts
    hardly informative for majority of users.

diff --git a/lisp/org.el b/lisp/org.el
index 7d8733448..a199a65c9 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -8645,6 +8645,15 @@ opened in Emacs."
    (when add-auto-mode
      (mapcar (lambda (x) (cons (car x) 'emacs)) auto-mode-alist))))
 
+(defun org--error-process-sentinel (proc event)
+  "Show a message if process failed (exited with non-zero code
+or killed by a signal.  Pass the function as :SENTINEL argument
+of `make-process'."
+  (unless (string-match "finished" event)
+    (message "Command %s: %s."
+             (mapconcat 'identity (process-command proc) " ")
+             (substring event 0 -1))))
+
 ;;;###autoload
 (defun org-open-file (path &optional in-emacs line search)
   "Open the file at PATH.
@@ -8766,7 +8775,17 @@ If the file does not exist, throw an error."
 
       (save-window-excursion
 	(message "Running %s...done" cmd)
-	(start-process-shell-command cmd nil cmd)
+	(if (eq system-type 'gnu/linux)
+	  ;; Handlers as "gio open" and kde-open5 start viewer in background
+	  ;; and exit immediately. Avoid start-process since it assumes
+	  ;; :connection-type 'pty and kills children processes with SIGHUP
+	  ;; when temporary terminal session is finished.
+	  (make-process
+	    :name "org-open-file" :connection-type 'pipe :noquery 't
+	    :buffer nil ; use "*Messages*" for debugging
+	    :sentinel 'org--error-process-sentinel
+	    :command (list shell-file-name shell-command-switch cmd))
+	  (start-process-shell-command cmd nil cmd))
 	(and (boundp 'org-wait) (numberp org-wait) (sit-for org-wait))))
      ((or (stringp cmd)
 	  (eq cmd 'emacs))

[PATCH] org.el: Avoid xdg-open silent failure

[Maxim Nikulin]

[-- Attachment #1: Type: text/plain, Size: 1276 bytes --]

On 31/01/2021 23:33, Eli Zaretskii wrote:
>> From: Maxim Nikulin <m.a.nikulin@gmail.com>
>> Date: Sun, 31 Jan 2021 22:57:57 +0700
>> Cc: 44824@debbugs.gnu.org
>>
>>>> To fix the problem it is better to use (make-process :connection-type
>>>> 'pipe ...) that unfortunately has no higher level wrappers.
>>>
>>> Wouldn't it work to let-bind process-connection-type to nil around the
>>> function that starts the async subprocess?
>>
>> Sorry, for me it easier to reason how to express it in terms of system
>> calls and terminal process groups than if let-bind could override a
>> variable when lexical-bind is set to true.
> 
> Well, I think we should try this, because if it works, it will show us
> a way to fix the problem.  (I don't see how lexical-binding could
> interfere with let-binding.)

I have prepared a patch that uses `make-process'. I hope, error 
reporting is improved a bit, but in the case of this particular problem 
failure is still quiet since xdg-open exits successfully.

I could not estimate effect of such change on windows, so pipe process
is used only on linux. I am unsure concerning mac however.

Another question is if failure message should be suppressed when 
`waiting-for-user-input-p' returns truth to keep user prompt and input 
unaffected.

[-- Attachment #2: open-file-pipe-process.patch --]
[-- Type: text/x-patch, Size: 2912 bytes --]

commit 0001bff24864ee16598b4701a05cf40e9abc83af
Author: Max Nikulin <manikulin@gmail.com>
Date:   Wed Feb 17 16:35:58 2021 +0000

    org.el: Avoid xdg-open silent failure
    
    * lisp/org.el (org-open-file): Use 'pipe :connection-type instead of
    'pty to prevent killing of background process on handler exit.
    
    Problem happens only in some desktop environments where configured
    through `org-file-apps' or mailcap handlers launches actual viewer
    (as defined in .desktop files and obtained from mimeapps.list)
    in background.  E.g. xdg-open invokes "gio open" or kde-open5 for Gnome
    or KDE accordingly and these handlers launches e.g. eog or okular in
    background.  As soon as main process exits, temporary terminal session
    created by `start-process-shell-command' is terminated.  As a result
    background processes receive SIGHUP.
    
    Previously command were executed with no buffer, so the change
    does not affect "needsterminal" and "copiousoutput" mailcap features,
    they are not supported as earlier.
    
    If handler main process fails then show a message with exit reason.
    Output (including error messages) is ignored as before.
    Gtk application tends to report significant amount of failed asserts
    hardly informative for majority of users.

diff --git a/lisp/org.el b/lisp/org.el
index 7d8733448..a199a65c9 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -8645,6 +8645,15 @@ opened in Emacs."
    (when add-auto-mode
      (mapcar (lambda (x) (cons (car x) 'emacs)) auto-mode-alist))))
 
+(defun org--error-process-sentinel (proc event)
+  "Show a message if process failed (exited with non-zero code
+or killed by a signal.  Pass the function as :SENTINEL argument
+of `make-process'."
+  (unless (string-match "finished" event)
+    (message "Command %s: %s."
+             (mapconcat 'identity (process-command proc) " ")
+             (substring event 0 -1))))
+
 ;;;###autoload
 (defun org-open-file (path &optional in-emacs line search)
   "Open the file at PATH.
@@ -8766,7 +8775,17 @@ If the file does not exist, throw an error."
 
       (save-window-excursion
 	(message "Running %s...done" cmd)
-	(start-process-shell-command cmd nil cmd)
+	(if (eq system-type 'gnu/linux)
+	  ;; Handlers as "gio open" and kde-open5 start viewer in background
+	  ;; and exit immediately. Avoid start-process since it assumes
+	  ;; :connection-type 'pty and kills children processes with SIGHUP
+	  ;; when temporary terminal session is finished.
+	  (make-process
+	    :name "org-open-file" :connection-type 'pipe :noquery 't
+	    :buffer nil ; use "*Messages*" for debugging
+	    :sentinel 'org--error-process-sentinel
+	    :command (list shell-file-name shell-command-switch cmd))
+	  (start-process-shell-command cmd nil cmd))
 	(and (boundp 'org-wait) (numberp org-wait) (sit-for org-wait))))
      ((or (stringp cmd)
 	  (eq cmd 'emacs))

bug#44824: [PATCH] org.el: Avoid xdg-open silent failure

[Eli Zaretskii]

> From: Maxim Nikulin <m.a.nikulin@gmail.com>
> Date: Thu, 18 Feb 2021 19:56:03 +0700
> Cc: 44824@debbugs.gnu.org
> 
> I could not estimate effect of such change on windows, so pipe process
> is used only on linux. I am unsure concerning mac however.

On Windows Emacs always uses pipes, because we don't have PTYs there.
And there's no xdg-open on MS-Windows anyway, so it's a moot point.

Re: bug#44824: [PATCH] org.el: Avoid xdg-open silent failure

[Maxim Nikulin]

On 18/02/2021 21:48, Eli Zaretskii wrote:
>> From: Maxim Nikulin
>> Date: Thu, 18 Feb 2021 19:56:03 +0700
>>
>> I could not estimate effect of such change on windows, so pipe process
>> is used only on linux. I am unsure concerning mac however.
> 
> On Windows Emacs always uses pipes, because we don't have PTYs there.
> And there's no xdg-open on MS-Windows anyway, so it's a moot point.

Should I consider your response as a suggestion to remove the `if' 
related to `system-type'?

If I remember correctly, on windows it is possible to communicate with a 
process through stdin and stdout only if the application is compiled as 
a *console* one. That is why it is unclear for me, which component 
handles console or GUI distinction when process is specified as pipe in 
lisp code, and whether all arguments of `make-process', that I have 
used, are suitable on windows.

"start file.pdf" executed in cmd.exe launches an application that does 
not block command prompt. In this sense it similar to background 
processes launched by kde-open5 or "gio open". However I am unaware if 
there is something similar to process groups on windows that leads to 
termination of all group members when leader process finishes.

bug#44824: [PATCH] org.el: Avoid xdg-open silent failure

[Eli Zaretskii]

> From: Maxim Nikulin <m.a.nikulin@gmail.com>
> Date: Fri, 19 Feb 2021 19:29:49 +0700
> Cc: 44824@debbugs.gnu.org
> 
> > On Windows Emacs always uses pipes, because we don't have PTYs there.
> > And there's no xdg-open on MS-Windows anyway, so it's a moot point.
> 
> Should I consider your response as a suggestion to remove the `if' 
> related to `system-type'?

Yes, that 'if' isn't necessary.

> If I remember correctly, on windows it is possible to communicate with a 
> process through stdin and stdout only if the application is compiled as 
> a *console* one.

That's true.  But in this case we don't really want to communicate
with the sub-process, do we?  We just want to invoke it and let it
run.  So the fact that there's no way of communicating with the
sub-process is not important here, as the pipes will not be used.  We
just need to specify pipes because that works around the problem with
xdg-open.

> "start file.pdf" executed in cmd.exe launches an application that does 
> not block command prompt. In this sense it similar to background 
> processes launched by kde-open5 or "gio open". However I am unaware if 
> there is something similar to process groups on windows that leads to 
> termination of all group members when leader process finishes.

Things are fairly similar on Windows.  But is this really relevant to
the issue at hand?  There's no xdg-open on Windows, so whatever
problems you had with xdg-open will never happen on Windows.  the
proposed patch fixes the problem only on systems where org.el invokes
the PDF viewer via xdg-open.  Right?

bug#44824: [PATCH] org.el: Avoid xdg-open silent failure

[Eli Zaretskii]

> From: Maxim Nikulin <m.a.nikulin@gmail.com>
> Date: Fri, 19 Feb 2021 19:29:49 +0700
> Cc: 44824@debbugs.gnu.org
> 
> > On Windows Emacs always uses pipes, because we don't have PTYs there.
> > And there's no xdg-open on MS-Windows anyway, so it's a moot point.
> 
> Should I consider your response as a suggestion to remove the `if' 
> related to `system-type'?

Yes, that 'if' isn't necessary.

> If I remember correctly, on windows it is possible to communicate with a 
> process through stdin and stdout only if the application is compiled as 
> a *console* one.

That's true.  But in this case we don't really want to communicate
with the sub-process, do we?  We just want to invoke it and let it
run.  So the fact that there's no way of communicating with the
sub-process is not important here, as the pipes will not be used.  We
just need to specify pipes because that works around the problem with
xdg-open.

> "start file.pdf" executed in cmd.exe launches an application that does 
> not block command prompt. In this sense it similar to background 
> processes launched by kde-open5 or "gio open". However I am unaware if 
> there is something similar to process groups on windows that leads to 
> termination of all group members when leader process finishes.

Things are fairly similar on Windows.  But is this really relevant to
the issue at hand?  There's no xdg-open on Windows, so whatever
problems you had with xdg-open will never happen on Windows.  the
proposed patch fixes the problem only on systems where org.el invokes
the PDF viewer via xdg-open.  Right?

bug#44824: [PATCH] org.el: Avoid xdg-open silent failure

[Maxim Nikulin]

On 19/02/2021 21:54, Eli Zaretskii wrote:
>> From: Maxim Nikulin
>> Date: Fri, 19 Feb 2021 19:29:49 +0700
>>
>>> On Windows Emacs always uses pipes, because we don't have PTYs there.
>>> And there's no xdg-open on MS-Windows anyway, so it's a moot point.
>>
>> Should I consider your response as a suggestion to remove the `if'
>> related to `system-type'?
> 
> Yes, that 'if' isn't necessary.

Let's wait for comments comments from org developers. I do not mind to 
remove the `if' but they could ask for other changes.

>> If I remember correctly, on windows it is possible to communicate with a
>> process through stdin and stdout only if the application is compiled as
>> a *console* one.
> 
> That's true.  But in this case we don't really want to communicate
> with the sub-process, do we?  We just want to invoke it and let it
> run.  So the fact that there's no way of communicating with the
> sub-process is not important here, as the pipes will not be used.  We
> just need to specify pipes because that works around the problem with
> xdg-open.

The reason to switch to pipe process here is to avoid side effects of 
*terminal session*. PTY is redundant here. Pipe process is the only 
alternative available in emacs. Currently there is no data exchange with 
subprocess.

Reading of stderr and stdout from the handler and its descendants could 
be useful to realize a cause of a problem when it happens. At least on 
linux it is enough to specify some buffer instead of `nil' or add a 
:filter function. The reason why I did not make such change is that Gtk 
applications tends to report significant number of failed asserts. I do 
not like to pollute the "*Messages*" buffer. I am in doubts if a 
dedicated buffer should be created for such purpose.

Reading of stdout could be useful in mailcap handlers e.g. to display 
text from PDF file when only text terminal is available (no X11 or 
wayland). Currently emacs ignores "needsterminal" and "copiousoutput" 
flags in mailcap. Example of an entry for PDF:

application/pdf; pdftotext -layout %s -; test=test -z "$DISPLAY"; 
copiousoutput

Likely only a small part of users could benefit from such feature.

>> "start file.pdf" executed in cmd.exe launches an application that does
>> not block command prompt. In this sense it similar to background
>> processes launched by kde-open5 or "gio open". However I am unaware if
>> there is something similar to process groups on windows that leads to
>> termination of all group members when leader process finishes.
> 
> Things are fairly similar on Windows.  But is this really relevant to
> the issue at hand?  There's no xdg-open on Windows, so whatever
> problems you had with xdg-open will never happen on Windows.  the
> proposed patch fixes the problem only on systems where org.el invokes
> the PDF viewer via xdg-open.  Right?

1. It is not a problem of namely xdg-open. It is (mostly reasonable but 
with some caveats however) specific of DE-specific handlers as "gio 
open" and kde-open5. They exit just after spawning of a process with 
actual handler. (Just like "start file.pdf" on windows.) I am afraid 
that I could break something on windows, so I would prefer that somebody 
will test the changes on non-linux systems.

2. It is not specific to PDF files, the problem could happen with any 
format if wrapper handler such as "gio open" is invoked instead of e.g. 
direct execution of eog.

Re: bug#44824: [PATCH] org.el: Avoid xdg-open silent failure

[Maxim Nikulin]

On 19/02/2021 21:54, Eli Zaretskii wrote:
>> From: Maxim Nikulin
>> Date: Fri, 19 Feb 2021 19:29:49 +0700
>>
>>> On Windows Emacs always uses pipes, because we don't have PTYs there.
>>> And there's no xdg-open on MS-Windows anyway, so it's a moot point.
>>
>> Should I consider your response as a suggestion to remove the `if'
>> related to `system-type'?
> 
> Yes, that 'if' isn't necessary.

Let's wait for comments comments from org developers. I do not mind to 
remove the `if' but they could ask for other changes.

>> If I remember correctly, on windows it is possible to communicate with a
>> process through stdin and stdout only if the application is compiled as
>> a *console* one.
> 
> That's true.  But in this case we don't really want to communicate
> with the sub-process, do we?  We just want to invoke it and let it
> run.  So the fact that there's no way of communicating with the
> sub-process is not important here, as the pipes will not be used.  We
> just need to specify pipes because that works around the problem with
> xdg-open.

The reason to switch to pipe process here is to avoid side effects of 
*terminal session*. PTY is redundant here. Pipe process is the only 
alternative available in emacs. Currently there is no data exchange with 
subprocess.

Reading of stderr and stdout from the handler and its descendants could 
be useful to realize a cause of a problem when it happens. At least on 
linux it is enough to specify some buffer instead of `nil' or add a 
:filter function. The reason why I did not make such change is that Gtk 
applications tends to report significant number of failed asserts. I do 
not like to pollute the "*Messages*" buffer. I am in doubts if a 
dedicated buffer should be created for such purpose.

Reading of stdout could be useful in mailcap handlers e.g. to display 
text from PDF file when only text terminal is available (no X11 or 
wayland). Currently emacs ignores "needsterminal" and "copiousoutput" 
flags in mailcap. Example of an entry for PDF:

application/pdf; pdftotext -layout %s -; test=test -z "$DISPLAY"; 
copiousoutput

Likely only a small part of users could benefit from such feature.

>> "start file.pdf" executed in cmd.exe launches an application that does
>> not block command prompt. In this sense it similar to background
>> processes launched by kde-open5 or "gio open". However I am unaware if
>> there is something similar to process groups on windows that leads to
>> termination of all group members when leader process finishes.
> 
> Things are fairly similar on Windows.  But is this really relevant to
> the issue at hand?  There's no xdg-open on Windows, so whatever
> problems you had with xdg-open will never happen on Windows.  the
> proposed patch fixes the problem only on systems where org.el invokes
> the PDF viewer via xdg-open.  Right?

1. It is not a problem of namely xdg-open. It is (mostly reasonable but 
with some caveats however) specific of DE-specific handlers as "gio 
open" and kde-open5. They exit just after spawning of a process with 
actual handler. (Just like "start file.pdf" on windows.) I am afraid 
that I could break something on windows, so I would prefer that somebody 
will test the changes on non-linux systems.

2. It is not specific to PDF files, the problem could happen with any 
format if wrapper handler such as "gio open" is invoked instead of e.g. 
direct execution of eog.

bug#44824: [PATCH] org.el: Avoid xdg-open silent failure

[Maxim Nikulin]

On 18/02/2021 21:48, Eli Zaretskii wrote:
>> From: Maxim Nikulin
>> Date: Thu, 18 Feb 2021 19:56:03 +0700
>>
>> I could not estimate effect of such change on windows, so pipe process
>> is used only on linux. I am unsure concerning mac however.
> 
> On Windows Emacs always uses pipes, because we don't have PTYs there.
> And there's no xdg-open on MS-Windows anyway, so it's a moot point.

Should I consider your response as a suggestion to remove the `if' 
related to `system-type'?

If I remember correctly, on windows it is possible to communicate with a 
process through stdin and stdout only if the application is compiled as 
a *console* one. That is why it is unclear for me, which component 
handles console or GUI distinction when process is specified as pipe in 
lisp code, and whether all arguments of `make-process', that I have 
used, are suitable on windows.

"start file.pdf" executed in cmd.exe launches an application that does 
not block command prompt. In this sense it similar to background 
processes launched by kde-open5 or "gio open". However I am unaware if 
there is something similar to process groups on windows that leads to 
termination of all group members when leader process finishes.

bug#44824: [PATCH] org.el: Avoid xdg-open silent failure

[Kyle Meyer]

Maxim Nikulin writes:

>     org.el: Avoid xdg-open silent failure
>     
>     * lisp/org.el (org-open-file): Use 'pipe :connection-type instead of
>     'pty to prevent killing of background process on handler exit.
>     
>     Problem happens only in some desktop environments where configured
>     through `org-file-apps' or mailcap handlers launches actual viewer
>     (as defined in .desktop files and obtained from mimeapps.list)
>     in background.  E.g. xdg-open invokes "gio open" or kde-open5 for Gnome
>     or KDE accordingly and these handlers launches e.g. eog or okular in
>     background.  As soon as main process exits, temporary terminal session
>     created by `start-process-shell-command' is terminated.  As a result
>     background processes receive SIGHUP.
>     
>     Previously command were executed with no buffer, so the change
>     does not affect "needsterminal" and "copiousoutput" mailcap features,
>     they are not supported as earlier.
>     
>     If handler main process fails then show a message with exit reason.
>     Output (including error messages) is ignored as before.
>     Gtk application tends to report significant amount of failed asserts
>     hardly informative for majority of users.

Thanks for the detailed commit message.

A few comments in addition to Eli's advice to drop the
(eq system-type 'gnu/linux) condition...

> diff --git a/lisp/org.el b/lisp/org.el
> index 7d8733448..a199a65c9 100644
> --- a/lisp/org.el
> +++ b/lisp/org.el
> @@ -8645,6 +8645,15 @@ opened in Emacs."
>     (when add-auto-mode
>       (mapcar (lambda (x) (cons (car x) 'emacs)) auto-mode-alist))))
>  
> +(defun org--error-process-sentinel (proc event)
> +  "Show a message if process failed (exited with non-zero code
> +or killed by a signal.  Pass the function as :SENTINEL argument

Please rework the first sentence so that it fits on the first line,
though I'd be in favor dropping the function and using a lambda in the
make-process call.

> +of `make-process'."
> +  (unless (string-match "finished" event)

There's no need for substring matching, right?  So it could be

  (equal event "finished\n")

Or perhaps

  (when (and (memq (process-status proc) '(exit signal))
             (/= (process-exit-status proc) 0))
    ...)

> +    (message "Command %s: %s."
> +             (mapconcat 'identity (process-command proc) " ")

s/'identity/#'identity/

> +             (substring event 0 -1))))
> +
>  ;;;###autoload
>  (defun org-open-file (path &optional in-emacs line search)
>    "Open the file at PATH.
> @@ -8766,7 +8775,17 @@ If the file does not exist, throw an error."
>  
>        (save-window-excursion
>  	(message "Running %s...done" cmd)
> -	(start-process-shell-command cmd nil cmd)
> +	(if (eq system-type 'gnu/linux)
> +	  ;; Handlers as "gio open" and kde-open5 start viewer in background

s/as/such as/ ?

> +	  ;; and exit immediately. Avoid start-process since it assumes

                                  ^ missing space

> +	  ;; :connection-type 'pty and kills children processes with SIGHUP
> +	  ;; when temporary terminal session is finished.
> +	  (make-process
> +	    :name "org-open-file" :connection-type 'pipe :noquery 't

s/'t/t/

> +	    :buffer nil ; use "*Messages*" for debugging
> +	    :sentinel 'org--error-process-sentinel
> +	    :command (list shell-file-name shell-command-switch cmd))
> +	  (start-process-shell-command cmd nil cmd))
>  	(and (boundp 'org-wait) (numberp org-wait) (sit-for org-wait))))
>       ((or (stringp cmd)
>  	  (eq cmd 'emacs))

Thanks.

bug#44824: [PATCH] org.el: Avoid xdg-open silent failure

[Kyle Meyer]

Maxim Nikulin writes:

>     org.el: Avoid xdg-open silent failure
>     
>     * lisp/org.el (org-open-file): Use 'pipe :connection-type instead of
>     'pty to prevent killing of background process on handler exit.
>     
>     Problem happens only in some desktop environments where configured
>     through `org-file-apps' or mailcap handlers launches actual viewer
>     (as defined in .desktop files and obtained from mimeapps.list)
>     in background.  E.g. xdg-open invokes "gio open" or kde-open5 for Gnome
>     or KDE accordingly and these handlers launches e.g. eog or okular in
>     background.  As soon as main process exits, temporary terminal session
>     created by `start-process-shell-command' is terminated.  As a result
>     background processes receive SIGHUP.
>     
>     Previously command were executed with no buffer, so the change
>     does not affect "needsterminal" and "copiousoutput" mailcap features,
>     they are not supported as earlier.
>     
>     If handler main process fails then show a message with exit reason.
>     Output (including error messages) is ignored as before.
>     Gtk application tends to report significant amount of failed asserts
>     hardly informative for majority of users.

Thanks for the detailed commit message.

A few comments in addition to Eli's advice to drop the
(eq system-type 'gnu/linux) condition...

> diff --git a/lisp/org.el b/lisp/org.el
> index 7d8733448..a199a65c9 100644
> --- a/lisp/org.el
> +++ b/lisp/org.el
> @@ -8645,6 +8645,15 @@ opened in Emacs."
>     (when add-auto-mode
>       (mapcar (lambda (x) (cons (car x) 'emacs)) auto-mode-alist))))
>  
> +(defun org--error-process-sentinel (proc event)
> +  "Show a message if process failed (exited with non-zero code
> +or killed by a signal.  Pass the function as :SENTINEL argument

Please rework the first sentence so that it fits on the first line,
though I'd be in favor dropping the function and using a lambda in the
make-process call.

> +of `make-process'."
> +  (unless (string-match "finished" event)

There's no need for substring matching, right?  So it could be

  (equal event "finished\n")

Or perhaps

  (when (and (memq (process-status proc) '(exit signal))
             (/= (process-exit-status proc) 0))
    ...)

> +    (message "Command %s: %s."
> +             (mapconcat 'identity (process-command proc) " ")

s/'identity/#'identity/

> +             (substring event 0 -1))))
> +
>  ;;;###autoload
>  (defun org-open-file (path &optional in-emacs line search)
>    "Open the file at PATH.
> @@ -8766,7 +8775,17 @@ If the file does not exist, throw an error."
>  
>        (save-window-excursion
>  	(message "Running %s...done" cmd)
> -	(start-process-shell-command cmd nil cmd)
> +	(if (eq system-type 'gnu/linux)
> +	  ;; Handlers as "gio open" and kde-open5 start viewer in background

s/as/such as/ ?

> +	  ;; and exit immediately. Avoid start-process since it assumes

                                  ^ missing space

> +	  ;; :connection-type 'pty and kills children processes with SIGHUP
> +	  ;; when temporary terminal session is finished.
> +	  (make-process
> +	    :name "org-open-file" :connection-type 'pipe :noquery 't

s/'t/t/

> +	    :buffer nil ; use "*Messages*" for debugging
> +	    :sentinel 'org--error-process-sentinel
> +	    :command (list shell-file-name shell-command-switch cmd))
> +	  (start-process-shell-command cmd nil cmd))
>  	(and (boundp 'org-wait) (numberp org-wait) (sit-for org-wait))))
>       ((or (stringp cmd)
>  	  (eq cmd 'emacs))

Thanks.

Re: bug#44824: [PATCH] org.el: Avoid xdg-open silent failure

[Maxim Nikulin]

[-- Attachment #1: Type: text/plain, Size: 1782 bytes --]

On 19/03/2021 10:50, Kyle Meyer wrote:
> Maxim Nikulin writes:
> A few comments in addition to Eli's advice to drop the
> (eq system-type 'gnu/linux) condition...

Feel free to commit the change suggested in
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=44824#82
instead of this patch.

>> +(defun org--error-process-sentinel (proc event)
>> +  "Show a message if process failed (exited with non-zero code
>> +or killed by a signal.  Pass the function as :SENTINEL argument
> 
> Please rework the first sentence so that it fits on the first line,
> though I'd be in favor dropping the function and using a lambda in the
> make-process call.

My impression is that org-open-file function is already too long and 
complex. Another reason to use standalone function is that I am unsure 
if elisp compiler and interpreter are smart enough to reuse single 
instance of lambda. I was afraid that every opened file caused creation 
of new sentinel possibly with a closure containing chain of stack 
frames. On the other hand even in worst case memory footprint is 
negligible in comparison to any GUI viewer.

>> +  (unless (string-match "finished" event)
> 
> There's no need for substring matching, right?  So it could be
> 
>    (equal event "finished\n")

I was surprised by final "\n" that is not always suitable and I was in 
doubts concerning its stability. I would prefer something like

     (starts-with event "finished")

Certainly match-data is not necessary, so even match-string-p is better.

>    (when (and (memq (process-status proc) '(exit signal))
>               (/= (process-exit-status proc) 0))

Thank you, I was too lazy to implement such kind of check myself. 
Certainly this variant is better.

I hope, I have addressed other your comments in the updated patch.


[-- Attachment #2: org-open-file-make-process-v2.patch --]
[-- Type: text/x-patch, Size: 2616 bytes --]

commit 5eca7764d94dd46b9f9a7792d1b786a3f03b20b6
Author: Max Nikulin <manikulin@gmail.com>
Date:   Wed Feb 17 16:35:58 2021 +0000

    org.el: Avoid xdg-open silent failure
    
    * lisp/org.el (org-open-file): Use 'pipe :connection-type instead of
    'pty to prevent killing of background process on handler exit.
    (Bug#44824)
    
    Problem happens only in some desktop environments where configured
    through `org-file-apps' or mailcap handlers launches actual viewer
    (as defined in .desktop files and obtained from mimeapps.list)
    in background.  E.g. xdg-open invokes "gio open" or kde-open5 for Gnome
    or KDE accordingly and these handlers launches e.g. eog or okular in
    background.  As soon as main process exits, temporary terminal session
    created by `start-process-shell-command' is terminated.  As a result
    background processes receive SIGHUP.
    
    Previously command were executed with no buffer, so the change
    does not affect "needsterminal" and "copiousoutput" mailcap features,
    they are not supported as earlier.
    
    If handler main process fails then show a message with exit reason.
    Output (including error messages) is ignored as before.
    Gtk application tends to report significant amount of failed asserts
    hardly informative for majority of users.

diff --git a/lisp/org.el b/lisp/org.el
index 4db2dbe04..c58708a5f 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -8768,7 +8768,21 @@ If the file does not exist, throw an error."
 
       (save-window-excursion
 	(message "Running %s...done" cmd)
-	(start-process-shell-command cmd nil cmd)
+        ;; Handlers such as "gio open" and kde-open5 start viewer in background
+        ;; and exit immediately.  Avoid `start-process' since it assumes
+        ;; :connection-type 'pty and kills children processes with SIGHUP
+        ;; when temporary terminal session is finished.
+        (make-process
+         :name "org-open-file" :connection-type 'pipe :noquery t
+         :buffer nil ; use "*Messages*" for debugging
+         :sentinel (lambda (proc event)
+                     (when (and (memq (process-status proc) '(exit signal))
+                                (/= (process-exit-status proc) 0))
+                       (message
+                        "Command %s: %s."
+                        (mapconcat #'identity (process-command proc) " ")
+                        (substring event 0 -1))))
+         :command (list shell-file-name shell-command-switch cmd))
 	(and (boundp 'org-wait) (numberp org-wait) (sit-for org-wait))))
      ((or (stringp cmd)
 	  (eq cmd 'emacs))

bug#44824: [PATCH] org.el: Avoid xdg-open silent failure

[Kyle Meyer]

Maxim Nikulin writes:

> I hope, I have addressed other your comments in the updated patch.
>
> commit 5eca7764d94dd46b9f9a7792d1b786a3f03b20b6
> Author: Max Nikulin <manikulin@gmail.com>
> Date:   Wed Feb 17 16:35:58 2021 +0000
>
>     org.el: Avoid xdg-open silent failure

Thanks.  A note for future patches: your patch isn't in a format that's
ready to be consumed by git-am.  git-format-patch can help you here.

Applied to the Org repo (5db61eb0f), adding a TINYCHANGE cookie to the
commit message.  Please consider completing the copyright paperwork for
future patches
(see <https://orgmode.org/worg/org-contribute.html#copyright-issues>).

If I understand correctly, this bug can be closed, but please reopen if
I'm mistaken.

bug#44824: [PATCH] org.el: Avoid xdg-open silent failure

[Kyle Meyer]

Maxim Nikulin writes:

> I hope, I have addressed other your comments in the updated patch.
>
> commit 5eca7764d94dd46b9f9a7792d1b786a3f03b20b6
> Author: Max Nikulin <manikulin@gmail.com>
> Date:   Wed Feb 17 16:35:58 2021 +0000
>
>     org.el: Avoid xdg-open silent failure

Thanks.  A note for future patches: your patch isn't in a format that's
ready to be consumed by git-am.  git-format-patch can help you here.

Applied to the Org repo (5db61eb0f), adding a TINYCHANGE cookie to the
commit message.  Please consider completing the copyright paperwork for
future patches
(see <https://orgmode.org/worg/org-contribute.html#copyright-issues>).

If I understand correctly, this bug can be closed, but please reopen if
I'm mistaken.

bug#44824: [PATCH] org.el: Avoid xdg-open silent failure

[Maxim Nikulin]

[-- Attachment #1: Type: text/plain, Size: 1782 bytes --]

On 19/03/2021 10:50, Kyle Meyer wrote:
> Maxim Nikulin writes:
> A few comments in addition to Eli's advice to drop the
> (eq system-type 'gnu/linux) condition...

Feel free to commit the change suggested in
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=44824#82
instead of this patch.

>> +(defun org--error-process-sentinel (proc event)
>> +  "Show a message if process failed (exited with non-zero code
>> +or killed by a signal.  Pass the function as :SENTINEL argument
> 
> Please rework the first sentence so that it fits on the first line,
> though I'd be in favor dropping the function and using a lambda in the
> make-process call.

My impression is that org-open-file function is already too long and 
complex. Another reason to use standalone function is that I am unsure 
if elisp compiler and interpreter are smart enough to reuse single 
instance of lambda. I was afraid that every opened file caused creation 
of new sentinel possibly with a closure containing chain of stack 
frames. On the other hand even in worst case memory footprint is 
negligible in comparison to any GUI viewer.

>> +  (unless (string-match "finished" event)
> 
> There's no need for substring matching, right?  So it could be
> 
>    (equal event "finished\n")

I was surprised by final "\n" that is not always suitable and I was in 
doubts concerning its stability. I would prefer something like

     (starts-with event "finished")

Certainly match-data is not necessary, so even match-string-p is better.

>    (when (and (memq (process-status proc) '(exit signal))
>               (/= (process-exit-status proc) 0))

Thank you, I was too lazy to implement such kind of check myself. 
Certainly this variant is better.

I hope, I have addressed other your comments in the updated patch.


[-- Attachment #2: org-open-file-make-process-v2.patch --]
[-- Type: text/x-patch, Size: 2616 bytes --]

commit 5eca7764d94dd46b9f9a7792d1b786a3f03b20b6
Author: Max Nikulin <manikulin@gmail.com>
Date:   Wed Feb 17 16:35:58 2021 +0000

    org.el: Avoid xdg-open silent failure
    
    * lisp/org.el (org-open-file): Use 'pipe :connection-type instead of
    'pty to prevent killing of background process on handler exit.
    (Bug#44824)
    
    Problem happens only in some desktop environments where configured
    through `org-file-apps' or mailcap handlers launches actual viewer
    (as defined in .desktop files and obtained from mimeapps.list)
    in background.  E.g. xdg-open invokes "gio open" or kde-open5 for Gnome
    or KDE accordingly and these handlers launches e.g. eog or okular in
    background.  As soon as main process exits, temporary terminal session
    created by `start-process-shell-command' is terminated.  As a result
    background processes receive SIGHUP.
    
    Previously command were executed with no buffer, so the change
    does not affect "needsterminal" and "copiousoutput" mailcap features,
    they are not supported as earlier.
    
    If handler main process fails then show a message with exit reason.
    Output (including error messages) is ignored as before.
    Gtk application tends to report significant amount of failed asserts
    hardly informative for majority of users.

diff --git a/lisp/org.el b/lisp/org.el
index 4db2dbe04..c58708a5f 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -8768,7 +8768,21 @@ If the file does not exist, throw an error."
 
       (save-window-excursion
 	(message "Running %s...done" cmd)
-	(start-process-shell-command cmd nil cmd)
+        ;; Handlers such as "gio open" and kde-open5 start viewer in background
+        ;; and exit immediately.  Avoid `start-process' since it assumes
+        ;; :connection-type 'pty and kills children processes with SIGHUP
+        ;; when temporary terminal session is finished.
+        (make-process
+         :name "org-open-file" :connection-type 'pipe :noquery t
+         :buffer nil ; use "*Messages*" for debugging
+         :sentinel (lambda (proc event)
+                     (when (and (memq (process-status proc) '(exit signal))
+                                (/= (process-exit-status proc) 0))
+                       (message
+                        "Command %s: %s."
+                        (mapconcat #'identity (process-command proc) " ")
+                        (substring event 0 -1))))
+         :command (list shell-file-name shell-command-switch cmd))
 	(and (boundp 'org-wait) (numberp org-wait) (sit-for org-wait))))
      ((or (stringp cmd)
 	  (eq cmd 'emacs))

bug#44824: 27.1; Org export as pdf and open file does not open it

[Eli Zaretskii]

> From: Maxim Nikulin <m.a.nikulin@gmail.com>
> Date: Sat, 30 Jan 2021 20:31:53 +0700
> Cc: gbiotti@gmail.com
> 
> > How about asking the xdg-open developers to help us figure out the
> > reason?  Or, failing that, debug xdg-open in the problematic
> > situations to find out what fails there and why?  E.g., could it be
> > that it fails because stdin/stdout is a PTY? what happens if you bind
> > process-connection-type to nil when starting the async subprocess?
> 
> I do not think, it is xdg-open problem. It just calls kde-open5 that 
> spawns actual handler and immediately exits.

I didn't say it was their problem, I suggested to ask them to help us
understand why xdg-open doesn't work in those cases, under the
assumption that they are familiar with their code better than us.

bug#44824: 27.1; Org export as pdf and open file does not open it

[gbiotti]

[-- Attachment #1: Type: text/plain, Size: 1538 bytes --]

Il 30/01/2021 14:31, Maxim Nikulin ha wrote:
> On 30/01/2021 15:42, Eli Zaretskii wrote:
>>>
>>> This works:
>>> M-! xdg-open /tmp/test.pdf RET
>>>
>>> This doesn't work:
>>> M-& xdg-open /tmp/test.pdf RET
>>>
>>> This doesn't work:
>>> M-x shell RET xdg-open /tmp/test.pdf RET
>
> Geraldo, "M-x shell" case is rather strange. Could you, please, 
> confirm ones more that okular window with the file content does not 
> appear if you call xdg-open from an *interactive* emacs shell buffer? 
> The link to an emacs-orgmode list message, that I have posted earlier, 
> explains why async-shell-command *may* fail while shell-command should 
> work reliably. I am really surprised by failure when command is 
> executed in a [e]shell buffer.
>

I confirm.
I can see the Okular icon appear and disappear immediately in the panel.
As mentioned I can make a video of everything, but I have no idea where 
to upload it.
If it is okay to make the video and you think it is useful please tell 
me which commands to execute for more information on the operating 
environment

>> How about asking the xdg-open developers to help us figure out the
>> reason?  Or, failing that, debug xdg-open in the problematic
>> situations to find out what fails there and why?  E.g., could it be
>> that it fails because stdin/stdout is a PTY? what happens if you bind
>> process-connection-type to nil when starting the async subprocess?
>
> I do not think, it is xdg-open problem. It just calls kde-open5 that 
> spawns actual handler and immediately exits.


[-- Attachment #2: Type: text/html, Size: 4325 bytes --]

bug#44824: 27.1; Org export as pdf and open file does not open it

[Bhavin Gandhi]

On Sat, 30 Jan 2021 at 19:04, Maxim Nikulin <m.a.nikulin@gmail.com> wrote:
> […]
>
> Geraldo, "M-x shell" case is rather strange. Could you, please, confirm
> ones more that okular window with the file content does not appear if
> you call xdg-open from an *interactive* emacs shell buffer? The link to
> an emacs-orgmode list message, that I have posted earlier, explains why
> async-shell-command *may* fail while shell-command should work reliably.
> I am really surprised by failure when command is executed in a [e]shell
> buffer.
>

I was expecting something similar, surprisingly here is what I observed
on my system (with emacs -Q, GNOME and Evince — Document Viewer).

M-x eshell
$ xdg-open ~/Documents/test.pdf
[Nothing happens]

M-x shell
$ xdg-open ~/Documents/test.pdf
[Evince pops up with the PDF]

M-! xdg-open ~/Documents/test.pdf
[Evince pops up with the PDF]

M-& xdg-open ~/Documents/test.pdf
[Nothing happens]

Emacs: 27.1.91
GNOME: 3.38.3
xdg-open 1.1.3+
-- 
Warm Regards,
Bhavin Gandhi (bhavin192) | https://geeksocket.in

bug#44824: 27.1; Org export as pdf and open file does not open it

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> This doesn't work:
>> M-x shell RET xdg-open /tmp/test.pdf RET
>
> How about asking the xdg-open developers to help us figure out the
> reason?  Or, failing that, debug xdg-open in the problematic
> situations to find out what fails there and why?  E.g., could it be
> that it fails because stdin/stdout is a PTY? what happens if you bind
> process-connection-type to nil when starting the async subprocess?

I'm unable to reproduce the problem at all -- all the various ways of
calling xdg-open work fine for me (on this Debian bullseye laptop w/
Gnome Shell).

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: bug#44824: 27.1; Org export as pdf and open file does not open it

[Tim Cross]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> This doesn't work:
>>> M-x shell RET xdg-open /tmp/test.pdf RET
>>
>> How about asking the xdg-open developers to help us figure out the
>> reason?  Or, failing that, debug xdg-open in the problematic
>> situations to find out what fails there and why?  E.g., could it be
>> that it fails because stdin/stdout is a PTY? what happens if you bind
>> process-connection-type to nil when starting the async subprocess?
>
> I'm unable to reproduce the problem at all -- all the various ways of
> calling xdg-open work fine for me (on this Debian bullseye laptop w/
> Gnome Shell).

For me, I get

M-! xdg-open /tmp/test.pdf works
M-x shell <ret> xdg-open /tmp/test.pdf works
M-& xdg-open /tmp/test.pdf fails
M-x eshell <ret> xdg-open /tmp/test.pdf fails

The two which fail do not report any error - just now pdf viewer open.

I also have no problems with org export menu when I choose export to pdf
and open.

This was on a Ubuntu 20.10, latest Emacs 27 (27.5.91), mate DE and
default shell zsh.

--
Tim Cross

Re: bug#44824: 27.1; Org export as pdf and open file does not open it

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1747 bytes --]

On Sun, Jan 31, 2021 at 06:39:40PM +1100, Tim Cross wrote:
> 
> Lars Ingebrigtsen <larsi@gnus.org> writes:
> 
> > Eli Zaretskii <eliz@gnu.org> writes:
> >
> >>> This doesn't work:
> >>> M-x shell RET xdg-open /tmp/test.pdf RET
> >>
> >> How about asking the xdg-open developers to help us figure out the
> >> reason?  Or, failing that, debug xdg-open in the problematic
> >> situations to find out what fails there and why?  E.g., could it be
> >> that it fails because stdin/stdout is a PTY? what happens if you bind
> >> process-connection-type to nil when starting the async subprocess?
> >
> > I'm unable to reproduce the problem at all -- all the various ways of
> > calling xdg-open work fine for me (on this Debian bullseye laptop w/
> > Gnome Shell).
> 
> For me, I get
> 
> M-! xdg-open /tmp/test.pdf works
> M-x shell <ret> xdg-open /tmp/test.pdf works
> M-& xdg-open /tmp/test.pdf fails
> M-x eshell <ret> xdg-open /tmp/test.pdf fails
> 
> The two which fail do not report any error - just now pdf viewer open.
> 
> I also have no problems with org export menu when I choose export to pdf
> and open.
> 
> This was on a Ubuntu 20.10, latest Emacs 27 (27.5.91), mate DE and
> default shell zsh.

This chaotic behaviour gives me the impression that it's an environment
thing: desktop environments have the tendency to prime the environment
variables in "creative" ways, often different from what a login shell
would do.

One var to look for would be, of course, PATH, but I don't know what
other environment variables `xdg-open' looks at; perhaps USER, HOME
are relevant, perhaps more.

May be comparing what `env' says under those different invocations
could shed some light.

Cheers
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

bug#44824: 27.1; Org export as pdf and open file does not open it

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Sat, 30 Jan 2021 07:09:50 +0100
> Cc: 44824@debbugs.gnu.org
> 
> This works:
> M-! xdg-open /tmp/test.pdf RET
> 
> This doesn't work:
> M-& xdg-open /tmp/test.pdf RET
> 
> This doesn't work:
> M-x shell RET xdg-open /tmp/test.pdf RET

How about asking the xdg-open developers to help us figure out the
reason?  Or, failing that, debug xdg-open in the problematic
situations to find out what fails there and why?  E.g., could it be
that it fails because stdin/stdout is a PTY? what happens if you bind
process-connection-type to nil when starting the async subprocess?

bug#44824: 27.1; Org export as pdf and open file does not open it

[Maxim Nikulin]

On 28/01/2021 18:31, gbiotti@gmail.com wrote:
> 
> If I try in eshell buffer 'xdg-open /tmp/test.pdf && sleep 3' my cursor 
> blinks
> with the Okular icon for a few seconds and then nothing happens.

If I correctly get what you describe as "blinks", it could last for some 
time after process failure. On the other hand I am surprised that no 
other diagnostics is shown as a KDE dialog or as an error message in the 
eshell buffer. In the case of eshell, it is likely another problem, not 
killing handler with SIGHUP due to instantly closed pty.

What happens if you run the following commands in the Emacs *eshell 
buffer* (with an existing PDF file, of course)? In particular I am 
interested in output appeared in the buffer.

1. okular /tmp/test.pdf

2. kde-open5 /tmp/test.pdf

3. sh -x /usr/bin/xdg-open /tmp/test.pdf

I'm almost sure that okular is configured as the handler, but just to 
confirm it:

4. xdg-mime query default application/pdf

I assume that PDF file is a valid one (not corrupted, not empty, not 
accessible due to file permissions, etc.) since okular is able to report 
such problems.

You could also attach to emacs process with strace, try to launch 
handler, kill strace, and inspect log file which processes are spawned 
("execve" lines) and their exit codes. Run strace from ordinary terminal 
(not from emacs) with proper process ID:

strace -o /tmp/emacs.strace -f -s 128 -tt -p PID_OF_EMACS_PROCESS

bug#44824: 27.1; Org export as pdf and open file does not open it

[Maxim Nikulin]

On 27/01/2021 10:36, Lars Ingebrigtsen wrote:
> 
>> Executing the command "xdg-open /path/to/file.pdf" in a terminal
>> (Konsole) works.

The problem may be related to SIGHUP sent to children due to pty created 
by emacs and closed as soon as the handler exits:

https://orgmode.org/list/ru4d75$11sc$1@ciao.gmane.io/

As a workaround you could try to set another handler for PDF files in 
mailcap, e.g. okular directly without xdg-open wrapper. Handler should 
not try to launch actual application in background with immediate exit.

Currently start-process elisp function is used internally in 
org-open-file. Low level (make-process :connection-type 'pipe ...) 
should allow to avoid pty+SIGHUP problem. Maybe there are some 
consequences of such change that I am not aware of.

bug#44824: 27.1; Org export as pdf and open file does not open it

[Glenn Morris]

Ref eg https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25234#8
https://bugzilla.gnome.org/show_bug.cgi?id=652262
https://gitlab.gnome.org/GNOME/glib/-/issues/1208
and others going back over a decade.
I think Emacs should have a PROBLEMS entry about this.

bug#44824: 27.1; Org export as pdf and open file does not open it

[Glenn Morris]

Ref eg https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25234#8
https://bugzilla.gnome.org/show_bug.cgi?id=652262
https://gitlab.gnome.org/GNOME/glib/-/issues/1208
and others going back over a decade.
I think Emacs should have a PROBLEMS entry about this.

Re: bug#44824: 27.1; Org export as pdf and open file does not open it

[Maxim Nikulin]

On 27/01/2021 10:36, Lars Ingebrigtsen wrote:
> 
>> Executing the command "xdg-open /path/to/file.pdf" in a terminal
>> (Konsole) works.

The problem may be related to SIGHUP sent to children due to pty created 
by emacs and closed as soon as the handler exits:

https://orgmode.org/list/ru4d75$11sc$1@ciao.gmane.io/

As a workaround you could try to set another handler for PDF files in 
mailcap, e.g. okular directly without xdg-open wrapper. Handler should 
not try to launch actual application in background with immediate exit.

Currently start-process elisp function is used internally in 
org-open-file. Low level (make-process :connection-type 'pipe ...) 
should allow to avoid pty+SIGHUP problem. Maybe there are some 
consequences of such change that I am not aware of.

bug#44824: 27.1; Org export as pdf and open file does not open it

[Maxim Nikulin]

I hope, the following link to another emacs-orgmode mail list archive 
will not be mangled by the debbugs web interface, unlike the previous one:
https://lists.gnu.org/archive/html/emacs-orgmode/2021-01/msg00327.html