bug#36771: 27.0.50; mailcap documentation and effect missmatch

bug#36771: 27.0.50; mailcap documentation and effect missmatch

[Tomas Nordin]

Hello Emacs

emacs -Q

In the scratch buffer, eval each of the following:

(require 'mailcap)
(mailcap-mime-info "text/html") ;; -> "view %s"
mailcap-prefer-mailcap-viewers ;; -> t
(setq mailcap-prefer-mailcap-viewers nil) ;; -> nil
(mailcap-mime-info "text/html") ;; -> "/usr/bin/firefox-esr %s"

I expect the result of mailcap-mime-info to be "/usr/bin/firefox-esr %s"
when mailcap-prefer-mailcap-viewers is t because the doc of that
variable says

   If non-nil, prefer viewers specified in ~/.mailcap.
   If nil, the most specific viewer will be chosen, even if there is
   a general override in ~/.mailcap.  For instance, if /etc/mailcap
   has an entry for "image/gif", that one will be chosen even if
   you have an entry for "image/*" in your ~/.mailcap file.

and

$ cat ~/.mailcap | grep '^text/html'
text/html; /usr/bin/firefox-esr %s; description=HTML Text; test=test -n "$DISPLAY";  nametemplate=%s.html

What do you think? 

If you agree this is a bug, I could provide that a bisect session gave
me that it started to behave this way by commit
7e47d44da4b54c518c5e09b4f3d58dafdd43033d

Before that I think I got "/usr/bin/firefox-esr %s" by default. It was
after a rebuild of emacs I noted a difference using notmuch trying to
view some html part in external viewer.

In addition I am confused why mailcap-mime-info produces "view %s" in
any case because

$ cat /etc/mailcap | grep '^text/html'
text/html; /usr/bin/sensible-browser %s; description=HTML Text; nametemplate=%s.html
text/html; /usr/bin/firefox-esr %s; description=HTML Text; test=test -n "$DISPLAY";  nametemplate=%s.html
text/html; gnome-builder %s; test=test -n "$DISPLAY"
text/html; /usr/bin/w3m -T text/html %s; needsterminal; description=HTML Text; nametemplate=%s.html
text/html; /usr/bin/w3m -I %{charset} -dump -T text/html %s; copiousoutput; description=HTML Text; nametemplate=%s.html

I would naively think that the above would result in sensible-browser maybe.

In GNU Emacs 27.0.50 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.24.5)
 of 2019-07-16 built on fliptop
Repository revision: 6253541c76a449780815f4a8fd75a9aa70b931ae
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12004000
System Description: Debian GNU/Linux 10 (buster)

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Mark set
Mark saved where search started
tn-eval-insert-last-sexp
mailcap
"view %s"
t
nil
"/usr/bin/firefox-esr %s"

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND GPM DBUS GSETTINGS GLIB NOTIFY INOTIFY
ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM THREADS PDUMPER LCMS2 GMP

Important settings:
  value of $LC_TIME: sv_SE.UTF-8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
format-spec subr-x rfc822 mml easymenu mml-sec password-cache epa
derived epg epg-config gnus-util rmail rmail-loaddefs
text-property-search time-date seq byte-opt gv bytecomp byte-compile
cconv mm-decode mm-bodies mm-encode mailabbrev gmm-utils mailheader
cl-loaddefs cl-lib sendmail mail-utils mail-parse rfc2231 rfc2047
rfc2045 mm-util ietf-drums mail-prsvr mailcap misearch multi-isearch
elec-pair tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page menu-bar
rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 56375 8481)
 (symbols 48 6019 1)
 (strings 32 20246 1857)
 (string-bytes 1 576796)
 (vectors 16 9978)
 (vector-slots 8 127710 7016)
 (floats 8 18 40)
 (intervals 56 431 1)
 (buffers 992 12))

bug#36771: 27.0.50; mailcap documentation and effect missmatch

[Stefan Kangas]

Tomas Nordin <tomasn@posteo.net> writes:

> emacs -Q
>
> In the scratch buffer, eval each of the following:
>
> (require 'mailcap)
> (mailcap-mime-info "text/html") ;; -> "view %s"
> mailcap-prefer-mailcap-viewers ;; -> t
> (setq mailcap-prefer-mailcap-viewers t) ;; -> nil
> (mailcap-mime-info "text/html") ;; -> "/usr/bin/firefox-esr %s"
>
> I expect the result of mailcap-mime-info to be "/usr/bin/firefox-esr %s"
> when mailcap-prefer-mailcap-viewers is t because the doc of that
> variable says
>
>    If non-nil, prefer viewers specified in ~/.mailcap.
>    If nil, the most specific viewer will be chosen, even if there is
>    a general override in ~/.mailcap.  For instance, if /etc/mailcap
>    has an entry for "image/gif", that one will be chosen even if
>    you have an entry for "image/*" in your ~/.mailcap file.
>
> and
>
> $ cat ~/.mailcap | grep '^text/html'
> text/html; /usr/bin/firefox-esr %s; description=HTML Text; test=test -n "$DISPLAY";  nametemplate=%s.html
>
> What do you think?

Indeed, this seems to me like it's contradicting the doc string.

> If you agree this is a bug, I could provide that a bisect session gave
> me that it started to behave this way by commit
> 7e47d44da4b54c518c5e09b4f3d58dafdd43033d

Lars, that commit is yours.  What do you think?

I tried digging into this code, but found mailcap.el too confusing overall...

> In addition I am confused why mailcap-mime-info produces "view %s" in
> any case because
>
> $ cat /etc/mailcap | grep '^text/html'
> text/html; /usr/bin/sensible-browser %s; description=HTML Text; nametemplate=%s.html
> text/html; /usr/bin/firefox-esr %s; description=HTML Text; test=test -n "$DISPLAY";  nametemplate=%s.html
> text/html; gnome-builder %s; test=test -n "$DISPLAY"
> text/html; /usr/bin/w3m -T text/html %s; needsterminal; description=HTML Text; nametemplate=%s.html
> text/html; /usr/bin/w3m -I %{charset} -dump -T text/html %s; copiousoutput; description=HTML Text; nametemplate=%s.html
>
> I would naively think that the above would result in sensible-browser maybe.

Agreed.  Here I would have expected to get the more specific entry, but
I get the least specific entry.  On my system "view" in /etc/mailcap is:

text/*; view %s; edit=vim %s; compose=vim %s; test=test -x
/usr/bin/vim; needsterminal
text/*; view %s; edit=vi %s; compose=vi %s; needsterminal

Best regards,
Stefan Kangas

bug#36771: 27.0.50; mailcap documentation and effect missmatch

[Lars Ingebrigtsen]

Stefan Kangas <stefan@marxist.se> writes:

> Agreed.  Here I would have expected to get the more specific entry, but
> I get the least specific entry.  On my system "view" in /etc/mailcap is:
>
> text/*; view %s; edit=vim %s; compose=vim %s; test=test -x
> /usr/bin/vim; needsterminal
> text/*; view %s; edit=vi %s; compose=vi %s; needsterminal

You used to get the most specific entry no matter where it was, but I
think the change was supposed to allow ~/.mailcap to override even
more-specific entries in /etc/mailcap.

So text/* in ~/.mailcap will win over text/plain in /etc/mailcap.

But I haven't looked closely at this bug report...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#36771: 27.0.50; mailcap documentation and effect missmatch

[Tomas Nordin]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Stefan Kangas <stefan@marxist.se> writes:
>
>> Agreed.  Here I would have expected to get the more specific entry, but
>> I get the least specific entry.  On my system "view" in /etc/mailcap is:
>>
>> text/*; view %s; edit=vim %s; compose=vim %s; test=test -x
>> /usr/bin/vim; needsterminal
>> text/*; view %s; edit=vi %s; compose=vi %s; needsterminal
>
> You used to get the most specific entry no matter where it was, but I
> think the change was supposed to allow ~/.mailcap to override even
> more-specific entries in /etc/mailcap.
>
> So text/* in ~/.mailcap will win over text/plain in /etc/mailcap.
>
> But I haven't looked closely at this bug report...

I could maybe summarize the report by this bald claim: The effective
logic of the `mailcap-prefer-mailcap-viewers` variable is reversed
relative its docstring.

What do you think?

Best regards
--
Tomas

bug#36771: 27.0.50; mailcap documentation and effect missmatch

[Lars Ingebrigtsen]

Stefan Kangas <stefan@marxist.se> writes:

> I tried digging into this code, but found mailcap.el too confusing overall...

Yes, it's really, really confusing.

There's so many reversals in the way things are sorted that it's amazing
that it sometimes halfway works at all.

I think I'll rewrite it some, and mark stuff explicitly where they came
from (user/Emacs/system), and then use that to decide what to use.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#36771: 27.0.50; mailcap documentation and effect missmatch

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Stefan Kangas <stefan@marxist.se> writes:
>
>> I tried digging into this code, but found mailcap.el too confusing overall...
>
> Yes, it's really, really confusing.
>
> There's so many reversals in the way things are sorted that it's amazing
> that it sometimes halfway works at all.
>
> I think I'll rewrite it some, and mark stuff explicitly where they came
> from (user/Emacs/system), and then use that to decide what to use.

I think this should perhaps now work on the trunk -- at least it works
in my test cases.  Could you test?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#36771: 27.0.50; mailcap documentation and effect missmatch

[Tomas Nordin]

Hello

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Lars Ingebrigtsen <larsi@gnus.org> writes:
>
>> Stefan Kangas <stefan@marxist.se> writes:
>>
>>> I tried digging into this code, but found mailcap.el too confusing overall...
>>
>> Yes, it's really, really confusing.
>>
>> There's so many reversals in the way things are sorted that it's amazing
>> that it sometimes halfway works at all.
>>
>> I think I'll rewrite it some, and mark stuff explicitly where they came
>> from (user/Emacs/system), and then use that to decide what to use.
>
> I think this should perhaps now work on the trunk -- at least it works
> in my test cases.  Could you test?

I don't know how to properly test some patched lisp files without
rebuilding and reinstalling. If I manually load the patched files seq.el
and mailcap.el in my "normal" emacs it works as I would expect it to
work, thanks. If I do emacs -Q and manually load those files and

(require 'mailcap)
(mailcap-mime-info "text/html")

I get

  Debugger entered--Lisp error: (void-function when-let)
  ...

But maybe that test case is confused. What do you think?

Best regards
--
Tomas

bug#36771: 27.0.50; mailcap documentation and effect missmatch

[Lars Ingebrigtsen]

Tomas Nordin <tomasn@posteo.net> writes:

> I don't know how to properly test some patched lisp files without
> rebuilding and reinstalling.

You don't need to reinstall, really -- just:

git clone git://git.savannah.gnu.org/emacs.git
cd emacs
make
./src/emacs &

> If I manually load the patched files seq.el
> and mailcap.el in my "normal" emacs it works as I would expect it to
> work, thanks.

Great; thanks for testing.  Since this seems to work now, I'm closing
this bug report.

> If I do emacs -Q and manually load those files and
>
> (require 'mailcap)
> (mailcap-mime-info "text/html")
>
> I get
>
>   Debugger entered--Lisp error: (void-function when-let)
>   ...
>
> But maybe that test case is confused. What do you think?

You probably need to say (require 'subr-x) in that case.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#36771: 27.0.50; mailcap documentation and effect missmatch

[Tomas Nordin]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Tomas Nordin <tomasn@posteo.net> writes:
>
>> I don't know how to properly test some patched lisp files without
>> rebuilding and reinstalling.
>
> You don't need to reinstall, really -- just:
>
> git clone git://git.savannah.gnu.org/emacs.git
> cd emacs
> make
> ./src/emacs &

Thats what I meant, it felt like an overkill to rebuild for some updated
elisp.

>
>> If I manually load the patched files seq.el
>> and mailcap.el in my "normal" emacs it works as I would expect it to
>> work, thanks.
>
> Great; thanks for testing.  Since this seems to work now, I'm closing
> this bug report.

Ok, thanks for fixing.