bug#70054: 30.0.50; Unknown ids in References: break threading

bug#70054: 30.0.50; Unknown ids in References: break threading

[Jose A. Ortega Ruiz via Bug reports for GNU Emacs, the Swiss army knife of text editors]

This is possibly not a bug per se, but certainly a nuisance. When using
the ProtonMail bridge service to read email in Gnus, some internal headers
are added to the messages, all of them prefixed by X-pm- so that they
pose no problems. But among them is one internal message id (X-pm-id)
that then gets added to the References: header contents as needed. Thus,
instead of seeing the original header, e.g.

--8<---------------cut here---------------start------------->8---
References: <1399485980.12016.1.camel@Iris>
 <CAJsg1E_Spqmm+wiwuKj6oeCFtZsyoTX9hRY1wAJ4T=TR2yKSmQ@mail.gmail.com>
 <E1WiXKp-0003iD-2k@fencepost.gnu.org> <nn8uqblav2.fsf@bacon.lysator.liu.se>
 <536C9013.1010806@gnu.org>
 <CACujMDMbDfPP_7Es2Q0umgrgo9GH3_RhTYsfs3JKdmc2hORbRA@mail.gmail.com>
 <E1WjRba-0005Aw-6q@fencepost.gnu.org> <1399810320.5752.0.camel@Iris>
 <20140511122347.GA4055@intra> <1399814588.5752.4.camel@Iris>
 <20140511183915.GA13216@fencepost.gnu.org>
--8<---------------cut here---------------end--------------->8---

one gets

--8<---------------cut here---------------start------------->8---
References: <1399485980.12016.1.camel@Iris>
 <CAJsg1E_Spqmm+wiwuKj6oeCFtZsyoTX9hRY1wAJ4T=TR2yKSmQ@mail.gmail.com>
 <E1WiXKp-0003iD-2k@fencepost.gnu.org> <nn8uqblav2.fsf@bacon.lysator.liu.se>
 <536C9013.1010806@gnu.org>
 <CACujMDMbDfPP_7Es2Q0umgrgo9GH3_RhTYsfs3JKdmc2hORbRA@mail.gmail.com>
 <E1WjRba-0005Aw-6q@fencepost.gnu.org> <1399810320.5752.0.camel@Iris>
 <20140511122347.GA4055@intra> <1399814588.5752.4.camel@Iris>
 <20140511183915.GA13216@fencepost.gnu.org>
 <c3j0GRUnm7ZRQKF-bDqrBcUaFhUN184gOb3xe4f2rcPF-kJwXJB2clP_k0wkt8BGK_85N2ro4YkuIBmgZC7kOg==@protonmail.internalid> <t_dhLBndvM2vyK9s5Qx7Njsmmk0oTeVhk-78ViYQwbwRCsok-6SuIxGrG9qKdBdg-ekWFdqwPKOenA86C1Vn9g==@protonmail.internalid>
--8<---------------cut here---------------end--------------->8---

Note how two extraneous ids have been added to the list. I've checked
that if I remove them by editing the message threading works again. With
them, all I see is a flat tree where all mails are direct children of
the first one. It's as if having an id in the References that Gnus
cannot find broke proper threading. That's my theory anyway :)

Many thanks!
jao



In GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, cairo version
 1.18.0) of 2024-03-23 built on osgiliath
Repository revision: 7e32e8392ab77f9df08a1f11831cbba2242d721f
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101011
System Description: Debian GNU/Linux trixie/sid

Configured using:
 'configure --prefix /usr/local/stow/emacs30jit --with-x-toolkit=no
 --with-native-compilation --with-imagemagick'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ
IMAGEMAGICK JPEG JSON LCMS2 LIBOTF LIBSELINUX LIBXML2 M17N_FLT MODULES
NATIVE_COMP NOTIFY INOTIFY OLDXMENU PDUMPER PNG RSVG SECCOMP SOUND
SQLITE3 THREADS TIFF TREE_SITTER WEBP X11 XDBE XIM XINPUT2 XPM ZLIB

Important settings:
  value of $LANG: en_GB.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Summary

Minor modes in effect:
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  eat-eshell-visual-command-mode: t
  eat-eshell-mode: t
  circe-lagmon-mode: t
  telega-root-auto-fill-mode: t
  telega-active-video-chats-mode: t
  telega-active-locations-mode: t
  telega-patrons-mode: t
  telega-mode-line-mode: t
  telega-active-stories-mode: t
  global-diff-hl-mode: t
  eshell-syntax-highlighting-global-mode: t
  winner-mode: t
  global-auto-revert-mode: t
  vertico-multiform-mode: t
  vertico-mode: t
  marginalia-mode: t
  persistent-scratch-autosave-mode: t
  jao-minibuffer-mode: t
  minibuffer-electric-default-mode: t
  minibuffer-depth-indicate-mode: t
  repeat-mode: t
  server-mode: t
  savehist-mode: t
  recentf-mode: t
  save-place-mode: t
  override-global-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  minibuffer-regexp-mode: t
  buffer-read-only: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
/home/jao/lib/elisp/notmuch/notmuch-compat hides /usr/local/share/emacs/site-lisp/notmuch-compat
/home/jao/lib/elisp/notmuch/notmuch hides /usr/local/share/emacs/site-lisp/notmuch
/home/jao/lib/elisp/notmuch/notmuch-tree hides /usr/local/share/emacs/site-lisp/notmuch-tree
/home/jao/lib/elisp/notmuch/notmuch-lib hides /usr/local/share/emacs/site-lisp/notmuch-lib
/home/jao/lib/elisp/notmuch/notmuch-message hides /usr/local/share/emacs/site-lisp/notmuch-message
/home/jao/lib/elisp/notmuch/notmuch-address hides /usr/local/share/emacs/site-lisp/notmuch-address
/home/jao/lib/elisp/notmuch/notmuch-show hides /usr/local/share/emacs/site-lisp/notmuch-show
/home/jao/lib/elisp/notmuch/notmuch-wash hides /usr/local/share/emacs/site-lisp/notmuch-wash
/home/jao/lib/elisp/notmuch/notmuch-print hides /usr/local/share/emacs/site-lisp/notmuch-print
/home/jao/lib/elisp/notmuch/notmuch-company hides /usr/local/share/emacs/site-lisp/notmuch-company
/home/jao/lib/elisp/notmuch/notmuch-hello hides /usr/local/share/emacs/site-lisp/notmuch-hello
/home/jao/lib/elisp/notmuch/notmuch-parser hides /usr/local/share/emacs/site-lisp/notmuch-parser
/home/jao/lib/elisp/notmuch/coolj hides /usr/local/share/emacs/site-lisp/coolj
/home/jao/lib/elisp/notmuch/notmuch-jump hides /usr/local/share/emacs/site-lisp/notmuch-jump
/home/jao/lib/elisp/notmuch/notmuch-maildir-fcc hides /usr/local/share/emacs/site-lisp/notmuch-maildir-fcc
/home/jao/lib/elisp/notmuch/notmuch-tag hides /usr/local/share/emacs/site-lisp/notmuch-tag
/home/jao/lib/elisp/notmuch/notmuch-version hides /usr/local/share/emacs/site-lisp/notmuch-version
/home/jao/lib/elisp/notmuch/notmuch-mua hides /usr/local/share/emacs/site-lisp/notmuch-mua
/home/jao/lib/elisp/notmuch/notmuch-crypto hides /usr/local/share/emacs/site-lisp/notmuch-crypto
/home/jao/lib/elisp/notmuch/notmuch-draft hides /usr/local/share/emacs/site-lisp/notmuch-draft
/home/jao/lib/elisp/notmuch/notmuch-query hides /usr/local/share/emacs/site-lisp/notmuch-query
/home/jao/.emacs.d/elpa.30/transient-20240321.2209/transient hides /usr/local/stow/emacs30jit/share/emacs/30.0.50/lisp/transient
/home/jao/etc/emacs/site/custom hides /usr/local/stow/emacs30jit/share/emacs/30.0.50/lisp/custom

Features:
(shadow gnus-cus gnus-diary nndiary gnus-fun gnus-html gnus-kill
gnus-logic gnus-mh mh-comp mh-scan mh-gnus mh-e mh-buffers mh-loaddefs
gnus-registry registry rmailsum rmail gnus-salt gnus-uu yenc gnus-vm
pulse gnus-cite gnus-async gnus-bcklg gnus-dup gnus-ml gnus-topic nnml
nnnil bbdb-mua gnus-icalendar ol-gnus nnselect gnus-demon gnus-delay
gnus-draft gnus-agent gnus-srvr gnus-score score-mode nnvirtual nntp
gnus-cache gnus-msg gnus-art mm-uu mml2015 nndraft nnmh gnus-sum
gnus-group gnus-undo gnus-start gnus-dbus gnus-cloud nnimap nnmail
mail-source utf7 nnoo gnus-spec gnus-int gnus-range gnus-win cl-print
debug backtrace consult-imenu rng-xsd xsd-regexp rng-cmpct rng-nxml
rng-valid rng-loc rng-uri rng-parse nxml-parse rng-match rng-dt rng-util
rng-pttrn nxml-ns nxml-mode nxml-outln nxml-rap nxml-util nxml-enc
xmltok mhtml-mode css-mode js c-ts-common cc-mode cc-fonts cc-guess
cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs sgml-mode
org-static-blog org-cliplink org-cliplink-transport org-cliplink-string
dictionary external-completion dictionary-connection
emacsql-sqlite-builtin qp sort mailalias bbdb-message mail-extr
haskell-doc inf-haskell haskell-decl-scan hlint-refactor flymake-hlint
flymake-easy mastodon-notifications app-launcher pcmpl-unix quail
password-store auth-source-pass embark-vc forge-list forge-commands
forge-semi forge-bitbucket forge-gogs gogs forge-gitea gtea forge-gitlab
glab forge-github forge-notify forge-revnote forge-pullreq forge-issue
forge-topic yaml forge-post markdown-toc markdown-mode forge-repo forge
forge-core forge-db closql emacsql-sqlite-common emacsql
emacsql-compiler eieio-base buck ghub-graphql treepy gsexp ghub
loaddefs-gen mm-archive package-vc display-line-numbers vc-hg vc-bzr
vc-src vc-sccs vc-svn vc-cvs vc-rcs bug-reference sh-script misearch
multi-isearch paredit whitespace display-fill-column-indicator
symbol-overlay shortdoc executable copyright tabify org-agenda
image-file image-converter oc-basic bibtex cal-iso cal-move shr-color
magit-extras magit-bookmark magit-submodule magit-blame magit-stash
magit-reflog magit-bisect magit-push magit-pull magit-fetch magit-clone
magit-remote magit-commit magit-sequence magit-notes magit-worktree
magit-tag magit-merge magit-branch magit-reset magit-files magit-refs
magit-status magit magit-repos magit-apply magit-wip magit-log
which-func magit-diff smerge-mode diff git-commit log-edit add-log
magit-core magit-autorevert magit-margin magit-transient magit-process
with-editor magit-mode magit-git magit-base magit-section
bash-completion em-unix em-script em-prompt em-pred em-ls em-hist
em-glob em-extpipe em-cmpl em-basic em-banner eat term/xterm xterm
tramp-cache time-stamp face-remap vertico-directory tramp-cmds textsec
uni-scripts idna-mapping ucs-normalize uni-confusable textsec-check
mastodon-media mastodon-profile circe-lagmon lui-track
circe-display-images circe lui-irc-colors irc lcs lui-logging lui-format
circe-compat mastodon-tl network-stream url-http url-gw nsm url-cache
url-auth mastodon-auth mastodon-client plstore slack slack-company
slack-unread slack-websocket slack-thread-event slack-room-event
slack-star-event slack-reaction-event slack-reply-event slack-typing
slack-slash-commands slack-message-event slack-event
slack-dialog-edit-element-buffer slack-dialog-buffer slack-dialog
slack-stars-buffer slack-search-result-buffer
slack-thread-message-compose-buffer slack-file-list-buffer
slack-file-info-buffer slack-all-threads-buffer slack-message-buffer
slack-user-profile-buffer slack-pinned-items-buffer slack-pinned-item
slack-thread-message-buffer slack-room-info-buffer slack-room-buffer
slack-message-share-buffer slack-message-edit-buffer
slack-room-message-compose-buffer slack-message-compose-buffer
slack-message-attachment-preview-buffer slack-action slack-star
slack-reminder slack-search slack-message-reaction slack-message-editor
slack-message-sender slack-message-notification slack-buffer
slack-message-formatter slack-thread slack-im slack-channel slack-group
slack-conversations slack-create-message slack-attachment
slack-selectable slack-bot-message slack-user-message slack-file
slack-message slack-message-faces slack-unescape slack-block
slack-mrkdwn slack-usergroup slack-reaction slack-modeline slack-room
slack-counts slack-user slack-bot slack-dnd-status slack-emoji
slack-image slack-request slack-log lui flyspell ispell slack-team
slack-team-ws slack-util websocket mule-util time jao-custom-exwm
exwm-systemtray xcb-systemtray xcb-xembed exwm-edit exwm exwm-input
xcb-keysyms xcb-xkb exwm-manage exwm-floating xcb-cursor xcb-render
exwm-layout exwm-workspace exwm-core xcb-ewmh xcb-icccm xcb xcb-xproto
xcb-types xcb-debug bigml bml-logs bml bml-misc bml-whizzml bml-clojure
bml-clj-tests bml-python bml-skels bml-utils multisession sqlite
whizzml-skeletons whizzml-mode sieve sieve-mode sieve-manage sasl
sasl-anonymous sasl-login sasl-plain xmobar xclip jao-wayland jao-spt
consult-spotify espotify jao-mpc jao-random-album jao-lyrics jao-mpris
mastodon mastodon-search mastodon-toot facemenu mastodon-iso emojify
apropos tar-mode arc-mode archive-mode mastodon-http telega-obsolete
telega telega-tdlib-events telega-webpage telega-match telega-root
telega-info telega-chat visual-fill-column telega-modes telega-company
telega-emoji telega-user telega-notifications telega-voip telega-msg
telega-story telega-tme telega-sticker telega-vvnote bindat
telega-ffplay telega-i18n telega-sort telega-filter telega-ins
telega-inline telega-util telega-folders telega-topic telega-media
telega-tdlib rainbow-identifiers dired-aux color telega-server
telega-core cursor-sensor telega-customize emacsbug jao-mullvad
bluetooth json-mode smie virtualenvwrapper gud s pie haskell
haskell-completions haskell-load haskell-commands highlight-uses-mode
haskell-modules haskell-sandbox haskell-navigate-imports haskell-repl
haskell-svg haskell-collapse hideshow haskell-debug
haskell-interactive-mode haskell-presentation-mode haskell-compile
haskell-process haskell-session haskell-hoogle consult-hoogle
hoogle-base align geiser-chez geiser-guile tramp trampver
tramp-integration tramp-message tramp-compat parse-time iso8601
tramp-loaddefs info-look geiser-debug geiser-repl geiser-image
geiser-capf geiser-doc geiser-menu geiser-autodoc geiser-edit
geiser-completion geiser-eval geiser-connection tq geiser-syntax scheme
geiser-impl help-fns radix-tree geiser-log geiser-popup view
geiser-custom geiser-base geiser package-lint let-alist finder lisp-mnt
edit-list outline-minor-faces git-modes gitignore-mode gitconfig-mode
conf-mode gitattributes-mode git-link git-timemachine diff-hl log-view
pcvs-util vc-dir ewoc vc jao-eshell-here eshell-autojump em-dirs
eshell-up git-ps1-mode em-term term disp-table ehelp
eshell-syntax-highlighting em-alias jao-custom-email jao-custom-notmuch
consult-notmuch org-capture jao-notmuch link-hint avy notmuch
notmuch-tree notmuch-jump notmuch-hello notmuch-show notmuch-print
notmuch-crypto notmuch-mua notmuch-message notmuch-draft
notmuch-maildir-fcc notmuch-address notmuch-company notmuch-parser
notmuch-wash coolj goto-addr icalendar notmuch-tag notmuch-lib
notmuch-version notmuch-compat hl-line mm-view mml-smime smime gnutls
dig bbdb-anniv bbdb-com crm bbdb bbdb-site timezone randomsig message
sendmail yank-media rfc822 mml mml-sec mm-decode mm-bodies mm-encode
mail-parse rfc2231 rfc2047 rfc2045 ietf-drums mailabbrev gmm-utils
jao-proton-utils nov imenu esxml-query saveplace-pdf-view jao-custom-eww
ol-eww jao-eww-session eww url-queue shr pixel-fill kinsoku url-file svg
puny mm-url gnus nnheader gnus-util range mm-util mail-prsvr
edit-indirect reverso dom request mailheader mail-utils jao-custom-blog
htmlize jao-custom-org jao-org-links jao-doc-view jao-pdf doc-view
image-mode exif ol-info ol-eshell esh-mode esh-var eshell esh-cmd
esh-ext esh-opt esh-proc esh-io esh-arg esh-module esh-groups esh-util
files-x jao-org-notes ob-shell shell ob-scheme ob-python python treesit
ob-org ob-ocaml ob-makefile ob-haskell ob-gnuplot ob-clojure ob-calc
calc-store calc-trail calc-ext calc calc-loaddefs rect calc-macs
org-tempo tempo ox-pandoc ht dash ox-org ox-texinfo ox-latex ox-html
table ox-ascii ox-publish ox org-fragtog embark-org org-element
org-persist xdg org-id org-refile avl-tree org ob ob-tangle ob-ref
ob-lob ob-table ob-exp org-macro org-src ob-comint org-pcomplete
pcomplete org-list org-footnote org-faces org-entities time-date
ob-emacs-lisp ob-core ob-eval org-cycle org-table ol org-fold
org-fold-core org-keys oc org-loaddefs find-func org-version org-compat
org-macs advice winner autorevert filenotify consult-recoll
embark-consult consult bookmark jao-custom-completion embark ffap
vertico-multiform vertico marginalia orderless jao-skel-latex
jao-skel-haskell haskell-mode haskell-cabal haskell-utils
haskell-font-lock haskell-indentation haskell-string
haskell-sort-imports haskell-lexeme haskell-align-imports
haskell-complete-module haskell-ghc-support noutline outline
flymake-proc flymake thingatpt etags fileloop generator xref project
dabbrev haskell-customize jao-compilation jao-skel-lisp jao-skel-geiser
jao-skel skeleton autoinsert wgrep grep compile text-property-search
comint ansi-osc ring jka-compr dired-duplicates dired-x dired
dired-loaddefs persistent-scratch cal-china lunar solar cal-dst
cal-bahai cal-islam cal-hebrew holidays holiday-loaddefs vc-git
diff-mode vc-dispatcher appt diary-lib diary-loaddefs cal-menu calendar
cal-loaddefs jao-tracking jao-afio jao-doc-session tracking shorten
jao-notify alert log4e notifications gntp battery diminish jao-mode-line
comp comp-cstr warnings comp-run comp-common rx jao-minibuffer
minibuf-eldef mb-depth man jao-light-theme jao-themes ansi-color
epa-file epa derived epg rfc6068 epg-config transient format-spec compat
repeat jao-shell edmacro kmacro jao-sleep dbus xml server savehist
recentf tree-widget saveplace persist cus-edit pp cus-load icons
wid-edit jao-gnus-private gnu-elpa-keyring-update cl-extra help-mode
use-package use-package-ensure use-package-delight use-package-diminish
use-package-bind-key bind-key easy-mmode use-package-core finder-inf
auctex-autoloads tex-site bash-completion-autoloads bbdb-autoloads
bluetooth-autoloads cider-autoloads clojure-mode-autoloads
consult-git-log-grep-autoloads consult-hoogle-autoloads
diff-hl-autoloads diminish-autoloads dired-duplicates-autoloads
eat-autoloads edit-indirect-autoloads edit-list-autoloads
embark-consult-autoloads consult-autoloads embark-vc-autoloads
embark-autoloads eshell-autojump-autoloads
eshell-syntax-highlighting-autoloads eshell-up-autoloads exwm-autoloads
exwm-edit-autoloads flymake-hlint-autoloads flymake-easy-autoloads
forge-autoloads closql-autoloads emacsql-autoloads ghub-autoloads
git-link-autoloads git-modes-autoloads git-ps1-mode-autoloads
git-timemachine-autoloads gnu-elpa-keyring-update-autoloads
goto-chg-autoloads haskell-mode-autoloads hlint-refactor-autoloads
htmlize-autoloads jenkins-autoloads json-mode-autoloads
link-hint-autoloads avy-autoloads magit-autoloads pcase
git-commit-autoloads magit-section-autoloads marginalia-autoloads
markdown-toc-autoloads markdown-mode-autoloads mastodon-autoloads
nov-autoloads esxml-autoloads kv-autoloads oauth2-autoloads
orderless-autoloads org-cliplink-autoloads org-fragtog-autoloads
org-static-blog-autoloads outline-minor-faces-autoloads
ox-pandoc-autoloads package-lint-autoloads paredit-autoloads
parseedn-autoloads parseclj-autoloads password-store-autoloads
persist-autoloads persistent-scratch-autoloads queue-autoloads
reverso-autoloads saveplace-pdf-view-autoloads sesman-autoloads
slack-autoloads emojify-autoloads ht-autoloads alert-autoloads
log4e-autoloads gntp-autoloads circe-autoloads request-autoloads
spinner-autoloads symbol-overlay-autoloads telega-autoloads
rainbow-identifiers-autoloads transient-autoloads treepy-autoloads
vertico-autoloads virtualenvwrapper-autoloads s-autoloads dash-autoloads
visual-fill-column-autoloads websocket-autoloads wgrep-autoloads info
with-editor-autoloads xclip-autoloads xelb-autoloads yaml-autoloads
yaml-mode-autoloads package browse-url url url-proxy url-privacy
url-expand url-methods url-history url-cookie generate-lisp-file
url-domsuf url-util mailcap url-handlers url-parse auth-source cl-seq
eieio eieio-core cl-macs password-cache json subr-x map byte-opt gv
bytecomp byte-compile url-vars cl-loaddefs cl-lib rmc iso-transl tooltip
cconv eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type
elisp-mode mwheel term/x-win x-win term/common-win x-dnd touch-screen
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer nadvice seq
simple cl-generic indonesian philippine cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite emoji-zwj charscript charprop case-table
epa-hook jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button
loaddefs theme-loaddefs faces cus-face macroexp files window
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget keymap hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
cairo xinput2 x multi-tty move-toolbar make-network-process
native-compile emacs)

Memory information:
((conses 16 5090495 955039) (symbols 48 129083 476) (strings 32 889091 60253)
 (string-bytes 1 48867778) (vectors 16 286400) (vector-slots 8 5354724 529085)
 (floats 8 245615 39315) (intervals 56 231412 28620) (buffers 992 133))

-- 
Keep me away from the wisdom which does not cry, the philosophy which does
not laugh and the greatness which does not bow before children.
 -Khalil Gibran, mystic, poet and artist (1883-1931)

bug#70054: 30.0.50; Unknown ids in References: break threading

[Eli Zaretskii]

> Date: Thu, 28 Mar 2024 13:18:25 +0000
> From:  "Jose A. Ortega Ruiz" via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> 
> This is possibly not a bug per se, but certainly a nuisance. When using
> the ProtonMail bridge service to read email in Gnus, some internal headers
> are added to the messages, all of them prefixed by X-pm- so that they
> pose no problems. But among them is one internal message id (X-pm-id)
> that then gets added to the References: header contents as needed. Thus,
> instead of seeing the original header, e.g.
> 
> --8<---------------cut here---------------start------------->8---
> References: <1399485980.12016.1.camel@Iris>
>  <CAJsg1E_Spqmm+wiwuKj6oeCFtZsyoTX9hRY1wAJ4T=TR2yKSmQ@mail.gmail.com>
>  <E1WiXKp-0003iD-2k@fencepost.gnu.org> <nn8uqblav2.fsf@bacon.lysator.liu.se>
>  <536C9013.1010806@gnu.org>
>  <CACujMDMbDfPP_7Es2Q0umgrgo9GH3_RhTYsfs3JKdmc2hORbRA@mail.gmail.com>
>  <E1WjRba-0005Aw-6q@fencepost.gnu.org> <1399810320.5752.0.camel@Iris>
>  <20140511122347.GA4055@intra> <1399814588.5752.4.camel@Iris>
>  <20140511183915.GA13216@fencepost.gnu.org>
> --8<---------------cut here---------------end--------------->8---
> 
> one gets
> 
> --8<---------------cut here---------------start------------->8---
> References: <1399485980.12016.1.camel@Iris>
>  <CAJsg1E_Spqmm+wiwuKj6oeCFtZsyoTX9hRY1wAJ4T=TR2yKSmQ@mail.gmail.com>
>  <E1WiXKp-0003iD-2k@fencepost.gnu.org> <nn8uqblav2.fsf@bacon.lysator.liu.se>
>  <536C9013.1010806@gnu.org>
>  <CACujMDMbDfPP_7Es2Q0umgrgo9GH3_RhTYsfs3JKdmc2hORbRA@mail.gmail.com>
>  <E1WjRba-0005Aw-6q@fencepost.gnu.org> <1399810320.5752.0.camel@Iris>
>  <20140511122347.GA4055@intra> <1399814588.5752.4.camel@Iris>
>  <20140511183915.GA13216@fencepost.gnu.org>
>  <c3j0GRUnm7ZRQKF-bDqrBcUaFhUN184gOb3xe4f2rcPF-kJwXJB2clP_k0wkt8BGK_85N2ro4YkuIBmgZC7kOg==@protonmail.internalid> <t_dhLBndvM2vyK9s5Qx7Njsmmk0oTeVhk-78ViYQwbwRCsok-6SuIxGrG9qKdBdg-ekWFdqwPKOenA86C1Vn9g==@protonmail.internalid>
> --8<---------------cut here---------------end--------------->8---
> 
> Note how two extraneous ids have been added to the list. I've checked
> that if I remove them by editing the message threading works again. With
> them, all I see is a flat tree where all mails are direct children of
> the first one. It's as if having an id in the References that Gnus
> cannot find broke proper threading. That's my theory anyway :)

Eric, any comments?

bug#70054: 30.0.50; Unknown ids in References: break threading

[Eli Zaretskii]

Ping!  Eric, any comments?

> Cc: 70054@debbugs.gnu.org
> Date: Sat, 06 Apr 2024 12:18:22 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> 
> > Date: Thu, 28 Mar 2024 13:18:25 +0000
> > From:  "Jose A. Ortega Ruiz" via "Bug reports for GNU Emacs,
> >  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> > 
> > 
> > This is possibly not a bug per se, but certainly a nuisance. When using
> > the ProtonMail bridge service to read email in Gnus, some internal headers
> > are added to the messages, all of them prefixed by X-pm- so that they
> > pose no problems. But among them is one internal message id (X-pm-id)
> > that then gets added to the References: header contents as needed. Thus,
> > instead of seeing the original header, e.g.
> > 
> > --8<---------------cut here---------------start------------->8---
> > References: <1399485980.12016.1.camel@Iris>
> >  <CAJsg1E_Spqmm+wiwuKj6oeCFtZsyoTX9hRY1wAJ4T=TR2yKSmQ@mail.gmail.com>
> >  <E1WiXKp-0003iD-2k@fencepost.gnu.org> <nn8uqblav2.fsf@bacon.lysator.liu.se>
> >  <536C9013.1010806@gnu.org>
> >  <CACujMDMbDfPP_7Es2Q0umgrgo9GH3_RhTYsfs3JKdmc2hORbRA@mail.gmail.com>
> >  <E1WjRba-0005Aw-6q@fencepost.gnu.org> <1399810320.5752.0.camel@Iris>
> >  <20140511122347.GA4055@intra> <1399814588.5752.4.camel@Iris>
> >  <20140511183915.GA13216@fencepost.gnu.org>
> > --8<---------------cut here---------------end--------------->8---
> > 
> > one gets
> > 
> > --8<---------------cut here---------------start------------->8---
> > References: <1399485980.12016.1.camel@Iris>
> >  <CAJsg1E_Spqmm+wiwuKj6oeCFtZsyoTX9hRY1wAJ4T=TR2yKSmQ@mail.gmail.com>
> >  <E1WiXKp-0003iD-2k@fencepost.gnu.org> <nn8uqblav2.fsf@bacon.lysator.liu.se>
> >  <536C9013.1010806@gnu.org>
> >  <CACujMDMbDfPP_7Es2Q0umgrgo9GH3_RhTYsfs3JKdmc2hORbRA@mail.gmail.com>
> >  <E1WjRba-0005Aw-6q@fencepost.gnu.org> <1399810320.5752.0.camel@Iris>
> >  <20140511122347.GA4055@intra> <1399814588.5752.4.camel@Iris>
> >  <20140511183915.GA13216@fencepost.gnu.org>
> >  <c3j0GRUnm7ZRQKF-bDqrBcUaFhUN184gOb3xe4f2rcPF-kJwXJB2clP_k0wkt8BGK_85N2ro4YkuIBmgZC7kOg==@protonmail.internalid> <t_dhLBndvM2vyK9s5Qx7Njsmmk0oTeVhk-78ViYQwbwRCsok-6SuIxGrG9qKdBdg-ekWFdqwPKOenA86C1Vn9g==@protonmail.internalid>
> > --8<---------------cut here---------------end--------------->8---
> > 
> > Note how two extraneous ids have been added to the list. I've checked
> > that if I remove them by editing the message threading works again. With
> > them, all I see is a flat tree where all mails are direct children of
> > the first one. It's as if having an id in the References that Gnus
> > cannot find broke proper threading. That's my theory anyway :)
> 
> Eric, any comments?
> 
> 
> 
> 

bug#70054: 30.0.50; Unknown ids in References: break threading

[Eric Abrahamsen]

On 04/18/24 12:04 PM, Eli Zaretskii wrote:
> Ping!  Eric, any comments?
>
>> Cc: 70054@debbugs.gnu.org
>> Date: Sat, 06 Apr 2024 12:18:22 +0300
>> From: Eli Zaretskii <eliz@gnu.org>
>> 
>> > Date: Thu, 28 Mar 2024 13:18:25 +0000
>> > From:  "Jose A. Ortega Ruiz" via "Bug reports for GNU Emacs,
>> >  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
>> > 
>> > 
>> > This is possibly not a bug per se, but certainly a nuisance. When using
>> > the ProtonMail bridge service to read email in Gnus, some internal headers
>> > are added to the messages, all of them prefixed by X-pm- so that they
>> > pose no problems. But among them is one internal message id (X-pm-id)
>> > that then gets added to the References: header contents as needed. Thus,
>> > instead of seeing the original header, e.g.
>> > 
>> > --8<---------------cut here---------------start------------->8---
>> > References: <1399485980.12016.1.camel@Iris>
>> >  <CAJsg1E_Spqmm+wiwuKj6oeCFtZsyoTX9hRY1wAJ4T=TR2yKSmQ@mail.gmail.com>
>> >  <E1WiXKp-0003iD-2k@fencepost.gnu.org> <nn8uqblav2.fsf@bacon.lysator.liu.se>
>> >  <536C9013.1010806@gnu.org>
>> >  <CACujMDMbDfPP_7Es2Q0umgrgo9GH3_RhTYsfs3JKdmc2hORbRA@mail.gmail.com>
>> >  <E1WjRba-0005Aw-6q@fencepost.gnu.org> <1399810320.5752.0.camel@Iris>
>> >  <20140511122347.GA4055@intra> <1399814588.5752.4.camel@Iris>
>> >  <20140511183915.GA13216@fencepost.gnu.org>
>> > --8<---------------cut here---------------end--------------->8---
>> > 
>> > one gets
>> > 
>> > --8<---------------cut here---------------start------------->8---
>> > References: <1399485980.12016.1.camel@Iris>
>> >  <CAJsg1E_Spqmm+wiwuKj6oeCFtZsyoTX9hRY1wAJ4T=TR2yKSmQ@mail.gmail.com>
>> >  <E1WiXKp-0003iD-2k@fencepost.gnu.org> <nn8uqblav2.fsf@bacon.lysator.liu.se>
>> >  <536C9013.1010806@gnu.org>
>> >  <CACujMDMbDfPP_7Es2Q0umgrgo9GH3_RhTYsfs3JKdmc2hORbRA@mail.gmail.com>
>> >  <E1WjRba-0005Aw-6q@fencepost.gnu.org> <1399810320.5752.0.camel@Iris>
>> >  <20140511122347.GA4055@intra> <1399814588.5752.4.camel@Iris>
>> >  <20140511183915.GA13216@fencepost.gnu.org>
>> >  <c3j0GRUnm7ZRQKF-bDqrBcUaFhUN184gOb3xe4f2rcPF-kJwXJB2clP_k0wkt8BGK_85N2ro4YkuIBmgZC7kOg==@protonmail.internalid>
>> > <t_dhLBndvM2vyK9s5Qx7Njsmmk0oTeVhk-78ViYQwbwRCsok-6SuIxGrG9qKdBdg-ekWFdqwPKOenA86C1Vn9g==@protonmail.internalid>
>> > --8<---------------cut here---------------end--------------->8---
>> > 
>> > Note how two extraneous ids have been added to the list. I've checked
>> > that if I remove them by editing the message threading works again. With
>> > them, all I see is a flat tree where all mails are direct children of
>> > the first one. It's as if having an id in the References that Gnus
>> > cannot find broke proper threading. That's my theory anyway :)
>> 
>> Eric, any comments?

This certainly does sound like a bug. This is a pretty complex part of
Gnus, and it might take a while to track down. Jose, can you tell me
your value of `gnus-build-sparse-threads'? I would think that, if Gnus
*isn't* trying to build sparse threads, these unknown Message-IDs would
simply be treated like messages that haven't been loaded into the
Summary buffer, and ignored. Can you tell me your current value for that
variable, and also if/how changing the value (particularly between 'more
and 't) changes the behavior here?

Thanks!
Eric

bug#70054: 30.0.50; Unknown ids in References: break threading

[Jose A Ortega Ruiz]

On Sun, Apr 21 2024, Eric Abrahamsen wrote:

[...]

>>> Eric, any comments?
>
> This certainly does sound like a bug. This is a pretty complex part of
> Gnus, and it might take a while to track down. Jose, can you tell me
> your value of `gnus-build-sparse-threads'? I would think that, if Gnus
> *isn't* trying to build sparse threads, these unknown Message-IDs would
> simply be treated like messages that haven't been loaded into the
> Summary buffer, and ignored. Can you tell me your current value for that
> variable, and also if/how changing the value (particularly between 'more
> and 't) changes the behavior here?

Currently, I've got it set to nil. Setting it to t seems to almost
totally break the listing: there's some threading but it's displayed as
every single message being indented respect to the previous one most of
the time (i know from displaying those threads using notmuch that that's
not the case at all), and most messages have wrong dates and are marked
as cancelled or deleted when i try to access them the usual way
(e.g. with RET in the summary buffer). Maybe there's some conflict with
something cached, but it really seems to break hell loose :)

Thanks!
jao

bug#70054: 30.0.50; Unknown ids in References: break threading

[Eric Abrahamsen]

Jose A Ortega Ruiz <jao@gnu.org> writes:

> On Sun, Apr 21 2024, Eric Abrahamsen wrote:
>
> [...]
>
>>>> Eric, any comments?
>>
>> This certainly does sound like a bug. This is a pretty complex part of
>> Gnus, and it might take a while to track down. Jose, can you tell me
>> your value of `gnus-build-sparse-threads'? I would think that, if Gnus
>> *isn't* trying to build sparse threads, these unknown Message-IDs would
>> simply be treated like messages that haven't been loaded into the
>> Summary buffer, and ignored. Can you tell me your current value for that
>> variable, and also if/how changing the value (particularly between 'more
>> and 't) changes the behavior here?
>
> Currently, I've got it set to nil. Setting it to t seems to almost
> totally break the listing: there's some threading but it's displayed as
> every single message being indented respect to the previous one most of
> the time (i know from displaying those threads using notmuch that that's
> not the case at all), and most messages have wrong dates and are marked
> as cancelled or deleted when i try to access them the usual way
> (e.g. with RET in the summary buffer). Maybe there's some conflict with
> something cached, but it really seems to break hell loose :)

Gruesome! I experimented with nnmaildir in gnus-mock, adding spurious
message-ids to the References header, and nothing happened; they were
just ignored.

You mentioned the Proton bridge, I assume that's nnimap from Gnus' point
of view? I tried the same, editing a message in Dovecot's maildir store,
but it really didn't like that -- every time I tried to enter the group
the imap process died.

Can you tell me a little more about how you're accessing these messages,
inside Gnus?

Thanks,
Eric

bug#70054: 30.0.50; Unknown ids in References: break threading

[Jose A Ortega Ruiz]

Hi Eric,

Thanks for looking into this!

On Mon, Apr 22 2024, Eric Abrahamsen wrote:

[...]

> Can you tell me a little more about how you're accessing these messages,
> inside Gnus?

I am using nnmail.

I use fetchmail to fetch the messages from the imap server exposed by
ProtonMail's bridge, storing them in a local maildirs.

Then, I declare those maildirs in `mail-sources', and the messages are
read using Gnus's nnmail backend. That is, what Gnus ultimately uses is
nnmail with sources that happen to be a maildirs; imap is not involved.

Thus, elements in the mail-sources list have the form
(maildir "/home/jao/maildirs/foo").  Finally, although i don't think that
matters, a set of rules in `nnmail-split-fancy' tell Gnus to what nnmail
group each message belongs.

Cheers,
jao
-- 
We live on an island surrounded by a sea of ignorance. As our island
of knowledge grows, so does the shore of our ignorance. - John Wheeler

bug#70054: 30.0.50; Unknown ids in References: break threading

[Andreas Schwab]

On Apr 23 2024, Jose A Ortega Ruiz wrote:

> Then, I declare those maildirs in `mail-sources', and the messages are
> read using Gnus's nnmail backend. That is, what Gnus ultimately uses is
> nnmail with sources that happen to be a maildirs; imap is not involved.

nnmail is not a gnus backend on its own, more like an abstract base
class.  Did you mean nnmaildir?

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

bug#70054: 30.0.50; Unknown ids in References: break threading

[Jose A Ortega Ruiz]

On Tue, Apr 23 2024, Andreas Schwab wrote:

> On Apr 23 2024, Jose A Ortega Ruiz wrote:
>
>> Then, I declare those maildirs in `mail-sources', and the messages are
>> read using Gnus's nnmail backend. That is, what Gnus ultimately uses is
>> nnmail with sources that happen to be a maildirs; imap is not involved.
>
> nnmail is not a gnus backend on its own, more like an abstract base
> class.  Did you mean nnmaildir?

Sorry, no, I meant nnml. As I mention in another email, the problem also
appears with nnimap.

bug#70054: 30.0.50; Unknown ids in References: break threading

[Eli Zaretskii]

> Cc: Eric Abrahamsen <eric@ericabrahamsen.net>, 70054@debbugs.gnu.org
> From: Jose A Ortega Ruiz <jao@gnu.org>
> Date: Tue, 23 Apr 2024 14:52:56 +0100
> 
> On Tue, Apr 23 2024, Andreas Schwab wrote:
> 
> > On Apr 23 2024, Jose A Ortega Ruiz wrote:
> >
> >> Then, I declare those maildirs in `mail-sources', and the messages are
> >> read using Gnus's nnmail backend. That is, what Gnus ultimately uses is
> >> nnmail with sources that happen to be a maildirs; imap is not involved.
> >
> > nnmail is not a gnus backend on its own, more like an abstract base
> > class.  Did you mean nnmaildir?
> 
> Sorry, no, I meant nnml. As I mention in another email, the problem also
> appears with nnimap.

Ping!  Can we make some further progress with this bug report?

bug#70054: 30.0.50; Unknown ids in References: break threading

[Andrew Cohen]

>>>>> "EZ" == Eli Zaretskii <eliz@gnu.org> writes:

[...]

    EZ> Ping!  Can we make some further progress with this bug report?

I worked with Jose to track everything down. This isn't a gnus bug and
the bug report can be closed.

Slightly more detail: the protonmail bridge adds non-existent ids to the
references header of messages (presumably used in some internal way by
the protonmail bridge). 'gnus is resilient in handling these: if
collecting threads by subject (the default) everything "just works". And
in case you (optionally) want to only use the references header for
threading, gnus has a way to alter the headers for exactly this
occasion: 'gnus-alter-header-function.  So the following:

(setq gnus-alter-header-function 'fix-protonmail-references)

(defun fix-protonmail-references (header)
  (setf (mail-header-references header)
        (mapconcat
         #'(lambda (x) (if (string-search "protonmail.internalid" x) "" x))
         (gnus-split-references (mail-header-references header)) " "))
  header)

will work.

I think this deserves to be in the gnus manual.  I have a commit in my local
tree to do this, which I will push to master.

-- 
Andrew Cohen

bug#70054: 30.0.50; Unknown ids in References: break threading

[Eli Zaretskii]

> Cc: Eric Abrahamsen <eric@ericabrahamsen.net>, Jose A Ortega Ruiz <jao@gnu.org>
> From: Andrew Cohen <acohen@ust.hk>
> Date: Thu, 09 May 2024 15:55:11 +0800
> 
> >>>>> "EZ" == Eli Zaretskii <eliz@gnu.org> writes:
> 
> [...]
> 
>     EZ> Ping!  Can we make some further progress with this bug report?
> 
> I worked with Jose to track everything down. This isn't a gnus bug and
> the bug report can be closed.

Thanks, done with this message.

> Slightly more detail: the protonmail bridge adds non-existent ids to the
> references header of messages (presumably used in some internal way by
> the protonmail bridge). 'gnus is resilient in handling these: if
> collecting threads by subject (the default) everything "just works". And
> in case you (optionally) want to only use the references header for
> threading, gnus has a way to alter the headers for exactly this
> occasion: 'gnus-alter-header-function.  So the following:
> 
> (setq gnus-alter-header-function 'fix-protonmail-references)
> 
> (defun fix-protonmail-references (header)
>   (setf (mail-header-references header)
>         (mapconcat
>          #'(lambda (x) (if (string-search "protonmail.internalid" x) "" x))
>          (gnus-split-references (mail-header-references header)) " "))
>   header)
> 
> will work.
> 
> I think this deserves to be in the gnus manual.  I have a commit in my local
> tree to do this, which I will push to master.

Thanks again.