bug#54162: 29.0.50; notmuch search for nnmaildir is broken

bug#54162: 29.0.50; notmuch search for nnmaildir is broken

[Jose A Ortega Ruiz]

hi,

nnmaildir servers can be configured without apparent problem in emacs 28
(pre-release branch) with notmuch as their search engine.  for instance,
if my maildirs live in ~/var/mail and i index them with notmuch, i can
use as gnus-secondary-select-methods something like:

     ((nnmaildir ""
                 (directory "~/var/mail/")
                 (gnus-search-engine gnus-search-notmuch
                                     (remove-prefix "/home/jao/var/mail/"))))

and searches work just fine.  however, in current master, all searches
simply report, very quickly, empty results, without even logging
anything (despite gnus-verbose set to 10).

i suspect this is a side-effect of the latest fixes for AW in search
results for nnml, but i don't have time to dig deeper, sorry.

cheers,
jao



In GNU Emacs 29.0.50 (build 2, x86_64-pc-linux-gnu, cairo version 1.16.0)
 of 2022-02-24 built on rivendell
Repository revision: 8d2fbd4a146bcd01fd905fb500230c7c662f2a01
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101003
System Description: Debian GNU/Linux bookworm/sid

Configured using:
 'configure --prefix=/usr/local/stow/emacs --with-x-toolkit=no
 --with-imagemagick'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ
IMAGEMAGICK JPEG JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2
M17N_FLT MODULES NOTIFY INOTIFY OLDXMENU PDUMPER PNG RSVG SECCOMP SOUND
THREADS TIFF X11 XDBE XIM XPM ZLIB

Important settings:
  value of $LANG: en_GB.UTF-8
  locale-coding-system: utf-8-unix

Major mode: notmuch-hello

Minor modes in effect:
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  circe-lagmon-mode: t
  telega-root-auto-fill-mode: t
  telega-active-locations-mode: t
  telega-patrons-mode: t
  telega-mode-line-mode: t
  global-diff-hl-mode: t
  eshell-vterm-mode: t
  eshell-syntax-highlighting-global-mode: t
  pdf-occur-global-minor-mode: t
  shell-dirtrack-mode: t
  winner-mode: t
  global-auto-revert-mode: t
  corfu-global-mode: t
  corfu-mode: t
  marginalia-mode: t
  mct-minibuffer-mode: t
  persistent-scratch-autosave-mode: t
  global-so-long-mode: t
  display-battery-mode: t
  jao-minibuffer-mode: t
  minibuffer-electric-default-mode: t
  minibuffer-depth-indicate-mode: t
  repeat-mode: t
  savehist-mode: t
  recentf-mode: t
  save-place-mode: t
  override-global-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t

Load-path shadows:
/home/jao/lib/elisp/org-static-blog/org-static-blog hides /home/jao/.emacs.d/elpa.29/org-static-blog-20220110.739/org-static-blog
/home/jao/etc/emacs/site/custom hides /usr/local/stow/emacs/share/emacs/29.0.50/lisp/custom
/home/jao/.emacs.d/elpa.29/transient-20220216.2303/transient hides /usr/local/stow/emacs/share/emacs/29.0.50/lisp/transient

Features:
(shadow mailalias bbdb-message gnus-cite mail-extr gnus-bcklg
gnus-search eieio-opt speedbar ezimage dframe gnus-async gnus-dup sort
gnus-ml qp pcmpl-unix gnus-eform nnfolder gnus-topic nnmaildir bbdb-gnus
nnnil bbdb-mua gnus-icalendar ol-gnus nnselect gnus-delay gnus-draft
gnus-agent gnus-srvr gnus-score score-mode nnvirtual gnus-cache gnus-msg
gnus-art mm-uu mml2015 gnus-sum nndraft nnmh gnus-demon nntp gnus-group
gnus-undo gnus-start gnus-dbus gnus-cloud nnimap nnmail mail-source utf7
netrc nnoo gnus-spec gnus-int gnus-range gnus-win cl-print debug
backtrace app-launcher goto-chg magit-extras pulse tramp-cache vc-mtn
vc-hg vc-bzr vc-src vc-sccs vc-svn vc-cvs vc-rcs bug-reference org-id
tabify man misearch multi-isearch link-hint avy sh-script hlint-refactor
bash-completion em-unix em-script em-prompt em-ls em-hist em-pred
em-glob em-extpipe em-cmpl em-basic em-banner consult-icomplete
icomplete ediff ediff-merg ediff-mult ediff-wind ediff-diff ediff-help
ediff-init ediff-util magit-bookmark magit-submodule magit-obsolete
magit-blame magit-stash magit-reflog magit-bisect magit-push magit-pull
magit-fetch magit-clone magit-remote magit-commit magit-sequence
magit-notes magit-worktree magit-tag magit-merge magit-branch
magit-reset magit-files magit-refs magit-status magit magit-repos
magit-apply magit-wip magit-log magit-diff smerge-mode diff git-commit
log-edit add-log magit-core magit-autorevert magit-margin
magit-transient magit-process with-editor magit-mode magit-git
magit-base which-func ido magit-section autoload mm-archive executable
copyright paredit display-fill-column-indicator smartscan shr-color
textsec uni-scripts idna-mapping ucs-normalize uni-confusable
textsec-check url-http url-gw url-cache url-auth org-duration org-agenda
org-appear cdlatex texmathp image-file image-converter cal-iso cal-move
mule-util emojify apropos tar-mode arc-mode archive-mode ht
network-stream circe-display-images circe-lagmon lui-track circe
lui-irc-colors irc lcs lui-logging lui-format circe-compat slack
slack-company slack-unread slack-websocket slack-thread-event
slack-room-event slack-star-event slack-reaction-event slack-reply-event
slack-typing slack-slash-commands slack-message-event slack-event
slack-dialog-edit-element-buffer slack-dialog-buffer slack-dialog
slack-stars-buffer slack-search-result-buffer
slack-thread-message-compose-buffer slack-file-list-buffer
slack-file-info-buffer slack-all-threads-buffer slack-message-buffer
slack-user-profile-buffer slack-pinned-items-buffer slack-pinned-item
slack-thread-message-buffer slack-room-info-buffer slack-room-buffer
slack-message-share-buffer slack-message-edit-buffer
slack-room-message-compose-buffer slack-message-compose-buffer
slack-message-attachment-preview-buffer slack-action slack-star
slack-reminder slack-search slack-message-reaction slack-message-editor
slack-message-sender slack-message-notification slack-buffer
slack-message-formatter slack-thread slack-im slack-channel slack-group
slack-conversations slack-create-message slack-attachment
slack-selectable slack-bot-message slack-user-message slack-file
slack-message slack-message-faces slack-unescape slack-block
slack-mrkdwn slack-usergroup slack-reaction slack-modeline slack-room
slack-counts slack-user slack-bot slack-dnd-status slack-emoji
slack-image slack-request slack-log request lui flyspell ispell
slack-team slack-team-ws slack-util websocket bigml bml-logs bml
bml-misc bml-whizzml bml-clojure bml-clj-tests bml-python bml-skels
bml-utils whizzml-skeletons skeleton whizzml-mode lice sieve sieve-mode
sieve-manage sasl sasl-anonymous sasl-login sasl-plain jao-mpc
jao-random-album jao-lyrics jao-mpris telega-obsolete telega
telega-tdlib-events telega-webpage visual-fill-column telega-root
telega-info telega-chat telega-modes telega-company telega-user
telega-notifications telega-voip telega-msg telega-tme telega-sticker
telega-i18n telega-vvnote bindat telega-ffplay telega-media telega-sort
telega-filter telega-ins telega-folders telega-inline telega-tdlib
telega-util rainbow-identifiers dired-aux telega-server telega-core
cursor-sensor telega-customize emacsbug sendmail jao-mullvad bluetooth
enwc enwc-backend json-mode json-snatcher js cc-mode cc-fonts cc-guess
cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs
virtualenvwrapper gud ediprolog pie haskell-doc inf-haskell
haskell-decl-scan haskell haskell-completions haskell-load
haskell-commands highlight-uses-mode haskell-modules haskell-sandbox
haskell-navigate-imports haskell-repl haskell-svg haskell-collapse
hideshow haskell-debug haskell-interactive-mode
haskell-presentation-mode haskell-compile haskell-hoogle haskell-process
haskell-session haskell-mode haskell-cabal haskell-utils
haskell-font-lock haskell-indentation haskell-string
haskell-sort-imports haskell-lexeme haskell-align-imports
haskell-complete-module haskell-ghc-support flymake-proc flymake
warnings dabbrev haskell-customize geiser-guile info-look geiser-repl
geiser-compile geiser-debug geiser-image geiser-company geiser-doc
geiser-menu geiser-edit etags fileloop xref project geiser-completion
geiser-autodoc geiser-eval geiser-connection geiser-syntax scheme
geiser-impl help-fns radix-tree geiser-log geiser-popup view
geiser-custom geiser-base geiser idris-mode idris-commands
idris-hole-list idris-ipkg-mode idris-tree-info idris-warnings-tree
idris-info idris-repl idris-highlight-input idris-prover inferior-idris
idris-warnings idris-log idris-events idris-simple-indent idris-syntax
idris-common-utils idris-settings idris-keys idris-core idris-compat
prop-menu package-lint finder lisp-mnt edit-list git-modes
gitignore-mode gitconfig-mode conf-mode gitattributes-mode git-link
git-timemachine diff-hl log-view pcvs-util vc-dir ewoc vc
jao-eshell-here eshell-autojump em-dirs esh-var eshell-up git-ps1-mode
eshell-vterm em-term eshell-syntax-highlighting em-alias vterm tramp
tramp-loaddefs trampver tramp-integration files-x tramp-compat
parse-time ls-lisp face-remap term disp-table ehelp vterm-module
term/xterm xterm saveplace-pdf-view pdf-occur ibuf-ext ibuffer
ibuffer-loaddefs tablist tablist-filter semantic/wisent/comp
semantic/wisent semantic/wisent/wisent semantic/util-modes semantic/util
semantic semantic/tag semantic/lex semantic/fw mode-local cedet
pdf-isearch pdf-misc consult-notmuch org-capture org-refile jao-notmuch
notmuch notmuch-tree notmuch-jump notmuch-hello notmuch-show
notmuch-print notmuch-crypto notmuch-mua notmuch-message notmuch-draft
notmuch-maildir-fcc notmuch-address notmuch-company notmuch-parser
notmuch-wash coolj notmuch-query goto-addr icalendar notmuch-tag
notmuch-lib notmuch-version notmuch-compat pcase hl-line mm-view
mml-smime smime dig bbdb-anniv bbdb-com crm bbdb bbdb-site timezone
randomsig message yank-media rfc822 mml mml-sec mm-decode mm-bodies
mm-encode mail-parse rfc2231 rfc2047 rfc2045 ietf-drums mailabbrev
gmm-utils mailheader socks gnutls nsm rmc elpher ol-eww jao-eww-session
eww xdg url-queue shr pixel-fill kinsoku url-file url-dired svg dom puny
mm-url gnus nnheader gnus-util mail-utils range mm-util mail-prsvr
markdown-toc dash s markdown-mode htmlize jao-org-links jao-maildir
jao-doc-view let-alist doc-view pdf-tools pdf-view pdf-cache pdf-info tq
pdf-util pdf-macs image-mode exif ol-info ol-bbdb ol-eshell esh-mode
eshell esh-cmd esh-ext esh-opt esh-proc esh-io esh-arg esh-module
esh-groups esh-util jao-org-notes ob-shell shell ob-scheme ob-python
python ob-org ob-ocaml ob-makefile ob-haskell ob-gnuplot ob-clojure
ob-calc calc-store calc-trail calc-ext calc calc-loaddefs rect calc-macs
ob-prolog prolog smie align poly-org polymode poly-lock polymode-base
polymode-weave polymode-export polymode-compat polymode-methods
polymode-core polymode-classes eieio-custom eieio-base color org-tempo
tempo ox-texinfo ox-latex ox-html table ox-ascii ox-publish ox
org-element avl-tree generator org-fragtog jao-afio winner autorevert
filenotify autoinsert embark-consult embark ffap thingatpt
consult-recoll jao-compilation consult bookmark corfu marginalia mct
orderless imenu find-dired dired-x dired dired-loaddefs wgrep grep
compile text-property-search jka-compr persistent-scratch so-long
cal-china lunar solar cal-dst cal-bahai cal-islam cal-hebrew holidays
hol-loaddefs vc-git diff-mode vc-dispatcher appt diary-lib
diary-loaddefs jao-tracking tracking shorten jao-notify alert log4e
notifications gntp battery jao-minibuffer minibuf-eldef mb-depth
jao-recoll diminish jao-light-theme jao-themes pinentry epa-file epa
derived epg rfc6068 epg-config transient cus-edit pp cus-load repeat
edmacro kmacro jao-sleep dbus xml server savehist recentf tree-widget
wid-edit saveplace gnu-elpa-keyring-update use-package
use-package-ensure use-package-delight use-package-diminish
use-package-bind-key bind-key use-package-core cl-extra help-mode org ob
ob-tangle ob-ref ob-lob ob-table ob-exp org-macro org-footnote org-src
ob-comint org-pcomplete pcomplete comint ansi-color ring org-list
org-faces org-entities noutline outline easy-mmode org-version
ob-emacs-lisp ob-core ob-eval org-table oc-basic bibtex iso8601
time-date ol org-keys oc org-compat advice org-macs org-loaddefs
format-spec find-func cal-menu calendar cal-loaddefs finder-inf tex-site
rx info package browse-url url url-proxy url-privacy url-expand
url-methods url-history url-cookie url-domsuf url-util mailcap
url-handlers url-parse auth-source cl-seq eieio eieio-core cl-macs
eieio-loaddefs password-cache json map url-vars seq gv subr-x byte-opt
bytecomp byte-compile cconv cl-loaddefs cl-lib iso-transl tooltip eldoc
paren electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode
mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite emoji-zwj charscript charprop case-table
epa-hook jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice
button loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting cairo x
multi-tty make-network-process emacs)

Memory information:
((conses 16 3621256 1535351)
 (symbols 48 107153 259)
 (strings 32 632361 255758)
 (string-bytes 1 26529858)
 (vectors 16 229678)
 (vector-slots 8 7012555 1473150)
 (floats 8 9493 3903)
 (intervals 56 292760 93400)
 (buffers 992 117))

-- 
If the doors of perception were cleansed everything would appear to man as
it is, infinite.
 -William Blake, poet, engraver, and painter (1757-1827)

bug#54162: 29.0.50; notmuch search for nnmaildir is broken

[Eric Abrahamsen]

Jose A Ortega Ruiz <jao@gnu.org> writes:

> hi,
>
> nnmaildir servers can be configured without apparent problem in emacs 28
> (pre-release branch) with notmuch as their search engine.  for instance,
> if my maildirs live in ~/var/mail and i index them with notmuch, i can
> use as gnus-secondary-select-methods something like:
>
>      ((nnmaildir ""
>                  (directory "~/var/mail/")
>                  (gnus-search-engine gnus-search-notmuch
>                                      (remove-prefix "/home/jao/var/mail/"))))
>
> and searches work just fine.  however, in current master, all searches
> simply report, very quickly, empty results, without even logging
> anything (despite gnus-verbose set to 10).
>
> i suspect this is a side-effect of the latest fixes for AW in search
> results for nnml, but i don't have time to dig deeper, sorry.
>
> cheers,
> jao

Noooooooooooooo......

Okay, that out of the way, I wonder if the fact that your server doesn't
have a "name" is causing the error. I'll try setting up something like
that locally and reporting back.

Thanks,
Eric

bug#54162: 29.0.50; notmuch search for nnmaildir is broken

[jao]

On Fri, Feb 25 2022, Eric Abrahamsen wrote:

[...]

> Noooooooooooooo......

my thoughts exactly! :)

> Okay, that out of the way, I wonder if the fact that your server doesn't
> have a "name" is causing the error. I'll try setting up something like
> that locally and reporting back.

yes, i just tried and, if i give a name to the server, things seem to
work.  phew! it'd be nice to have it working with an empty name, but i
can certainly live with that.

many thanks for the quick response,
jao
-- 
I never lose sight of the fact that just being is fun.
  -Katharine Hepburn, actress (1907-2003)

bug#54162: 29.0.50; notmuch search for nnmaildir is broken

[Eric Abrahamsen]

jao <jao@gnu.org> writes:

> On Fri, Feb 25 2022, Eric Abrahamsen wrote:
>
> [...]
>
>> Noooooooooooooo......
>
> my thoughts exactly! :)
>
>> Okay, that out of the way, I wonder if the fact that your server doesn't
>> have a "name" is causing the error. I'll try setting up something like
>> that locally and reporting back.
>
> yes, i just tried and, if i give a name to the server, things seem to
> work.  phew! it'd be nice to have it working with an empty name, but i
> can certainly live with that.

Oh good, I'm glad it was an easy fix.

While I think empty name strings should be gently discouraged, it should
definitely still work. I'll figure out what's happened and close this
bug when I've got it.

Thanks for the report,
Eric

bug#54162: 29.0.50; notmuch search for nnmaildir is broken

[Eric Abrahamsen]

On 02/25/22 12:53 PM, Eric Abrahamsen wrote:
> jao <jao@gnu.org> writes:
>
>> On Fri, Feb 25 2022, Eric Abrahamsen wrote:
>>
>> [...]
>>
>>> Noooooooooooooo......
>>
>> my thoughts exactly! :)
>>
>>> Okay, that out of the way, I wonder if the fact that your server doesn't
>>> have a "name" is causing the error. I'll try setting up something like
>>> that locally and reporting back.
>>
>> yes, i just tried and, if i give a name to the server, things seem to
>> work.  phew! it'd be nice to have it working with an empty name, but i
>> can certainly live with that.
>
> Oh good, I'm glad it was an easy fix.
>
> While I think empty name strings should be gently discouraged, it should
> definitely still work. I'll figure out what's happened and close this
> bug when I've got it.

Bad news, I set up a nnmaildir server with a blank string for a name,
and it worked just fine. Right after you search a group and it fails,
can you show me the beginning value of this server in
`nnmaildir--servers': not the whole thing with all the message headers,
but the part that looks like:

(#1="" . #s(nnmaildir--srv :address #1# :method
			    (nnmaildir #1#
				       (directory "/tmp/emacs-gnus-mock-6ri742/test/")
				       (get-new-mail nil))
			    :prefix #1# :dir "/tmp/emacs-gnus-mock-6ri742/test/" :ls directory-files :groups #<hash-table equal 3/3 0x1588ffa44bcf> :curgrp #59=#s(nnmaildir--grp :name #7="mails" :new