bug#45650: 28.0.50; nnmaildir discards line count

bug#45650: 28.0.50; nnmaildir discards line count

[Justus-dev]

With the nnmaildir backend, NOV data are created with a line count of
-1, causing the %L summary-line format specifier to show a question
mark.

The culprit is this section in nnmaildir--update-nov in nnmaildir.el:

	(save-restriction
	  (if (search-forward "\n\n" nil 'noerror)
	      (progn
		(setq nov-mid (count-lines (point) (point-max)))
		(narrow-to-region (point-min) (1- (point))))
	    (setq nov-mid 0))
	  (goto-char (point-min))
	  (delete-char 1)
	  (setq nov (nnheader-parse-head t)
		field (or (mail-header-lines nov) 0)))
	(unless (or (zerop field) (nnmaildir--param pgname 'distrust-Lines:))
	  (setq nov-mid field))

This code assumes that if no Lines: header is found, mail-header-lines
returns 0.  However, mail-header-lines returns -1 in this case, causing
the correctly-counted number of lines in nov-mid to be overwritten with
-1.

Replacing (zerop field) by (<= field 0) fixes it.

While I think this is indeed a correct immediate fix, this function
should be refactored:

- Why count the number of lines in the message only to discard the
  result if a Lines: header is found afterwards? (Well, fixing this
  won't have much impact: Of the 161594 messages in my nnmaildir
  folders, only 2982 contain a Lines: header.)

- [unrelated] Above the quoted code section, the function checks if the
  NOV data have to be (re)created by parsing the message; if not, it
  raises an exception to return early.  This construct should be
  replaced by a conditional, perhaps splitting out the second half into
  its own function for readability.

I don't think any of the below is significant, except perhaps for the
fact that this problem was not present in Emacs 27.


In GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.24, cairo version 1.17.4)
 of 2021-01-03 built on piater
Repository revision: dc5dab469242c40803891518efe9c9f6bd650cea
Repository branch: makepkg
Windowing system distributor 'System Description: Arch Linux

Configured using:
 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --mandir=/usr/share/man --with-gameuser=:games
 --with-sound=alsa --with-modules --without-gconf --without-gsettings
 --with-nativecomp --with-pgtk --with-x-toolkit=gtk3 --without-xaw3d
 --without-m17n-flt --with-cairo --with-xwidgets
 --without-compress-install 'CFLAGS=-march=x86-64 -mtune=generic -O2
 -pipe -fno-plt -g -fuse-ld=gold -g -fuse-ld=gold'
 CPPFLAGS=-D_FORTIFY_SOURCE=2
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now'

Configured features:
JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GLIB NOTIFY INOTIFY ACL
GNUTLS LIBXML2 FREETYPE HARFBUZZ LIBOTF ZLIB TOOLKIT_SCROLL_BARS GTK3
PGTK XIM MODULES NATIVE_COMP THREADS XWIDGETS LIBSYSTEMD JSON PDUMPER
LCMS2

Important settings:
  value of $LC_COLLATE: de_AT.UTF-8
  value of $LC_MONETARY: de_AT.UTF-8
  value of $LC_TIME: de_AT.UTF-8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Group

Minor modes in effect:
  semantic-minor-modes-format: ((:eval (if (or semantic-highlight-edits-mode semantic-show-unmatched-syntax-mode)  S)))
  gnus-agent-group-mode: t
  gnus-topic-mode: t
  shell-dirtrack-mode: t
  gnus-undo-mode: t
  show-paren-mode: t
  override-global-mode: t
  pdfgrep-mode: t
  pdf-occur-global-minor-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
/usr/share/emacs/site-lisp/haskell-mode/w3m-haddock hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/w3m-haddock
/usr/share/emacs/site-lisp/haskell-mode/inf-haskell hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/inf-haskell
/usr/share/emacs/site-lisp/haskell-mode/highlight-uses-mode hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/highlight-uses-mode
/usr/share/emacs/site-lisp/haskell-mode/haskell hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell
/usr/share/emacs/site-lisp/haskell-mode/haskell-utils hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-utils
/usr/share/emacs/site-lisp/haskell-mode/haskell-unicode-input-method hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-unicode-input-method
/usr/share/emacs/site-lisp/haskell-mode/haskell-string hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-string
/usr/share/emacs/site-lisp/haskell-mode/haskell-sort-imports hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-sort-imports
/usr/share/emacs/site-lisp/haskell-mode/haskell-session hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-session
/usr/share/emacs/site-lisp/haskell-mode/haskell-sandbox hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-sandbox
/usr/share/emacs/site-lisp/haskell-mode/haskell-repl hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-repl
/usr/share/emacs/site-lisp/haskell-mode/haskell-process hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-process
/usr/share/emacs/site-lisp/haskell-mode/haskell-presentation-mode hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-presentation-mode
/usr/share/emacs/site-lisp/haskell-mode/haskell-navigate-imports hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-navigate-imports
/usr/share/emacs/site-lisp/haskell-mode/haskell-move-nested hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-move-nested
/usr/share/emacs/site-lisp/haskell-mode/haskell-modules hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-modules
/usr/share/emacs/site-lisp/haskell-mode/haskell-menu hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-menu
/usr/share/emacs/site-lisp/haskell-mode/haskell-load hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-load
/usr/share/emacs/site-lisp/haskell-mode/haskell-lexeme hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-lexeme
/usr/share/emacs/site-lisp/haskell-mode/haskell-interactive-mode hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-interactive-mode
/usr/share/emacs/site-lisp/haskell-mode/haskell-indentation hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-indentation
/usr/share/emacs/site-lisp/haskell-mode/haskell-indent hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-indent
/usr/share/emacs/site-lisp/haskell-mode/haskell-hoogle hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-hoogle
/usr/share/emacs/site-lisp/haskell-mode/haskell-font-lock hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-font-lock
/usr/share/emacs/site-lisp/haskell-mode/haskell-doc hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-doc
/usr/share/emacs/site-lisp/haskell-mode/haskell-decl-scan hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-decl-scan
/usr/share/emacs/site-lisp/haskell-mode/haskell-debug hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-debug
/usr/share/emacs/site-lisp/haskell-mode/haskell-customize hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-customize
/usr/share/emacs/site-lisp/haskell-mode/haskell-completions hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-completions
/usr/share/emacs/site-lisp/haskell-mode/haskell-complete-module hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-complete-module
/usr/share/emacs/site-lisp/haskell-mode/haskell-compile hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-compile
/usr/share/emacs/site-lisp/haskell-mode/haskell-compat hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-compat
/usr/share/emacs/site-lisp/haskell-mode/haskell-commands hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-commands
/usr/share/emacs/site-lisp/haskell-mode/haskell-collapse hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-collapse
/usr/share/emacs/site-lisp/haskell-mode/haskell-checkers hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-checkers
/usr/share/emacs/site-lisp/haskell-mode/haskell-cabal hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-cabal
/usr/share/emacs/site-lisp/haskell-mode/haskell-c2hs hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-c2hs
/usr/share/emacs/site-lisp/haskell-mode/haskell-align-imports hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/haskell-align-imports
/usr/share/emacs/site-lisp/haskell-mode/ghci-script-mode hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/ghci-script-mode
/usr/share/emacs/site-lisp/haskell-mode/ghc-core hides /usr/share/emacs/site-lisp/haskell-mode/build-26.1/ghc-core

Features:
(shadow emacsbug timezone vc-hg vc-git vc-bzr pdf-sync pdf-annot
pdf-outline pdf-links pdf-history smerge-mode diff diff-mode dabbrev rx
term ehelp shr-color flow-fill edebug debug backtrace pop3 gnus-gravatar
gravatar dns gnus-cus gnus-html gnus-kill gnus-logic gnus-mh mh-comp
mh-scan mh-gnus mh-e mh-compat mh-buffers mh-loaddefs gnus-registry
registry gnus-salt gnus-vm cl-print shortdoc locate mule-util ffap
spam-stat nnmairix nnml gnus-uu yenc gnus-sieve gnus-notifications
gnus-fun notifications gnus-eform gnus-dup gnus-diary nndiary gnus-demon
gnus-delay gnus-bookmark deuglify tramp-cmds misearch multi-isearch view
smiley utf-7 imap rfc2104 epa-file mailalias smtpmail sendmail
gnus-bcklg gnus-draft gnus-cite mail-extr gnus-async qp sort gnus-ml
disp-table gnus-topic mm-archive gnutls network-stream url-http url-gw
nsm url-cache url-auth nnrss nndraft nnmh nnfolder nnmaildir gnus-agent
gnus-srvr gnus-score score-mode nnvirtual gnus-msg nntp gnus-cache
tramp-cache tramp-sh tramp tramp-loaddefs trampver tramp-integration
files-x tramp-compat native-complete shell ls-lisp cus-theme xwidget
wid-browse tree-widget dired-aux org-clock diary-lib diary-loaddefs
cal-iso org-indent ol-eww eww xdg url-queue thingatpt mm-url ol-rmail
ol-mhe ol-irc ol-info ol-gnus nnselect gnus-search eieio-opt speedbar
ezimage dframe gnus-art mm-uu mml2015 mm-view mml-smime smime dig
gnus-sum shr kinsoku svg dom gnus-group gnus-undo gnus-start gnus-dbus
dbus xml gnus-cloud nnimap nnmail mail-source utf7 netrc nnoo gnus-spec
gnus-int gnus-range message rmc puny rfc822 mml mml-sec epa epg
epg-config mailabbrev gmm-utils mailheader gnus-win ol-docview ol-bibtex
bibtex ol-bbdb ol-w3m poly-org polymode poly-lock polymode-base
polymode-weave polymode-export polymode-compat polymode-methods
polymode-core polymode-classes eieio-custom eieio-base color matlab
derived matlab-compat pulse face-remap org-agenda paren gnus nnheader
gnus-util rmail rmail-loaddefs mail-utils gnus-dired parse-time iso8601
use-package-bind-key bind-key quelpa-use-package cl-extra
use-package-core quelpa mm-decode mm-bodies mm-encode mail-parse rfc2231
rfc2047 rfc2045 mm-util ietf-drums mail-prsvr lisp-mnt help-fns
radix-tree help-mode ox-odt rng-loc rng-uri rng-parse rng-match rng-dt
rng-util rng-pttrn nxml-parse nxml-ns nxml-enc xmltok nxml-util ox-latex
ox-icalendar ox-html table ox-ascii ox-publish ox org-element avl-tree
generator org-id org-refile haskell-mode-autoloads yaml-mode shell-here
org-recoll edmacro kmacro doc-view org ob ob-tangle ob-ref ob-lob
ob-table ob-exp org-macro org-footnote org-src ob-comint org-pcomplete
pcomplete org-list org-faces org-entities time-date noutline outline
easy-mmode org-version ob-emacs-lisp ob-core ob-eval org-table ol
org-keys org-compat org-macs org-loaddefs cal-menu calendar cal-loaddefs
pdfgrep grep pdf-occur ibuf-ext ibuffer ibuffer-loaddefs tablist
tablist-filter semantic/wisent/comp semantic/wisent
semantic/wisent/wisent semantic/util-modes semantic/util semantic
semantic/tag semantic/lex semantic/fw mode-local find-func cedet
pdf-isearch let-alist pdf-misc imenu pdf-tools compile comint ansi-color
ring cus-edit cus-start cus-load wid-edit pdf-view bookmark
text-property-search pp jka-compr pdf-cache pdf-info tq pdf-util advice
format-spec image-mode dired dired-loaddefs exif server info package
easymenu browse-url url url-proxy url-privacy url-expand url-methods
url-history url-cookie url-domsuf url-util mailcap url-handlers
url-parse auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs
password-cache json subr-x map url-vars seq byte-opt gv bytecomp
byte-compile cconv cl-loaddefs cl-lib iso-transl tooltip eldoc electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel term/pgtk-win
pgtk-win term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice button
loaddefs faces cus-face pcase macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote threads xwidget-internal dbusbind
inotify dynamic-setting font-render-setting cairo move-toolbar gtk
x-toolkit pgtk lcms2 multi-tty make-network-process nativecomp emacs)

Memory information:
((conses 16 2133617 264148)
 (symbols 48 55146 3)
 (strings 32 1087649 82508)
 (string-bytes 1 63791349)
 (vectors 16 393221)
 (vector-slots 8 6065362 537361)
 (floats 8 598 661)
 (intervals 56 87491 968)
 (buffers 984 90))

bug#45650: 28.0.50; nnmaildir discards line count

[Lars Ingebrigtsen]

Justus-dev@Piater.name writes:

> This code assumes that if no Lines: header is found, mail-header-lines
> returns 0.  However, mail-header-lines returns -1 in this case, causing
> the correctly-counted number of lines in nov-mid to be overwritten with
> -1.
>
> Replacing (zerop field) by (<= field 0) fixes it.
>
> While I think this is indeed a correct immediate fix,

That's a very confusing function, but this does indeed seem to be the
correct fix here, and I've now done as you suggested in Emacs 28.

> this function should be refactored:
>
> - Why count the number of lines in the message only to discard the
>   result if a Lines: header is found afterwards? (Well, fixing this
>   won't have much impact: Of the 161594 messages in my nnmaildir
>   folders, only 2982 contain a Lines: header.)
>
> - [unrelated] Above the quoted code section, the function checks if the
>   NOV data have to be (re)created by parsing the message; if not, it
>   raises an exception to return early.  This construct should be
>   replaced by a conditional, perhaps splitting out the second half into
>   its own function for readability.

Both things seem reasonable, so if somebody wants to do that (or rewrite
that function to be less confusing in general), I'm all for it.  But as
your change fixes this bug, I'm closing this bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no