bug#68373: 29.1; Bug in MH-E's support for using openssl to generate the checksum

bug#68373: 29.1; Bug in MH-E's support for using openssl to generate the checksum

[Marc Baaden]

The bug occurred with MH-E when displaying all new messages with
mh-index-new-messages. This supposedly shows first the folder name, then
it's new messages, then next folder name and so on. In my case all new
messages were shown in a list but folder names were missing. In the end
it turned out the reason was that no X-MHE-Checksum: was created in the
messages.

The machine was freshly re-installed after a crash. It natively has md5
installed, but not md5sum. Openssl was installed. In principle md5
should be invoked as fallback for missing md5sum. This did not happen
and no error was thrown. The supposed reason: checking for openssl
may have led to skip an explicit test for md5sum? There has been an
exchange on the mh-e-users list on this topic and it was concluded a bug
report should be filed.




In GNU Emacs 29.1 (build 1, x86_64-apple-darwin18.7.0, NS appkit-1671.60
 Version 10.14.6 (Build 18G9323)) of 2023-08-17 built on
 builder10-14.lan
System Description:  Mac OS X 10.15.7

Configured using:
 'configure --with-ns '--enable-locallisppath=/Library/Application
 Support/Emacs/${version}/site-lisp:/Library/Application
 Support/Emacs/site-lisp' --with-modules 'CFLAGS=-DFD_SETSIZE=10000
 -DDARWIN_UNLIMITED_SELECT' --with-x-toolkit=no'

Configured features:
ACL GLIB GMP GNUTLS JPEG JSON LIBXML2 MODULES NOTIFY KQUEUE NS PDUMPER
PNG RSVG SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER ZLIB

Important settings:
  value of $LANG: fr_FR.UTF-8
  locale-coding-system: utf-8-unix

Major mode: MH-Folder

Minor modes in effect:
  TeX-PDF-mode: t
  TeX-source-correlate-mode: t
  mh-showing-mode: t
  server-mode: t
  desktop-save-mode: t
  override-global-mode: t
  which-key-mode: t
  save-place-mode: t
  recentf-mode: t
  global-hl-line-mode: t
  hl-line-mode: t
  electric-pair-mode: t
  corfu-popupinfo-mode: t
  corfu-terminal-mode: t
  global-corfu-mode: t
  corfu-mode: t
  marginalia-mode: t
  vertico-mode: t
  repeat-mode: t
  winner-mode: t
  savehist-mode: t
  global-so-long-mode: t
  delete-selection-mode: t
  global-auto-revert-mode: t
  auto-insert-mode: t
  doom-modeline-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  buffer-read-only: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
/Users/baaden/.emacs.d/elpa/helm-3.9.7/helm-packages hides /Users/baaden/.emacs.d/elpa/helm-core-3.9.7/helm-packages
/Users/baaden/.emacs.d/elpa/hydra-0.15.0/lv hides /Users/baaden/.emacs.d/elpa/lv-0.15.0/lv
/Users/baaden/.emacs.d/elpa/transient-0.5.3/transient hides /Applications/Emacs.app/Contents/Resources/lisp/transient
/Users/baaden/.emacs.d/elpa/use-package-20230427.12417/use-package-jump hides /Applications/Emacs.app/Contents/Resources/lisp/use-package/use-package-jump
/Users/baaden/.emacs.d/elpa/use-package-20230427.12417/use-package-ensure hides /Applications/Emacs.app/Contents/Resources/lisp/use-package/use-package-ensure
/Users/baaden/.emacs.d/elpa/use-package-20230427.12417/use-package-core hides /Applications/Emacs.app/Contents/Resources/lisp/use-package/use-package-core
/Users/baaden/.emacs.d/elpa/use-package-20230427.12417/use-package-delight hides /Applications/Emacs.app/Contents/Resources/lisp/use-package/use-package-delight
/Users/baaden/.emacs.d/elpa/use-package-20230427.12417/use-package-diminish hides /Applications/Emacs.app/Contents/Resources/lisp/use-package/use-package-diminish
/Users/baaden/.emacs.d/elpa/use-package-20230427.12417/use-package hides /Applications/Emacs.app/Contents/Resources/lisp/use-package/use-package
/Users/baaden/.emacs.d/elpa/use-package-20230427.12417/use-package-bind-key hides /Applications/Emacs.app/Contents/Resources/lisp/use-package/use-package-bind-key
/Users/baaden/.emacs.d/elpa/bind-key-20230203.210438/bind-key hides /Applications/Emacs.app/Contents/Resources/lisp/use-package/bind-key
/Users/baaden/.emacs.d/elpa/use-package-20230427.12417/use-package-lint hides /Applications/Emacs.app/Contents/Resources/lisp/use-package/use-package-lint
/Users/baaden/.emacs.d/elpa/seq-2.24/seq hides /Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/seq

Features:
(shadow sort emacsbug jka-compr flow-fill mh-thread timezone texinfo
texinfo-loaddefs toolbar-x reporter context plain-tex cus-start latex
latex-flymake flymake-proc flymake warnings tex-ispell tex-style tex crm
texmathp compile comint ansi-osc ansi-color loaddefs-gen tar-mode
arc-mode archive-mode network-stream url-cache url-http url-auth url-gw
nsm mh-identity mh-comp pulse face-remap mh-alias shr-color color smiley
textsec uni-scripts idna-mapping uni-confusable textsec-check qp
mm-archive mail-extr mh-mime mh-gnus mh-search mh-letter mh-show
goto-addr gnus-cite gnus-art mm-uu mml2015 mm-view mml-smime smime
gnutls dig gnus-sum shr pixel-fill kinsoku url-file svg dom gnus-group
gnus-undo gnus-start gnus-dbus dbus xml gnus-cloud nnimap nnmail
mail-source utf7 nnoo parse-time iso8601 gnus-spec gnus-int message
sendmail yank-media puny all-the-icons-dired all-the-icons
all-the-icons-faces data-material data-weathericons data-octicons
data-fileicons data-faicons data-alltheicons dired dired-loaddefs rfc822
mml mml-sec epa derived epg rfc6068 epg-config gmm-utils mailheader
gnus-win gnus-range gnus nnheader mail-utils range mh-funcs mh-seq
mh-inc mh-tool-bar mh-acros mh-xface mh-utils mailabbrev mh-folder
which-func imenu gnus-util mh-scan mh-e mh-buffers mh-loaddefs
term/xterm xterm server executable vc-git diff-mode vc-dispatcher
desktop frameset edmacro kmacro use-package-bind-key bind-key
quelpa-use-package quelpa mm-decode mm-bodies mm-encode mail-parse
rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr lisp-mnt help-fns
radix-tree general which-key use-package-diminish saveplace recentf
tree-widget hl-line crafted-updates-config easy-mmode pcase
crafted-writing-config elec-pair crafted-completion-config cape
corfu-popupinfo corfu-terminal popon corfu embark-consult consult
bookmark text-property-search embark ffap thingatpt orderless marginalia
vertico-directory vertico crafted-osx-config crafted-defaults-config
repeat winner savehist hydra ring lv so-long delsel icomplete autorevert
filenotify crafted-ui-packages crafted-writing-packages
crafted-completion-packages crafted-init-config autoinsert project
doom-modeline doom-modeline-segments doom-modeline-env
doom-modeline-core shrink-path rx f s dash nerd-icons nerd-icons-faces
nerd-icons-data nerd-icons-data-mdicon nerd-icons-data-flicon
nerd-icons-data-codicon nerd-icons-data-devicon nerd-icons-data-sucicon
nerd-icons-data-wicon nerd-icons-data-faicon nerd-icons-data-powerline
nerd-icons-data-octicon nerd-icons-data-pomicon nerd-icons-data-ipsicon
compat use-package-ensure doom-palenight-theme doom-themes
doom-themes-base use-package-core modus-vivendi-theme modus-themes
cl-extra help-mode 0blayout-autoloads 0x0-autoloads 0xc-autoloads
2048-game-autoloads 2bit-autoloads 4clojure-autoloads ag-autoloads
all-the-icons-dired-autoloads auctex-autoloads tex-site
auto-package-update-autoloads cape-autoloads citeproc-org-autoloads
citeproc-autoloads company-autoloads corfu-autoloads deferred-autoloads
dired-hide-dotfiles-autoloads dired-open-autoloads
dired-hacks-utils-autoloads dired-single-autoloads
doom-modeline-autoloads doom-themes-autoloads ef-themes-autoloads
embark-consult-autoloads consult-autoloads embark-autoloads
eshell-git-prompt-autoloads esxml-autoloads eterm-256color-autoloads
general-autoloads kv-autoloads lsp-treemacs-autoloads lsp-ui-autoloads
lsp-mode-autoloads magit-autoloads git-commit-autoloads
marginalia-autoloads modus-themes-autoloads nerd-icons-autoloads
org-bullets-autoloads org-ref-autoloads bibtex-completion-autoloads
key-chord-autoloads ivy-autoloads helm-bibtex-autoloads biblio-autoloads
biblio-core-autoloads helm-autoloads helm-core-autoloads async-autoloads
htmlize-autoloads parsebib-autoloads pdf-tools-autoloads
persist-autoloads plz-autoloads popup-autoloads python-mode-autoloads
quelpa-use-package-autoloads quelpa-autoloads queue-autoloads
request-autoloads shrink-path-autoloads simple-httpd-autoloads
spinner-autoloads string-inflection-autoloads finder-inf
tablist-autoloads taxy-magit-section-autoloads taxy-autoloads
magit-section-autoloads transient-autoloads treemacs-autoloads
cfrs-autoloads posframe-autoloads ht-autoloads pfuture-autoloads
ace-window-autoloads avy-autoloads typescript-mode-autoloads
vertico-autoloads visual-fill-column-autoloads vterm-autoloads
webfeeder-autoloads wfnames-autoloads which-key-autoloads
with-editor-autoloads info compat-autoloads seq-autoloads
xterm-color-autoloads crafted-early-init-config time-date cus-edit pp
cus-load icons wid-edit package browse-url url url-proxy url-privacy
url-expand url-methods url-history url-cookie generate-lisp-file
url-domsuf url-util mailcap url-handlers url-parse auth-source cl-seq
eieio eieio-core cl-macs password-cache json subr-x map byte-opt gv
bytecomp byte-compile url-vars cl-loaddefs cl-lib rmc iso-transl tooltip
cconv eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type
elisp-mode mwheel term/ns-win ns-win ucs-normalize mule-util
term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads kqueue cocoa ns multi-tty make-network-process emacs)

Memory information:
((conses 16 877092 105080)
 (symbols 48 41307 66)
 (strings 32 249809 19519)
 (string-bytes 1 5961976)
 (vectors 16 73974)
 (vector-slots 8 1969348 188016)
 (floats 8 1555 5225)
 (intervals 56 12292 5206)
 (buffers 984 101))

-- 
 Dr. Marc Baaden  - Institut de Biologie Physico-Chimique, Paris
 mailto:baaden@smplinux.de      -      http://www.baaden.ibpc.fr
 FAX: +33 15841 5026  -  Tel: +33 15841 5176  ou  +33 609 843217

bug#68373: 29.1; Bug in MH-E's support for using openssl to generate the checksum

[Mike Kupfer]

MH-E will look for md5sum, openssl, and md5, in that order.  The first
one that it finds will be used.  So with just openssl and md5 installed,
MH-E will use openssl.

Given that we are working with MacOS 10, I wonder if there is some
incompatibility between MH-E and the openssl that is installed.  This
could be in the command-line syntax or in the format of the output.

Marc, could you find a text file, run the following 2 commands, and then
reply with the output from the commands?

  openssl version

  openssl md5 <path_to_text_file>

thanks,
mike

bug#68373: 29.1; Bug in MH-E's support for using openssl to generate the checksum

[Marc Baaden]

Hi Mike,

Sure. Concerning MacOS, actually now I am on 12 because when my new
harddisk replacement (sort of memory stick) arrived, it was not
recognized by OS Catalina v10 and I was forced to upgrade. I hope this
will not mess up the bug follow-up.

I used your email text file for this test.

:541; openssl version
OpenSSL 3.1.2 1 Aug 2023 (Library: OpenSSL 3.1.2 1 Aug 2023)

:542; openssl md5 BU/6
MD5(BU/6)= f4aa56ee20c406082d260b0b3b0e3ca1

Let me know if you need anything else,
Kind regards,
Marc

Mike Kupfer <kupfer@rawbw.com> wrote:

> MH-E will look for md5sum, openssl, and md5, in that order.  The first
> one that it finds will be used.  So with just openssl and md5 installed,
> MH-E will use openssl.
> 
> Given that we are working with MacOS 10, I wonder if there is some
> incompatibility between MH-E and the openssl that is installed.  This
> could be in the command-line syntax or in the format of the output.
> 
> Marc, could you find a text file, run the following 2 commands, and then
> reply with the output from the commands?
> 
>   openssl version
> 
>   openssl md5 <path_to_text_file>
> 
> thanks,
> mike
> 

bug#68373: 29.1; Bug in MH-E's support for using openssl to generate the checksum

[Mike Kupfer]

Marc Baaden wrote:

> Sure. Concerning MacOS, actually now I am on 12 because when my new
> harddisk replacement (sort of memory stick) arrived, it was not
> recognized by OS Catalina v10 and I was forced to upgrade. 

Okay.

> I hope this will not mess up the bug follow-up.

It depends. :-)  If you hide the md5sum binary, can you still reproduce
the failure?

thanks,
mike

bug#68373: 29.1; Bug in MH-E's support for using openssl to generate the checksum

[Marc Baaden]

Hmm, I tried to reproduce the issue by renaming all my md5sum instances
so none will be found. It did however not reproduce the error
completely. I get a message "mh-search: Wrong type argument: consp,
nil", but I still have the folders shown and the "correct" behavior of
the unseen overview.

I will try if I can boot into a backed up system disk and reproduce the
original issue. Thank you for your patience.

Best,
Marc


Mike Kupfer <kupfer@rawbw.com> wrote:

> Marc Baaden wrote:
> 
> > Sure. Concerning MacOS, actually now I am on 12 because when my new
> > harddisk replacement (sort of memory stick) arrived, it was not
> > recognized by OS Catalina v10 and I was forced to upgrade. 
> 
> Okay.
> 
> > I hope this will not mess up the bug follow-up.
> 
> It depends. :-)  If you hide the md5sum binary, can you still reproduce
> the failure?
> 
> thanks,
> mike
> 

bug#68373: 29.1; Bug in MH-E's support for using openssl to generate the checksum

[Marc Baaden]

Dear Mike,

I now did additional tests to try and get in the same state where I
observed the issue initially, before I had to upgrade my system due to
hardware incompatibility. I had a system disk backup from which I could
run the previous OS (Mac OS Catalina). Still, I am unable to reproduce
the issue. I have hidden the installed md5sum executables, removed the
previous mhe-index folder and started the folder indexing several times,
first with my original Emacs config, then with Emacs -Q, but now every
time I get the correct behavior.

Sorry about this. I don't know what else I coud try to get back to the
state where the issue occurred.

Best wishes,
Marc


Mike Kupfer <kupfer@rawbw.com> wrote:

> MH-E will look for md5sum, openssl, and md5, in that order.  The first
> one that it finds will be used.  So with just openssl and md5 installed,
> MH-E will use openssl.
> 
> Given that we are working with MacOS 10, I wonder if there is some
> incompatibility between MH-E and the openssl that is installed.  This
> could be in the command-line syntax or in the format of the output.
> 
> Marc, could you find a text file, run the following 2 commands, and then
> reply with the output from the commands?
> 
>   openssl version
> 
>   openssl md5 <path_to_text_file>
> 
> thanks,
> mike
> 

bug#68373: 29.1; Bug in MH-E's support for using openssl to generate the checksum

[Mike Kupfer]

tags 68373 + unreproducible
thanks

Marc Baaden wrote:

> I have hidden the installed md5sum executables, removed the
> previous mhe-index folder and started the folder indexing several times,
> first with my original Emacs config, then with Emacs -Q, but now every
> time I get the correct behavior.

Ah well.  Thanks for trying.

best regards,
mike