bug#42765: 26.3; project-find-regexp broken in Emacs 26.3

bug#42765: 26.3; project-find-regexp broken in Emacs 26.3

[Philip K.]

Hi,

after loading project.el into Emacs 26.3, I noticed that
project-find-regexp doesn't work, because the xref--show-xrefs function
has changed it's parameter interpretation. While project.el assumes it
accepts a function, 26.3's Xref assumes it will recieve a list
("xrefs"). There command then fails with a somewhat cryptic error
message, that probably doesn't make sense for Elisp programmers.

I managed to fix this by installing a newer Xref version from ELPA, but
I think this situation should be handled more gracefully. Is there a
reason that project.el doesn't depend on the newer Xref version?


In GNU Emacs 26.3 (build 1, x86_64-redhat-linux-gnu, GTK+ Version 3.24.13)
 of 2020-01-28 built on buildhw-09.phx2.fedoraproject.org
Windowing system distributor 'Fedora Project', version 11.0.12008000
System Description:	Fedora release 32 (Thirty Two)

Recent messages:
0 new messages read
Parsing /home/phi/etc/mail/aliases... done
Starting new Ispell process /usr/bin/enchant2 with default dictionary...
Error enabling Flyspell mode:
(Searching for program No such file or directory /usr/bin/enchant2)
Mark set
Quit
No expansion found
Quit
Mark set

Configured using:
 'configure --build=x86_64-redhat-linux-gnu
 --host=x86_64-redhat-linux-gnu --program-prefix=
 --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr
 --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc
 --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64
 --libexecdir=/usr/libexec --localstatedir=/var
 --sharedstatedir=/var/lib --mandir=/usr/share/man
 --infodir=/usr/share/info --with-dbus --with-gif --with-jpeg --with-png
 --with-rsvg --with-tiff --with-xft --with-xpm --with-x-toolkit=gtk3
 --with-gpm=no --with-xwidgets --with-modules
 build_alias=x86_64-redhat-linux-gnu host_alias=x86_64-redhat-linux-gnu
 'CFLAGS=-DMAIL_USE_LOCKF -O2 -g -pipe -Wall -Werror=format-security
 -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions
 -fstack-protector-strong -grecord-gcc-switches
 -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic
 -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection'
 LDFLAGS=-Wl,-z,relro
 PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND DBUS GSETTINGS GLIB NOTIFY
ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS XWIDGETS LCMS2

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Emacs-Lisp

Minor modes in effect:
  TeX-PDF-mode: t
  global-magit-file-mode: t
  magit-file-mode: t
  magit-auto-revert-mode: t
  global-git-commit-mode: t
  async-bytecomp-package-mode: t
  shell-dirtrack-mode: t
  paredit-mode: t
  flymake-mode: t
  ivy-mode: t
  display-time-mode: t
  recentf-mode: t
  save-place-mode: t
  savehist-mode: t
  show-paren-mode: t
  winner-mode: t
  diff-auto-refine-mode: t
  electric-pair-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
/home/phi/.emacs.d/elpa/flymake-1.0.9/flymake hides /usr/share/emacs/26.3/lisp/progmodes/flymake
/home/phi/.emacs.d/elpa/project-0.5.0/project hides /usr/share/emacs/26.3/lisp/progmodes/project
/home/phi/.emacs.d/elpa/let-alist-1.0.6/let-alist hides /usr/share/emacs/26.3/lisp/emacs-lisp/let-alist

Features:
(shadow emacsbug mail-extr pulse cl-print hi-lock windmove gdb-mi bindat
gud etags arc-mode archive-mode hyperspec cc-awk preview prv-emacs
tex-buf font-latex texmathp latex latex-flymake tex-ispell tex-style tex
dbus tex-mode org-rmail org-mhe org-irc org-info org-gnus nnir gnus-sum
gnus-group gnus-undo gnus-start gnus-cloud nnimap nnmail mail-source
utf7 netrc nnoo gnus-spec gnus-int gnus-range gnus-win org-docview
doc-view image-mode org-bibtex bibtex org-bbdb org-w3m org-element
avl-tree generator org org-macro org-footnote org-pcomplete org-list
org-faces org-entities noutline outline org-version ob-emacs-lisp ob
ob-tangle org-src ob-ref ob-lob ob-table ob-keys ob-exp ob-comint
ob-core ob-eval org-compat org-macs org-loaddefs conf-mode calc-misc
calc-alg calc-ext calc-menu calc calc-loaddefs calc-macs dired-aux avy
files-x tramp-cache tramp-sh make-mode magit-bookmark magit-submodule
magit-obsolete magit-blame magit-stash magit-bisect magit-push
magit-pull magit-fetch magit-clone magit-remote magit-commit
magit-sequence magit-notes magit-worktree magit-tag magit-merge
magit-branch magit-reset magit-collab ghub-graphql treepy gsexp ghub
url-http tls url-gw nsm url-auth gnutls magit-files magit-refs
magit-status magit magit-repos magit-apply magit-wip magit-log
which-func magit-diff smerge-mode magit-core magit-autorevert autorevert
filenotify magit-process magit-margin magit-mode git-commit magit-git
magit-section magit-utils magit-popup crm log-edit pcvs-util add-log
with-editor async-bytecomp async server dash swiper bookmark flyspell
ispell nroff-mode char-fold misearch multi-isearch tabify imenu man
clang-capf sort ffap tramp tramp-compat tramp-loaddefs trampver
ucs-normalize parse-time advice whitespace help-at-pt jka-compr
eieio-opt speedbar sb-image ezimage dframe help-fns xref find-dired grep
vc-mtn vc-hg macrostep-c cmacexp macrostep cc-mode cc-fonts cc-guess
cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs vc-bzr
vc-src vc-sccs vc-svn vc-cvs vc-rcs vc vc-dispatcher project subr-x
shell pcomplete bang shr svg dom browse-url qp rmailmm message rmc puny
rfc822 mml mml-sec epa derived epg mm-decode mm-bodies mm-encode
mailabbrev gmm-utils mailheader mail-parse rfc2231 cl-extra time-stamp
paredit checkdoc help-mode flymake-proc flymake warnings init ivy delsel
colir color ivy-overlay edmacro kmacro rx pcase dired-x holidays
hol-loaddefs cal-menu calendar cal-loaddefs erc-goodies erc thingatpt
erc-backend erc-compat format-spec gnus nnheader gnus-util time sendmail
rmail rmail-loaddefs rfc2047 rfc2045 ietf-drums mm-util mail-prsvr
mail-utils hippie-exp recentf tree-widget saveplace savehist paren
winner cus-edit cus-start cus-load wid-edit ert pp find-func ewoc debug
url url-proxy url-privacy url-expand url-methods url-history url-cookie
url-domsuf url-util mailcap compile comint ansi-color ring vc-fossil
vc-git diff-mode easy-mmode elec-pair autoload radix-tree lisp-mnt dired
dired-loaddefs tex-site finder-inf slime-autoloads info package easymenu
epg-config url-handlers url-parse auth-source cl-seq eieio eieio-core
eieio-loaddefs password-cache url-vars clang-rename clang-include-fixer
let-alist json map seq byte-opt bytecomp byte-compile cconv clang-format
cl-macs gv xml cl-loaddefs cl-lib time-date mule-util tooltip eldoc
electric uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win
x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify lcms2 dynamic-setting
system-font-setting font-render-setting xwidget-internal move-toolbar
gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 889607 199216)
 (symbols 48 62145 1)
 (miscs 40 4764 8433)
 (strings 32 199456 21078)
 (string-bytes 1 5733700)
 (vectors 16 95662)
 (vector-slots 8 2270580 109232)
 (floats 8 608 1112)
 (intervals 56 40233 4024)
 (buffers 992 91))

bug#42765: 26.3; project-find-regexp broken in Emacs 26.3

[Dmitry Gutov]

Hi Philip!

On 08.08.2020 19:42, Philip K. wrote:

> after loading project.el into Emacs 26.3, I noticed that
> project-find-regexp doesn't work, because the xref--show-xrefs function
> has changed it's parameter interpretation. While project.el assumes it
> accepts a function, 26.3's Xref assumes it will recieve a list
> ("xrefs"). There command then fails with a somewhat cryptic error
> message, that probably doesn't make sense for Elisp programmers.

Thank you very much for testing this.

> I managed to fix this by installing a newer Xref version from ELPA, but
> I think this situation should be handled more gracefully. Is there a
> reason that project.el doesn't depend on the newer Xref version?

Unfortunately, that would make project.el and xref recursively depend on 
each other. And apparently that would be a problem: 
https://lists.gnu.org/archive/html/emacs-devel/2020-05/msg01615.html

I'm not sure what's the best way to fix this.

bug#42765: 26.3; project-find-regexp broken in Emacs 26.3

[Philip K.]

Dmitry Gutov <dgutov@yandex.ru> writes:

>> I managed to fix this by installing a newer Xref version from ELPA, but
>> I think this situation should be handled more gracefully. Is there a
>> reason that project.el doesn't depend on the newer Xref version?
>
> Unfortunately, that would make project.el and xref recursively depend on 
> each other. And apparently that would be a problem: 
> https://lists.gnu.org/archive/html/emacs-devel/2020-05/msg01615.html
>
> I'm not sure what's the best way to fix this.

Couldln't Xref be replaced with some other listing interface, like
what occur uses?

bug#42765: 26.3; project-find-regexp broken in Emacs 26.3

[Dmitry Gutov]

On 10.08.2020 13:46, Philip K. wrote:
> Couldln't Xref be replaced with some other listing interface, like
> what occur uses?

In theory -- yes.

Though Occur would need to be adapted for this to be possible, and I 
probably would enjoy the result less.

bug#42765: 26.3; project-find-regexp broken in Emacs 26.3

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1797 bytes --]

On Mon, Aug 10, 2020 at 2:17 AM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
> Hi Philip!
>
> On 08.08.2020 19:42, Philip K. wrote:
>
> > after loading project.el into Emacs 26.3, I noticed that
> > project-find-regexp doesn't work, because the xref--show-xrefs function
> > has changed it's parameter interpretation. While project.el assumes it
> > accepts a function, 26.3's Xref assumes it will recieve a list
> > ("xrefs"). There command then fails with a somewhat cryptic error
> > message, that probably doesn't make sense for Elisp programmers.
>
> Thank you very much for testing this.
>
> > I managed to fix this by installing a newer Xref version from ELPA, but
> > I think this situation should be handled more gracefully. Is there a
> > reason that project.el doesn't depend on the newer Xref version?
>
> Unfortunately, that would make project.el and xref recursively depend on
> each other. And apparently that would be a problem:
> https://lists.gnu.org/archive/html/emacs-devel/2020-05/msg01615.html
>
> I'm not sure what's the best way to fix this.

Cyclic dependencies are bad in any packaging system. If two packages
depend on each other as a cycle, they are for packaging purposes "the same
package".  So Stefan's suggestion to make a multi-file-package seems
sensible.

The other common way to solve this is to split one of the packages, breaking
the cycle.  Or maybe creating a third "interface" package and adding an
indirection
(not sure if that's what Philip is suggesting).

So there seem to be escape hatches here, the best one is found by looking
exactly at what breaks, what are the external interfaces of each package,
why
and where they _need_ to depend on each other. I'm afraid I don't have time
to do this right  now.

João

[-- Attachment #2: Type: text/html, Size: 2355 bytes --]

bug#42765: 26.3; project-find-regexp broken in Emacs 26.3

[Dmitry Gutov]

On 08.08.2020 19:42, Philip K. wrote:
> I managed to fix this by installing a newer Xref version from ELPA, but
> I think this situation should be handled more gracefully. Is there a
> reason that project.el doesn't depend on the newer Xref version?

I have pushed a fix for, hopefully. Commit 319463920c.

It inverts the dependencies, making xref only depend on project.el 
implicitly, by requiring Emacs >= 26.3, and making it work with the 
older API. And making project depend on the latest xref.

Hopefully this reversal won't create any problems when upgrading.

bug#42765: 26.3; project-find-regexp broken in Emacs 26.3

[Stefan Kangas]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 08.08.2020 19:42, Philip K. wrote:
>> I managed to fix this by installing a newer Xref version from ELPA, but
>> I think this situation should be handled more gracefully. Is there a
>> reason that project.el doesn't depend on the newer Xref version?
>
> I have pushed a fix for, hopefully. Commit 319463920c.
>
> It inverts the dependencies, making xref only depend on project.el implicitly,
> by requiring Emacs >= 26.3, and making it work with the older API. And making
> project depend on the latest xref.
>
> Hopefully this reversal won't create any problems when upgrading.

So is there anything more to do here, or should this bug be closed?

bug#42765: 26.3; project-find-regexp broken in Emacs 26.3

[Dmitry Gutov]

On 01.10.2020 15:50, Stefan Kangas wrote:
> Dmitry Gutov <dgutov@yandex.ru> writes:
> 
>> On 08.08.2020 19:42, Philip K. wrote:
>>> I managed to fix this by installing a newer Xref version from ELPA, but
>>> I think this situation should be handled more gracefully. Is there a
>>> reason that project.el doesn't depend on the newer Xref version?
>>
>> I have pushed a fix for, hopefully. Commit 319463920c.
>>
>> It inverts the dependencies, making xref only depend on project.el implicitly,
>> by requiring Emacs >= 26.3, and making it work with the older API. And making
>> project depend on the latest xref.
>>
>> Hopefully this reversal won't create any problems when upgrading.
> 
> So is there anything more to do here, or should this bug be closed?

No further reports, so let's close it, thanks.