bug#22613: 25.1.50; dired-aux missing require 'cl-lib

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Tino Calancha]

[-- Attachment #1: Type: text/plain, Size: 229 bytes --]


emacs -Q /tmp

M-! echo a > foo && echo a > bar RET
g
% m ^\(foo\|bar\)$ RET
Q a RET A RET


In GNU Emacs 25.1.50.1 (x86_64-unknown-linux-gnu, GTK+ Version 2.24.23)
Repository revision: 4ccd2688911b90bcb8cdcd2a8de608e09471a91a


[-- Attachment #2: Type: text/plain, Size: 378 bytes --]

diff --git a/dired-aux.el b/dired-aux.el
index 9bcb1f9..7c1b5e3 100644
--- a/dired-aux.el
+++ b/dired-aux.el
@@ -38,6 +38,7 @@
 ;; We need macros in dired.el to compile properly,
 ;; and we call subroutines in it too.
 (require 'dired)
+(require 'cl-lib)
 
 (defvar dired-create-files-failures nil
   "Variable where `dired-create-files' records failing file names.

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Eli Zaretskii]

> Date: Wed, 10 Feb 2016 22:37:22 +0900 (JST)
> From: Tino Calancha <f92capac@gmail.com>
> 
> M-! echo a > foo && echo a > bar RET
> g
> % m ^\(foo\|bar\)$ RET
> Q a RET A RET

cl-mapcan is an autoloaded function, so requiring cl-lib is not the
right solution, IMO.  Some other factor is at work here.

(This problem doesn't exist on the emacs-25 branch, btw.)

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> cl-mapcan is an autoloaded function, so requiring cl-lib is not the
> right solution, IMO.  Some other factor is at work here.

Yes, hmm, I also saw that once: cl-lib autoloads were not loaded
(i.e. the autoloads file had not been loaded, though it existed).
Didn't investigate at that moment, however.

Michael.

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: Tino Calancha <f92capac@gmail.com>,  22613@debbugs.gnu.org
> Date: Sat, 13 Feb 2016 15:32:45 +0100
> 
> > cl-mapcan is an autoloaded function, so requiring cl-lib is not the
> > right solution, IMO.  Some other factor is at work here.
> 
> Yes, hmm, I also saw that once: cl-lib autoloads were not loaded
> (i.e. the autoloads file had not been loaded, though it existed).

But the only one who seems to load cl-loaddefs.el is cl-lib.el, so
perhaps requiring it _is_ TRT after all.

Or maybe we should make cl-mapcan autoloaded in loaddefs.el?

What is confusing is that "emacs -Q" knows cl-mapcan is autoloaded,
even though its autoload form was not loaded, evidently.  Where does
it get this information, and why can't it get the autoload form from
the same place?

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> What is confusing is that "emacs -Q" knows cl-mapcan is autoloaded,
> even though its autoload form was not loaded, evidently.  Where does
> it get this information, and why can't it get the autoload form from
> the same place?

Maybe this is package.el related - cl-lib is a built-in package.  I had
hoped to find some information in (info "(elisp) Startup Summary").

Michael.

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: f92capac@gmail.com,  22613@debbugs.gnu.org
> Date: Sat, 13 Feb 2016 17:17:06 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > What is confusing is that "emacs -Q" knows cl-mapcan is autoloaded,
> > even though its autoload form was not loaded, evidently.  Where does
> > it get this information, and why can't it get the autoload form from
> > the same place?
> 
> Maybe this is package.el related - cl-lib is a built-in package.  I had
> hoped to find some information in (info "(elisp) Startup Summary").

Sorry, I'm confused: what does package.el have to do with this issue,
and what do you mean by "a built-in package" in this context?

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> Sorry, I'm confused: what does package.el have to do with this issue,
> and what do you mean by "a built-in package" in this context?

I mean: in the package manager, cl-lib is listed as "built-in" package.
And packages also have autoloads.  So maybe the autoload definition for
cl-mapcan we didn't understand why it is dloaded at all comes from the
package mechanism.  Just speculating.

Michael.

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Glenn Morris]

Eli Zaretskii wrote:

> But the only one who seems to load cl-loaddefs.el is cl-lib.el, so
> perhaps requiring it _is_ TRT after all.

Yes.

> Or maybe we should make cl-mapcan autoloaded in loaddefs.el?

No.

> What is confusing is that "emacs -Q" knows cl-mapcan is autoloaded,
> even though its autoload form was not loaded, evidently.

Because something is causing cl-lib to be loaded.

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: f92capac@gmail.com,  22613@debbugs.gnu.org
> Date: Sat, 13 Feb 2016 18:43:39 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Sorry, I'm confused: what does package.el have to do with this issue,
> > and what do you mean by "a built-in package" in this context?
> 
> I mean: in the package manager, cl-lib is listed as "built-in" package.
> And packages also have autoloads.  So maybe the autoload definition for
> cl-mapcan we didn't understand why it is dloaded at all comes from the
> package mechanism.  Just speculating.

Could be.  But what bothers me is that Emacs _knows_ it's autoloaded,
and from which file, but it cannot autoload it.  That shouldn't
happen, no matter where did the knowledge come from.  Right?

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: Michael Heerdegen <michael_heerdegen@web.de>,  f92capac@gmail.com,  22613@debbugs.gnu.org
> Date: Sat, 13 Feb 2016 12:58:35 -0500
> 
> Eli Zaretskii wrote:
> 
> > But the only one who seems to load cl-loaddefs.el is cl-lib.el, so
> > perhaps requiring it _is_ TRT after all.
> 
> Yes.

Done.

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Glenn Morris]

Eli Zaretskii wrote:

>> > But the only one who seems to load cl-loaddefs.el is cl-lib.el, so
>> > perhaps requiring it _is_ TRT after all.
>> 
>> Yes.
>
> Done.

Thanks, but wrong branch I think! :)
I did it simultaneously in emacs-25.

Not sure what's causing cl-lib to be loaded at startup - that shouldn't
happen.

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Glenn Morris]

Glenn Morris wrote:

> Not sure what's causing cl-lib to be loaded at startup - that shouldn't
> happen.

It only happens in -nw mode, so I guess it is the use of cl-defmethod in
term/xterm.el. I'm not sure if this (cl-lib always being loaded in emacs
-Q -nw) is a problem or not. I guess not? (Stefan, any opinion?)

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Stefan Monnier]

>> Not sure what's causing cl-lib to be loaded at startup - that shouldn't
>> happen.
> It only happens in -nw mode, so I guess it is the use of cl-defmethod in
> term/xterm.el. I'm not sure if this (cl-lib always being loaded in emacs
> -Q -nw) is a problem or not. I guess not? (Stefan, any opinion?)

Hmm... cl-defmethod should cause cl-lib to be loaded (cl-generic is
preloaded, after all).

Usually we consider it a bug if "emacs -Q" (or "emacs -nw -Q") needs to
load a package, tho we tend not to apply that rule to lisp/term/*.el.

In any case, the fact that "emacs -Q -nw" loads cl-lib is not intended
and I think it'd be good to track down why it does and if/how we can fix it.


        Stefan

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Stefan Monnier]

> Hmm... cl-defmethod should cause cl-lib to be loaded (cl-generic is
                           ^^^
                           not


        Stefan "Damn it!"

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Michael Heerdegen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Hmm... cl-defmethod should cause cl-lib to be loaded (cl-generic is
> preloaded, after all).

(yes, "should not")

It seems to come from the call to `byte-compile' in
`cl--generic-get-dispatcher' at load time.  "byte-comp" requires
"cl-extra", and that "cl-lib".

Michael.

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: michael_heerdegen@web.de,  f92capac@gmail.com,  22613@debbugs.gnu.org
> Date: Sat, 13 Feb 2016 13:20:19 -0500
> 
> Eli Zaretskii wrote:
> 
> >> > But the only one who seems to load cl-loaddefs.el is cl-lib.el, so
> >> > perhaps requiring it _is_ TRT after all.
> >> 
> >> Yes.
> >
> > Done.
> 
> Thanks, but wrong branch I think! :)
> I did it simultaneously in emacs-25.

No, I did it exactly to the right branch: this problem doesn't exist
on emacs-25 (as I wrote previously).

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Glenn Morris]

Eli Zaretskii wrote:

> No, I did it exactly to the right branch: this problem doesn't exist
> on emacs-25 (as I wrote previously).

There's no difference between dired-aux in emacs-25 and master.
Of course the emacs-25 version needs cl-lib, since it uses cl- functions.
Perhaps you were testing in an Emacs where cl-lib happened to be loaded.
Eg a -nw one, as we have discussed.

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: michael_heerdegen@web.de,  f92capac@gmail.com,  22613@debbugs.gnu.org
> Date: Sat, 13 Feb 2016 16:23:27 -0500
> 
> Perhaps you were testing in an Emacs where cl-lib happened to be loaded.
> Eg a -nw one, as we have discussed.

No, I didn't.

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Stefan Monnier]

>> Hmm... cl-defmethod should cause cl-lib to be loaded (cl-generic is
>> preloaded, after all).
> (yes, "should not")
> It seems to come from the call to `byte-compile' in

Oh, right, just as for the preloaded uses of cl-generic, if we want to
avoid loading cl-lib, we'd have to pre-fill cl-generic's dispatcher
cache with the specific dispatcher used (grep for
cl--generic-prefill-dispatchers to see what that looks like).


        Stefan

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Glenn Morris]

Eli Zaretskii wrote:

>> Perhaps you were testing in an Emacs where cl-lib happened to be loaded.
>> Eg a -nw one, as we have discussed.
>
> No, I didn't.

Then I have no explanation for how your dired-aux could possibly use
cl-* functions without loading cl-lib.

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: michael_heerdegen@web.de,  f92capac@gmail.com,  22613@debbugs.gnu.org
> Date: Sun, 14 Feb 2016 17:47:16 -0500
> 
> Eli Zaretskii wrote:
> 
> >> Perhaps you were testing in an Emacs where cl-lib happened to be loaded.
> >> Eg a -nw one, as we have discussed.
> >
> > No, I didn't.
> 
> Then I have no explanation for how your dired-aux could possibly use
> cl-* functions without loading cl-lib.

I don't have an explanation, either.  I just know that the OP's report
was for the master branch, and the problem doesn't happen on emacs-25.

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Stephen Berman]

On Mon, 15 Feb 2016 05:36:24 +0200 Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Glenn Morris <rgm@gnu.org>
>> Cc: michael_heerdegen@web.de,  f92capac@gmail.com,  22613@debbugs.gnu.org
>> Date: Sun, 14 Feb 2016 17:47:16 -0500
>> 
>> Eli Zaretskii wrote:
>> 
>> >> Perhaps you were testing in an Emacs where cl-lib happened to be loaded.
>> >> Eg a -nw one, as we have discussed.
>> >
>> > No, I didn't.
>> 
>> Then I have no explanation for how your dired-aux could possibly use
>> cl-* functions without loading cl-lib.
>
> I don't have an explanation, either.  I just know that the OP's report
> was for the master branch, and the problem doesn't happen on emacs-25.

I'm late to this thread, but when I execute the OP's recipe in emacs-25
revision a91b4b51ddf2575d821adb8b84fdf32cff83886e (i.e. before Glenn's
patch was applied), I do get the error "dired-do-find-regexp: Symbol’s
function definition is void: cl-mapcan".  My build: GNU Emacs 25.0.90.2
(x86_64-suse-linux-gnu, GTK+ Version 3.14.15) of 2016-02-11.

Steve Berman

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Michael Heerdegen]

Stephen Berman <stephen.berman@gmx.net> writes:

> > I don't have an explanation, either.  I just know that the OP's report
> > was for the master branch, and the problem doesn't happen on emacs-25.
>
> I'm late to this thread, but when I execute the OP's recipe in emacs-25
> revision a91b4b51ddf2575d821adb8b84fdf32cff83886e (i.e. before Glenn's
> patch was applied), I do get the error "dired-do-find-regexp: Symbol’s
> function definition is void: cl-mapcan".  My build: GNU Emacs 25.0.90.2
> (x86_64-suse-linux-gnu, GTK+ Version 3.14.15) of 2016-02-11.

Isn't there actually something wrong with autoloading?  `cl-mapcan' has
an autoload cookie above its def in "cl-extra".  But if you try to call
it in scratch or with M-: in emacs -Q, it's unknown.

C-h f cl-mapcan shows the correct doc then, but only because the help
stuff requires cl-lib.

Michael.

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Andreas Schwab]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Isn't there actually something wrong with autoloading?  `cl-mapcan' has
> an autoload cookie above its def in "cl-extra".  But if you try to call
> it in scratch or with M-: in emacs -Q, it's unknown.

Autoloads from cl-* are collected in cl-loaddefs, which is only loaded
by cl-lib.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Glenn Morris]

Stefan Monnier wrote:

> Oh, right, just as for the preloaded uses of cl-generic, if we want to
> avoid loading cl-lib, we'd have to pre-fill cl-generic's dispatcher
> cache with the specific dispatcher used (grep for
> cl--generic-prefill-dispatchers to see what that looks like).

I cargo-culted the following, which seems to work.
Is it what you meant?

--- a/lisp/emacs-lisp/cl-generic.el
+++ b/lisp/emacs-lisp/cl-generic.el
@@ -1019,6 +1019,10 @@ cl--generic-eql-used
 
 (cl--generic-prefill-dispatchers 0 (eql nil))
 (cl--generic-prefill-dispatchers window-system (eql nil))
+(cl--generic-prefill-dispatchers (terminal-parameter nil 'xterm--get-selection)
+                                 (eql nil))
+(cl--generic-prefill-dispatchers (terminal-parameter nil 'xterm--set-selection)
+                                 (eql nil))
 
 ;;; Support for cl-defstructs specializers.
 

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Stefan Monnier]

>> Oh, right, just as for the preloaded uses of cl-generic, if we want to
>> avoid loading cl-lib, we'd have to pre-fill cl-generic's dispatcher
>> cache with the specific dispatcher used (grep for
>> cl--generic-prefill-dispatchers to see what that looks like).

> I cargo-culted the following, which seems to work.
> Is it what you meant?

Looks about right, yes.


        Stefan

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Date: Tue, 16 Feb 2016 02:07:17 -0500
> Cc: Michael Heerdegen <michael_heerdegen@web.de>, 22613@debbugs.gnu.org
> 
> --- a/lisp/emacs-lisp/cl-generic.el
> +++ b/lisp/emacs-lisp/cl-generic.el
> @@ -1019,6 +1019,10 @@ cl--generic-eql-used
 
>  (cl--generic-prefill-dispatchers 0 (eql nil))
>  (cl--generic-prefill-dispatchers window-system (eql nil))
> +(cl--generic-prefill-dispatchers (terminal-parameter nil 'xterm--get-selection)
> +                                 (eql nil))
> +(cl--generic-prefill-dispatchers (terminal-parameter nil 'xterm--set-selection)
> +                                 (eql nil))

Is the problem really worth putting such "magic" into our sources?  If
it does, I think we should document this somewhere, at least.

Thanks.

bug#22613: 25.1.50; dired-aux missing require 'cl-lib

[Stefan Monnier]

>> +(cl--generic-prefill-dispatchers (terminal-parameter nil 'xterm--get-selection)
>> +                                 (eql nil))
>> +(cl--generic-prefill-dispatchers (terminal-parameter nil 'xterm--set-selection)
>> +                                 (eql nil))

> Is the problem really worth putting such "magic" into our sources?  If
> it does, I think we should document this somewhere, at least.

Maybe we can also improve the situation by putting those lines right
where they come from, and "preload" them via some kind of
autoload cookie.


        Stefan