bug#27933: emacs25: default info dir initialization slow with remote filesystems

bug#27933: emacs25: default info dir initialization slow with remote filesystems

[Rob Browning]

[If possible please preserve the 786707-forwarded@bugs.debian.org in replies]

It appears that when remote filesystems are involved, the probing to
initialize Info-default-directory-alist can be time consuming, even if
an INFOPATH has been specified.

This was reported as http://bugs.debian.org/786707, and additional
details are available there.

Thanks
-- 
Rob Browning
rlb @defaultvalue.org and @debian.org
GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A
GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4

bug#27933: emacs25: default info dir initialization slow with remote filesystems

[Glenn Morris]

Presumably this is less bad since some changes in Emacs master.
See 8ff9c4109fd and 2a2c6ee8d18.

http://lists.gnu.org/archive/html/emacs-diffs/2016-05/msg00336.html
http://lists.gnu.org/archive/html/emacs-diffs/2016-05/msg00338.html

bug#27933: emacs25: default info dir initialization slow with remote filesystems

[Steve McIntyre]

On Thu, Aug 03, 2017 at 11:29:52AM -0400, Glenn Morris wrote:
>
>Presumably this is less bad since some changes in Emacs master.
>See 8ff9c4109fd and 2a2c6ee8d18.
>
>http://lists.gnu.org/archive/html/emacs-diffs/2016-05/msg00336.html
>http://lists.gnu.org/archive/html/emacs-diffs/2016-05/msg00338.html

Hi Glenn,

I've checked yesterday using emacs25 (version 25.1+1-4 in Debian
stretch). I guess those changes didn't make it in before that release
- the second diff you've linked is exactly the trivial change I've
been using locally to solve the problem for me... :-)

-- 
Steve McIntyre, Cambridge, UK.                                steve@einval.com
"I can't ever sleep on planes ... call it irrational if you like, but I'm
 afraid I'll miss my stop" -- Vivek Das Mohapatra

bug#27933: emacs25: default info dir initialization slow with remote filesystems

[Glenn Morris]

Steve McIntyre wrote:

> I've checked yesterday using emacs25 (version 25.1+1-4 in Debian
> stretch). I guess those changes didn't make it in before that release

Indeed not.

bug#27933: emacs25: default info dir initialization slow with remote filesystems

[Steve McIntyre]

On Thu, Aug 03, 2017 at 12:41:48PM -0400, Glenn Morris wrote:
>Steve McIntyre wrote:
>
>> I've checked yesterday using emacs25 (version 25.1+1-4 in Debian
>> stretch). I guess those changes didn't make it in before that release
>
>Indeed not.

OK, cool - thanks for clarifying!

So, I am still curious about a couple of points from my initial bug
report. Maybe you can help with those too?

>2. Why on earth is emacs checking the info path at all on startup?
>   Shouldn't it wait until info functions are actually called for?
>
>3. The documentation talks about setting INFOPATH in the environment
>   to over-ride the use of this default directory list, but again
>   using strace I can see no change to behaviour here at all if I set
>   INFOPATH

-- 
Steve McIntyre, Cambridge, UK.                                steve@einval.com
"Arguing that you don't care about the right to privacy because you have
 nothing to hide is no different than saying you don't care about free
 speech because you have nothing to say."
   -- Edward Snowden

bug#27933: emacs25: default info dir initialization slow with remote filesystems

[Eli Zaretskii]

> Date: Thu, 3 Aug 2017 17:49:13 +0100
> From: Steve McIntyre <steve@einval.com>
> Cc: 27933@debbugs.gnu.org, 786707@bugs.debian.org,
> 	Rob Browning <rlb@defaultvalue.org>, 786707-forwarded@bugs.debian.org
> 
> >2. Why on earth is emacs checking the info path at all on startup?

It doesn't do that here, not in the current development sources.
After I start "emacs -Q", (featurep 'info) evaluates to nil and
Info-directory-list is unbound.

> >   Shouldn't it wait until info functions are actually called for?

It should, and it does.

> >3. The documentation talks about setting INFOPATH in the environment
> >   to over-ride the use of this default directory list, but again
> >   using strace I can see no change to behaviour here at all if I set
> >   INFOPATH

My reading of the code is that if INFOPATH is set in the environment
of the shell from which Emacs is started, then the value of that
variable is used in preference to Info-default-directory-list.  Maybe
you didn't start Emacs from the shell whose environment you changed?

bug#27933: emacs25: default info dir initialization slow with remote filesystems

[Glenn Morris]

Eli Zaretskii wrote:

>> >2. Why on earth is emacs checking the info path at all on startup?
>
> It doesn't do that here, not in the current development sources.
> After I start "emacs -Q", (featurep 'info) evaluates to nil and
> Info-directory-list is unbound.

Info-default-directory-list, which is the thing that stats the
filesystem, is autoloaded.

This area seems overly complicated, probably due to hysterical raisins.
IIRC we've discussed it before.

>> >3. The documentation talks about setting INFOPATH in the environment
>> >   to over-ride the use of this default directory list, but again
>> >   using strace I can see no change to behaviour here at all if I set
>> >   INFOPATH

The default still gets set, just not used. You might wonder, why bother
setting something that's not used...


Ideally Info-default-directory-list would go away. leaving only
Info-directory-list initialized in a standard fashion. But I haven't
looked at the details.

bug#27933: emacs25: default info dir initialization slow with remote filesystems

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: Steve McIntyre <steve@einval.com>,  27933@debbugs.gnu.org,  786707@bugs.debian.org,  rlb@defaultvalue.org,  786707-forwarded@bugs.debian.org
> Date: Thu, 03 Aug 2017 13:35:25 -0400
> 
> Eli Zaretskii wrote:
> 
> >> >2. Why on earth is emacs checking the info path at all on startup?
> >
> > It doesn't do that here, not in the current development sources.
> > After I start "emacs -Q", (featurep 'info) evaluates to nil and
> > Info-directory-list is unbound.
> 
> Info-default-directory-list, which is the thing that stats the
> filesystem, is autoloaded.

But who loads it?

bug#27933: emacs25: default info dir initialization slow with remote filesystems

[Glenn Morris]

Eli Zaretskii wrote:

>> Info-default-directory-list, which is the thing that stats the
>> filesystem, is autoloaded.
>
> But who loads it?

Autoloaded variables are dumped with Emacs.

bug#27933: emacs25: default info dir initialization slow with remote filesystems

[Lars Ingebrigtsen]

Glenn Morris <rgm@gnu.org> writes:

> Ideally Info-default-directory-list would go away. leaving only
> Info-directory-list initialized in a standard fashion. But I haven't
> looked at the details.

I poked a bit at this, and it seems hard to make this work in a
backwards-compatible way.  The problem is that we document stuff like
this to work:

(add-to-list 'Info-default-directory-list "~/Info")

And we compute Info-default-directory-list at startup (and it stats a
bunch of directories, which is what this bug report is about).  The user
can therefore assume that it's been filled, and may be removing
directories from it, too, so if we just initialise it (or do statting
while initialising Info-directory-list), we'll be breaking people's
setups.  I.e., people may have

(setq Info-default-directory-list (delete "/usr/share/info/" Info-default-directory-list))

in .emacs, and that won't work.

But I think we should indeed fix this, because statting those
directories at startup isn't ideal, but it'd be a breaking change.

I propose to make Info-default-directory-list nil by default, and
initialise Info-directory-alist from the directories that
Info-default-directory-list would normally find, as well as
Info-default-directory-list (unless INFODIR is set, and that overrides).

I think the breakage for users should be pretty minimal -- it would only
affect people that remove stuff from Info-default-directory-list, but
not those that add to it.

Any comments before I make this change?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#27933: emacs25: default info dir initialization slow with remote filesystems

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> I think the breakage for users should be pretty minimal -- it would only
> affect people that remove stuff from Info-default-directory-list, but
> not those that add to it.
>
> Any comments before I make this change?

There were no comments in a month, so I went ahead and made the change
in Emacs 29.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#27933: emacs25: default info dir initialization slow with remote filesystems

[Steve McIntyre]

On Fri, Mar 04, 2022 at 05:36:38PM +0100, Lars Ingebrigtsen wrote:
>Lars Ingebrigtsen <larsi@gnus.org> writes:
>
>> I think the breakage for users should be pretty minimal -- it would only
>> affect people that remove stuff from Info-default-directory-list, but
>> not those that add to it.
>>
>> Any comments before I make this change?
>
>There were no comments in a month, so I went ahead and made the change
>in Emacs 29.

Thanks for your work Lars!

Cheers,

Steve

-- 
Steve McIntyre, Cambridge, UK.                                steve@einval.com
"This dress doesn't reverse." -- Alden Spiess

bug#27933: emacs25: default info dir initialization slow with remote filesystems

[Alfred M. Szmidt]

   >    > Standard info does the right thing, and shows both local (those listed
   >    > by INFOPATH) dir files, and the system ones.
   >    >
   >    > Reverting the change (5f7db63a93f0c670d6f88dd24f922462abce7d8c) makes
   >    > things work as expected.
   >
   >    "Alfred M. Szmidt" <ams@gnu.org> writes:
   >
   >    > Sorry -- too many distractions while writing, I missed to add one
   >    > additional (important) note.  Started with -Q and then evaluating:
   >    >
   >    >   (add-hook 'Info-mode-hook
   >    >   	  (lambda ()
   >    >       	    (setq Info-additional-directory-list Info-default-directory-list)))
   >    >
   >    > before entering info.
   >
   >    Do you need to have both INFOPATH set and this hook set to reproduce the
   >    problem?
   >
   > Yes, exactly.

   The code here is pretty convoluted, but reading it, I think we just
   don't support doing something like that any more (since
   Info-default-directory-list isn't populated any more at this point).
   Which is why the changes here are listed under "Incompatible changes in
   Emacs 29.1".

Yes, it is convulted. :-/

But I think the situation gets slightly more convulted with this
change.  Now, there is no easy way of adding back the default
directories where Emacs searches.

The docstring could need a touch up, mentioning how
Info-default-directory-list and Info--default-directory-list work
together or hint towards it. The text is slightly misleading, since
the following is not really the case:

  This variable is used as the default for initializing
  `Info-default-directory-list' when Info is started, unless the
  environment variable INFOPATH is set.


I've tried to get the same behaviour that I had, namley that of
INFOPATH + Info-default-directory-list (in that order).  The only
thing that comes to mind is explicitly calling
Info--default-directory-list to set it up; which doesn't seem very
nice.

  (add-hook 'Info-mode-hook
  	  (lambda ()
  	    (setq Info-additional-directory-list 
  		  (if (fboundp 'Info--default-directory-list)
  		      (Info--default-directory-list)
  		    Info-default-directory-list))))
	      
If one could re-do all of this, having Info-additional-directory-list
be initialized to INFOPATH would have made most sense.  There can be
very few reasons to delete the default Emacs info directory locations,
and having INFOPATH override those always seemed backwards, and not
what stand-alone Info does -- so while the Emacs documentation says
that one should customize INFOPATH for both, the customizations aren't
really the same :-(

PS.  Shouldn't bugs become unarchived when someone replies to an
already archived one?  The discussion here is now lost in the ether...

bug#27933: emacs25: default info dir initialization slow with remote filesystems

[Lars Ingebrigtsen]

"Alfred M. Szmidt" <ams@gnu.org> writes:

> I've tried to get the same behaviour that I had, namley that of
> INFOPATH + Info-default-directory-list (in that order).  The only
> thing that comes to mind is explicitly calling
> Info--default-directory-list to set it up; which doesn't seem very
> nice.
>
>   (add-hook 'Info-mode-hook
>   	  (lambda ()
>   	    (setq Info-additional-directory-list 
>   		  (if (fboundp 'Info--default-directory-list)
>   		      (Info--default-directory-list)
>   		    Info-default-directory-list))))

Yes, that's basically what I had in mind for people who wants both.  But
in that case Info--default-directory-list should probably be renamed to
`Info-default-directory-list'...

> If one could re-do all of this, having Info-additional-directory-list
> be initialized to INFOPATH would have made most sense.  There can be
> very few reasons to delete the default Emacs info directory locations,
> and having INFOPATH override those always seemed backwards, and not
> what stand-alone Info does -- so while the Emacs documentation says
> that one should customize INFOPATH for both, the customizations aren't
> really the same :-(

Yup.  But that'd probably be even less backwards-compatible.

> PS.  Shouldn't bugs become unarchived when someone replies to an
> already archived one?  The discussion here is now lost in the ether...

Indeed; it's one of the most annoying things about debbugs.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#27933: emacs25: default info dir initialization slow with remote filesystems

[Drew Adams]

> > PS.  Shouldn't bugs become unarchived when someone replies to an
> > already archived one?  The discussion here is now lost in the
> > ether...
> 
> Indeed; it's one of the most annoying things about debbugs.

+1.

bug#27933: emacs25: default info dir initialization slow with remote filesystems

[Glenn Morris]

(Dropped unrelated Debian addresses)

Lars Ingebrigtsen wrote:

>> PS.  Shouldn't bugs become unarchived when someone replies to an
>> already archived one?  The discussion here is now lost in the ether...
>
> Indeed; it's one of the most annoying things about debbugs.

Patches can be sent to https://debbugs.gnu.org/54940

(Since the Emacs project is keen to move to another platform,
it's hard to see the motivation.)