auto-update of Info dir file?

auto-update of Info dir file?

[Drew Adams]

I haven't thought this through much or tried to investigate how it might
work, but it occurred to me that `info' might do something like this:

For each directory in `Info-directory-list', for each top-level file in the
directory (i.e. whose name doesn't end in "-N", N=1,2,3...): Check if there
is a corresponding entry in the Info `dir' file (in
`Info-dir-contents-directory'), and, if not, add one.

Perhaps there is already part of the mechanism for doing this. When I look
in some Info files now I see (START|END)-INFO-DIR-ENTRY, which makes me
think there must already be a means of adding the entry. Perhaps all that's
missing is to have `info' do that on the fly automatically for each
top-level Info file in the directories of `Info-directory-list'.

Some Info files might not have a (START|END)-INFO-DIR-ENTRY, in which case
only a simple entry would be added - say, using the file name as the link
itself and leaving the description blank. (Or perhaps a description could be
gleaned from the Info file.)

I mention this because of a question in help-gnu-emacs today and a reply
advising the user to add the entry to the `dir' file. Why not have Emacs do
that automatically, so the only user task would be to add the Info files to
the directory?

If this were costly in performance, it might be done only on first use of
`info' in a session.

Re: auto-update of Info dir file?

[Andreas Schwab]

"Drew Adams" <drew.adams@oracle.com> writes:

> Perhaps there is already part of the mechanism for doing this.

See install-info.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

RE: auto-update of Info dir file?

[Drew Adams]

    > Perhaps there is already part of the mechanism for doing this.

    See install-info.

I don't see it, but I believe you. As I said, the (START|END)-INFO-DIR-ENTRY
made me think that a means of adding the entry might already exist. I guess
that's what you're confirming.

Is this means (`install-info') within Emacs itself or is it part of a make
file or shell script? I was thinking of having Emacs `info' do this for
itself automatically.

I'm not looking to implement this myself. I thought it might be a useful
mini-feature to add after the release. Might be simple to do, and could
avoid some user confusion (and editing `dir' could be error-prone).

Re: auto-update of Info dir file?

[Andreas Schwab]

"Drew Adams" <drew.adams@oracle.com> writes:

> Is this means (`install-info') within Emacs itself or is it part of a make
> file or shell script?

install-info is part of the texinfo suite.  It's purpose is to update the
dir file.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: auto-update of Info dir file?

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Date: Mon, 15 May 2006 08:28:50 -0700
> 
> For each directory in `Info-directory-list', for each top-level file in the
> directory (i.e. whose name doesn't end in "-N", N=1,2,3...): Check if there
> is a corresponding entry in the Info `dir' file (in
> `Info-dir-contents-directory'), and, if not, add one.

I think this will slow down Info startup to an intolerable degree.
For example, I have more than 70 such top-level Info files in my Info
directory, some of them compressed.

> Perhaps there is already part of the mechanism for doing this. When I look
> in some Info files now I see (START|END)-INFO-DIR-ENTRY, which makes me
> think there must already be a means of adding the entry. Perhaps all that's
> missing is to have `info' do that on the fly automatically for each
> top-level Info file in the directories of `Info-directory-list'.

When a package is installed, its "make install" target should
automatically run the install-info program, which updates the menu in
DIR.  So, for systems that are well set up, this problem does not
exist, and I don't see why we should punish users of such systems on
behalf of the other kind.

> I mention this because of a question in help-gnu-emacs today and a reply
> advising the user to add the entry to the `dir' file.

That user got half the world and their dog mixed up.  I wouldn't
recommend jumping to conclusions based on that case alone.

> If this were costly in performance, it might be done only on first use of
> `info' in a session.

It will be still painful, IMO.

Re: auto-update of Info dir file?

[Stefan Monnier]

>> For each directory in `Info-directory-list', for each top-level file in the
>> directory (i.e. whose name doesn't end in "-N", N=1,2,3...): Check if there
>> is a corresponding entry in the Info `dir' file (in
>> `Info-dir-contents-directory'), and, if not, add one.

It may happen that some info files are in a directory where there are other
files as well.  If we could rely on the .info suffix it would help, but
sadly it is common for info files to have no extension.

> I think this will slow down Info startup to an intolerable degree.
> For example, I have more than 70 such top-level Info files in my Info
> directory, some of them compressed.

It's probably OK if we only do the check when the mtime of the parent dir is
more recent than the mtime of the dir file (and only under Unix, of course,
since this presumes POSIXy semantics).


        Stefan

Re: auto-update of Info dir file?

[Miles Bader]

Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> I think this will slow down Info startup to an intolerable degree.
>> For example, I have more than 70 such top-level Info files in my Info
>> directory, some of them compressed.
>
> It's probably OK if we only do the check when the mtime of the parent dir is
> more recent than the mtime of the dir file (and only under Unix, of course,
> since this presumes POSIXy semantics).

I don't understand why Drew's suggestion would be slow anyway -- it's a
simple syntactic check, which would only use the _names_ of the info
files in the directory listing.  It wouldn't even have to stat the info
files, much less read or decompress them.

-Miles
-- 
Any man who is a triangle, has thee right, when in Cartesian Space, to
have angles, which when summed, come to know more, nor no less, than
nine score degrees, should he so wish.  [TEMPLE OV THEE LEMUR]

Re: auto-update of Info dir file?

[David Kastrup]

Miles Bader <miles.bader@necel.com> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>>> I think this will slow down Info startup to an intolerable degree.
>>> For example, I have more than 70 such top-level Info files in my Info
>>> directory, some of them compressed.
>>
>> It's probably OK if we only do the check when the mtime of the parent dir is
>> more recent than the mtime of the dir file (and only under Unix, of course,
>> since this presumes POSIXy semantics).
>
> I don't understand why Drew's suggestion would be slow anyway -- it's a
> simple syntactic check, which would only use the _names_ of the info
> files in the directory listing.  It wouldn't even have to stat the info
> files, much less read or decompress them.

Depends on how the stuff is done.  One can easily make an O(n^2) or
worse algorithm for this kind of thing.  Probably the fastest way
would be to stuff all menu entries and then files unsorted into one
list, sort that list (with `sort' which is stable), and then go
through it once, entering all files into the menu which are not
immediately preceded by a menu entry in the list.

This would probably be faster and use less memory than using a hash.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: auto-update of Info dir file?

[Eli Zaretskii]

> Cc: Eli Zaretskii <eliz@gnu.org>, Drew Adams <drew.adams@oracle.com>,
>         emacs-devel@gnu.org
> From: Miles Bader <miles.bader@necel.com>
> Date: Tue, 16 May 2006 13:17:17 +0900
> 
> I don't understand why Drew's suggestion would be slow anyway -- it's a
> simple syntactic check, which would only use the _names_ of the info
> files in the directory listing.  It wouldn't even have to stat the info
> files, much less read or decompress them.

Really?  We probably have two different implementations in mind.  I
was thinking about finding in the file and adding to the top-level
menu the missing @direntry text in the right @dircategory, and for
that, you do have to read the file and decompress it.

Re: auto-update of Info dir file?

[Miles Bader]

Eli Zaretskii <eliz@gnu.org> writes:
>> I don't understand why Drew's suggestion would be slow anyway -- it's a
>> simple syntactic check, which would only use the _names_ of the info
>> files in the directory listing.  It wouldn't even have to stat the info
>> files, much less read or decompress them.
>
> Really?  We probably have two different implementations in mind.  I
> was thinking about finding in the file and adding to the top-level
> menu the missing @direntry text in the right @dircategory, and for
> that, you do have to read the file and decompress it.

Well my idea was that _most_ files would have proper dir entries, so
you'd only have to read the @direntry to supply entries for the few (if
any) that didn't.

So the only real overhead would be proportional to the number of info
files with missing entries, not to the total number of info files.

Hopefully the former number is very small (and as far as I've seen, this
is the case).  If it _isn't_ small then you've clearly got serious
problems with your info setup anyway, and it's arguably better for emacs
to bite the bullet and spend the time to make things usable rather than
giving the user a bogus info dir.

-Miles
-- 
80% of success is just showing up.  --Woody Allen

Re: auto-update of Info dir file?

[Eli Zaretskii]

> Cc: "Drew Adams" <drew.adams@oracle.com>,  emacs-devel@gnu.org
> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Mon, 15 May 2006 23:54:06 -0400
> 
> It's probably OK if we only do the check when the mtime of the parent dir is
> more recent than the mtime of the dir file (and only under Unix, of course,
> since this presumes POSIXy semantics).

Is it indeed guaranteed that the mtime of the directory is always
_after_ the last file modified in that directory?

Anyway, this sounds too fragile, as someone could touch or otherwise
affect the modtimes of the files and the directory, e.g. by removing
files or compressing them rather than adding.  And then what about
info/ subdirs?

I think we could solve the problem much easier: if the user wants a
manual called "mumble", and there's no such entry in DIR, look for a
_file_ called `mumble' with several possible extensions.  This is what
the stand-alone Info does, so adding this to Emacs will increase
compatibility between the two readers.  Doing this bears no run-time
penalty, and almost the same benefits for the user.

RE: auto-update of Info dir file?

[Drew Adams]

    I think we could solve the problem much easier: if the user wants a
    manual called "mumble",

What do you mean here by "the user wants"? Are you speaking of the user who
executes Emacs command `info' or the user (perhaps a sysadmin or programmer,
not necessarily the end-user of Emacs) who adds the Info files to the
directory?

The end user might not know that s?he wants a particular manual. If the
manual doesn't appear in the Info list (catch-22), then the user might not
even be aware that the manual exists. The idea was to have invocation of
Emacs command `info' populate Info with all available manuals, without the
end user needing to know anything about what they are.

Also, how will Emacs tell what "the user wants" in this context? Even if the
end user does know that s?he wants a particular manual (and that it should
be available), how is that communicated to Emacs in your scenario?

    and there's no such entry in DIR, look for a
    _file_ called `mumble' with several possible extensions.  This is what
    the stand-alone Info does, so adding this to Emacs will increase
    compatibility between the two readers.  Doing this bears no run-time
    penalty, and almost the same benefits for the user.

I think this should be push rather than pull. That is, populating Info
should not be demand-driven by the user; it should be done automatically,
based on all available Info files. It can be demand-driven in terms of
_when_ it happens (dynamically, when the user invokes Emacs command `info'
for the first time in a session), but not in terms of _which_ manuals end up
populating the `dir' file.

Re: auto-update of Info dir file?

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Date: Tue, 16 May 2006 10:54:09 -0700
> 
>     I think we could solve the problem much easier: if the user wants a
>     manual called "mumble",
> 
> What do you mean here by "the user wants"? Are you speaking of the user who
> executes Emacs command `info' or the user (perhaps a sysadmin or programmer,
> not necessarily the end-user of Emacs) who adds the Info files to the
> directory?

The former.

> The end user might not know that s?he wants a particular manual.

??? Really?  Tell me, when you want to read some manual, how do you
get to it if not by "C-h i d m MUMBLE <RET>" (or something
equivalent)?  In other words, you _always_ name the manual that you
want to read.

> If the
> manual doesn't appear in the Info list (catch-22), then the user might not
> even be aware that the manual exists.

How long is your top-level menu?  Mine is _very_ long, and the same
will happen to anyone who has glibc docs installed, because they add
an entry for each and every function in the library.  But even if
there's no glibc entries in DIR, a garden variety top-level menu is
quote long: for example, the one on gnu.org machines has 623 lines.

So, to avoid wasting precious time, I _never_ scan the menu to find
whether a topic I'm looking for is there, I type "m SOMETHING <RET>"
and watch the result: if there's no such topic, Info will bitch at me.

My suggestion is to try SOMETHING.info etc. if DIR has no menu item by
that name.

> The idea was to have invocation of
> Emacs command `info' populate Info with all available manuals, without the
> end user needing to know anything about what they are.

And I thought the idea was to allow the user to reach a manual even
though it was not installed by `install-info'.  Populating the
top-level menu with all manuals is a means to an end; I suggested a
different means to the very same end, I think.

> Also, how will Emacs tell what "the user wants" in this context? Even if the
> end user does know that s?he wants a particular manual (and that it should
> be available), how is that communicated to Emacs in your scenario?

It's the string the user types at `m's prompt in the top-level menu.

>     and there's no such entry in DIR, look for a
>     _file_ called `mumble' with several possible extensions.  This is what
>     the stand-alone Info does, so adding this to Emacs will increase
>     compatibility between the two readers.  Doing this bears no run-time
>     penalty, and almost the same benefits for the user.
> 
> I think this should be push rather than pull. That is, populating Info
> should not be demand-driven by the user; it should be done automatically,
> based on all available Info files. It can be demand-driven in terms of
> _when_ it happens (dynamically, when the user invokes Emacs command `info'
> for the first time in a session), but not in terms of _which_ manuals end up
> populating the `dir' file.

I didn't say anything about populating the menu.  My suggestion
involves no modifications to the menu, Info will just find the manual
even if it isn't in the menu.

To see how the stand-alone Info reader handles this, type at the
shell's prompt "info info-stnd".  There should be no menu entry that
begins with the string "info-stnd" in your DIR file, but the
stand-alone reader will find the right manual anyway.

RE: auto-update of Info dir file?

[Drew Adams]

    > The end user might not know that s?he wants a particular manual.

    ??? Really?

Really.

    Tell me, when you want to read some manual, how do you
    get to it if not by "C-h i d m MUMBLE <RET>" (or something
    equivalent)?  In other words, you _always_ name the manual that you
    want to read.

I don't want to belabor this, but my answer is no, I do not use `C-h i d m'
or `C-h i m' very often, and I suspect that some other users might not
either. I use `C-h i' and then click the manual's link in the menu.

Of course that's equivalent, but some users will use Info in an exploratory
fashion, to discover more of what is available. I have probably never looked
at at least 2/3 of the manuals that are in my `dir', but I might someday. If
I do, the first thing I will do is browse the list to see what I might be
interested in. That requires that the list be complete (populated).
Preferably, the list would be complete not only with file names but also
with brief descriptions (perhaps constructed automatically).

Yes, the `m' command, if fixed as you proposed (a la standalone `info'),
would let people explore via completion, so that's a good start. But why not
populate the _visible_ menu too, while we're at it? Why populate only the
invisible "menu" provided by `m' completion?

When you go to the library or bookstore, do you always go looking for a
particular book? Don't you sometimes browse the shelves to see what might be
interesting?

    > If the manual doesn't appear in the Info list (catch-22),
    > then the user might not even be aware that the manual exists.

    How long is your top-level menu?

On Emacs 20 on Windows: 28 manuals.
On Emacs 22 on Windows: 41 manuals.

    Mine is _very_ long, and the same
    will happen to anyone who has glibc docs installed, because they add
    an entry for each and every function in the library.  But even if
    there's no glibc entries in DIR, a garden variety top-level menu is
    quote long: for example, the one on gnu.org machines has 623 lines.

Well, yours is certainly longer than mine, Eli! ;-)

    So, to avoid wasting precious time, I _never_ scan the menu to find
    whether a topic I'm looking for is there, I type "m SOMETHING <RET>"
    and watch the result: if there's no such topic, Info will bitch at me.

Different users have different use patterns. Different users have different
menus. Different users have different sizes. I've been known to browse a
bookstore or library no matter how big it is (Library of Congress?). Call me
inefficient.

You really raise a different issue, I think, which is how to help users
manage a long `dir' menu? It might be useful to brainstorm ideas in that
area. Perhaps structuring the menu more? Perhaps providing a special search
functionality for it? I agree that the completion available via the `m'
command is useful in this regard (<ad>especially with Icicles
completion!</ad>), but perhaps there is room for additional help.

Library and bookstore card catalogs and computerized searches for books make
it possible to manage even _humongous_ inventories of books as an end user.
And they are not limited to, say, the equivalent of `C-h i d m
<ISBN-number>'.

Nevertheless, there is still a need (rather, a preference) on the part of
some users in some libraries on some rainy days to browse some of the stacks
and peruse some of the books. Some people never learn (`C-h i d m').

    > The idea was to have invocation of Emacs command `info'
    > populate Info with all available manuals, without the
    > end user needing to know anything about what they are.

    And I thought the idea was to allow the user to reach a manual even
    though it was not installed by `install-info'.

That's a consequence of the idea I proposed, which was to populate the menu
automatically. And yes, that is desirable. But it's not the full goal.

    Populating the top-level menu with all manuals is a means
    to an end; I suggested a different means to the very same end,
    I think.

The end is not the same if the end is reduced to a limited way (invisible
menu) to "reach a manual". I would like to see the visible menu populated
too, so users, especially novice users, can browse what they see there -
that's part of the end pursued by my proposal, it's not just a means to it.

    > Also, how will Emacs tell what "the user wants" in this
    > context? Even if the end user does know that s?he wants
    > a particular manual (and that it should
    > be available), how is that communicated to Emacs in your scenario?

    It's the string the user types at `m's prompt in the top-level menu.

Too limiting; see above.

    > I think this should be push rather than pull. That is, populating
    > Info should not be demand-driven by the user; it should be done
    > automatically, based on all available Info files. It can be
    > demand-driven in terms of _when_ it happens (dynamically, when
    > the user invokes Emacs command `info' for the first time in a
    > session), but not in terms of _which_ manuals end up
    > populating the `dir' file.

    I didn't say anything about populating the menu.  My suggestion
    involves no modifications to the menu, Info will just find the manual
    even if it isn't in the menu.

_I_ did say something about populating the menu, right from the start. My
proposal is different from yours, not only in the means, but also in the end
pursued. Making sure that completion for `m' has the complete list of books
is necessary, but it is not sufficient. The visible menu should also be
complete.

    To see how the stand-alone Info reader handles this, type at the
    shell's prompt "info info-stnd".  There should be no menu entry that
    begins with the string "info-stnd" in your DIR file, but the
    stand-alone reader will find the right manual anyway.

Yes, I have no problem with that, as far as it goes. I would also like to
see the visible menu show what's available, so users don't need a crystal
ball before they can know to type "info info-stnd". Even if seen in a
completion list, "info-stnd" is hardly parlant. The same thing goes for the
visible menu: It would be good to find some way to add a (constructed)
description for the books added automatically to the (visible) menu.

Ideally (another, future discussion, perhaps), users should be able to type
keywords or a regexp and see a summary of the hits among all manuals - to
see which manuals might be available and most appropriate for info on a
particular subject.

To me, this is about helping users to know which manuals are available. It
is not just about "reaching a manual" that you know you want to read.

Re: auto-update of Info dir file?

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Date: Tue, 16 May 2006 12:33:39 -0700
> 
>     To see how the stand-alone Info reader handles this, type at the
>     shell's prompt "info info-stnd".  There should be no menu entry that
>     begins with the string "info-stnd" in your DIR file, but the
>     stand-alone reader will find the right manual anyway.
> 
> Yes, I have no problem with that, as far as it goes. I would also like to
> see the visible menu show what's available, so users don't need a crystal
> ball before they can know to type "info info-stnd". Even if seen in a
> completion list, "info-stnd" is hardly parlant.

Don't take the example too literally: I've chosen info-stnd because I
could be sure the example will work on your system.  In the situation
that started this thread, it would have been "info elisp", which
doesn't really need a crystal ball at all.

> Ideally (another, future discussion, perhaps), users should be able to type
> keywords or a regexp and see a summary of the hits among all manuals - to
> see which manuals might be available and most appropriate for info on a
> particular subject.

You want "M-x info-apropos RET".  We already have that.

Re: auto-update of Info dir file?

[Richard Stallman]

Any change in this mechanism would involve Makeinfo as well as Emacs,
and might also involve changing all GNU packages.  It would not be
worth the trouble, so I am not going to do it.

Could we please drop the subject?

Re: auto-update of Info dir file?

[Stefan Monnier]

>> It's probably OK if we only do the check when the mtime of the parent dir is
>> more recent than the mtime of the dir file (and only under Unix, of course,
>> since this presumes POSIXy semantics).

> Is it indeed guaranteed that the mtime of the directory is always
> _after_ the last file modified in that directory?

No.  POSIX only says that the mtime of the dir reflects the last time a file
was added/removed from the directory, which is pretty much exactly what we
need here.  Lucky for us: if updating the dir file would cause the dir's
mtime stamp to be changed would lead us to lalaland.


        Stefan