loaddefs.el filenames as seen by `update-file-autoloads'

loaddefs.el filenames as seen by `update-file-autoloads'

[Thien-Thi Nguyen]

ciao a tutti,

under vms, filenames are case-insensitive and are upcased w/in emacs.
lisp/loaddefs.el records autoload forms that include filename fragments.
`update-file-autoloads' compares these fragments using `string=' and
fails to recognize existing entries, resulting in two entries for those
("foo" and "FOO").  i would like to add a check:

  (let ((load-name ...))
    (when (memq system-type '(vax-vms vms
                              ;; Add other systems whose filenames
                              ;; are case insensitive and for which
                              ;; `expand-file-name' DTRT, here.
			      ))
      (setq load-name (downcase load-name)))
    ...
    (when (string= load-name ...) ...))

changing `load-name' instead of making the comparison case-insensitive
means loaddefs.el will have lowercased frags (under vms), a change that
is transparent to `expand-file-name'.  is this ok?  other ideas?

thi

Re: loaddefs.el filenames as seen by `update-file-autoloads'

[Stefan Monnier]

> under vms, filenames are case-insensitive and are upcased w/in emacs.

Any reason why it's upcased ?
Is it just the tradition under VMS ?  At least under Emacs, the tradition
is rather to use lowercase for most file names, so it would make more
sense to downcase (and most places where we want to ignore case, we
downcase rather than upcase, your code below being a prime example).

Also it would be better to do what W32 does: preserve capitalization.
Is that possible ?

>   (let ((load-name ...))
>     (when (memq system-type '(vax-vms vms
>                               ;; Add other systems whose filenames
>                               ;; are case insensitive and for which
>                               ;; `expand-file-name' DTRT, here.
> 			      ))
>       (setq load-name (downcase load-name)))
>     ...
>     (when (string= load-name ...) ...))

I'd rather use file-name-equal.

Admittedly, since loaddefs.el is still under CVS control and is hence
shared between all kinds of systems, it might be better to just systematically
downcase, independently from the system in use.

Hold it.  What about the file name in the (autoload ...) form ?
Since the `autoload' might be used on case-sensitive systems, it needs
to have the right capitalization (now there's an example where
preserving capitalization in filenames (even if you ignore it)
is beneficial).

> changing `load-name' instead of making the comparison case-insensitive
> means loaddefs.el will have lowercased frags (under vms), a change that
> is transparent to `expand-file-name'.  is this ok?  other ideas?

I don't understand your references to expand-file-name.
Could you lay out what you mean ?


	Stefan

Re: loaddefs.el filenames as seen by `update-file-autoloads'

[Thien-Thi Nguyen]

   From: "Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu>
   Date: Mon, 12 May 2003 19:05:14 -0400

   Any reason why it's upcased ?
   Is it just the tradition under VMS ?

i don't know about the internals of other programs under vms, but yes,
traditionally (Richard Levitte's work) emacs has upcased filenames.  i
suppose there may be code that relies on this behavior by now.  one
notable exception is the buffer name -- that is downcased.

   At least under Emacs, the tradition is rather to use lowercase for
   most file names, so it would make more sense to downcase (and most
   places where we want to ignore case, we downcase rather than upcase,
   your code below being a prime example).

ok.

   Also it would be better to do what W32 does: preserve capitalization.
   Is that possible ?

a directory listing from the command line shows filenames all uppercase,
so upcasing everything internally is indeed preserving capitalization.
(this possibility is the current reality.)  perhaps i misunderstand you.

   I'd rather use file-name-equal.

thanks for the tip -- will check it out.  i'm working from 21.2 release
at the moment, will move to 21.3 after "make bootstrap" (actually "mms
bootstrap") works.

   Admittedly, since loaddefs.el is still under CVS control and is hence
   shared between all kinds of systems, it might be better to just
   systematically downcase, independently from the system in use.

yes, this is one important factor that i didn't mention: future
harmonization of build process, including supporting "make bootstrap"
from fresh cvs checkout.

   Hold it.  What about the file name in the (autoload ...) form ?

the cvs loaddefs.el uses lowercase.  the change i proposed would
maintain that convention.  this is possible because under vms,
`expand-file-name' upcases (and practically everything filesystem
related passes through that function at some point -- sometimes
multiply!).

   Since the `autoload' might be used on case-sensitive systems, it
   needs to have the right capitalization (now there's an example where
   preserving capitalization in filenames (even if you ignore it) is
   beneficial).

well, unixoids are case-sensitive.  we sidestep the problem there by
adhering in practice to lowercase filenames that moreover are unique
amongst themselves w/o regard to case.  the proposed change is not for
all systems, but only those that are vms-like.

   > changing `load-name' instead of making the comparison case-insensitive
   > means loaddefs.el will have lowercased frags (under vms), a change that
   > is transparent to `expand-file-name'.  is this ok?  other ideas?

   I don't understand your references to expand-file-name.
   Could you lay out what you mean ?

here is an example under vms:

  (expand-file-name "foo.el") => "AXPA:[TTN.EMACS.EMACS212_3]FOO.EL"

because autoloading eventually triggers loading which eventually uses
`expand-file-name', we don't need to store "FOO.EL" in loaddefs.el to
actually access "FOO.EL" on disk; "foo.el" is sufficient (under vms).

thi

Re: loaddefs.el filenames as seen by `update-file-autoloads'

[Stefan Monnier]

>    From: "Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu>
>    Date: Mon, 12 May 2003 19:05:14 -0400
> 
>    Any reason why it's upcased ?
>    Is it just the tradition under VMS ?
> 
> i don't know about the internals of other programs under vms, but yes,
> traditionally (Richard Levitte's work) emacs has upcased filenames.  i
> suppose there may be code that relies on this behavior by now.  one
> notable exception is the buffer name -- that is downcased.
> 
>    At least under Emacs, the tradition is rather to use lowercase for
>    most file names, so it would make more sense to downcase (and most
>    places where we want to ignore case, we downcase rather than upcase,
>    your code below being a prime example).
> 
> ok.
> 
>    Also it would be better to do what W32 does: preserve capitalization.
>    Is that possible ?
> 
> a directory listing from the command line shows filenames all uppercase,
> so upcasing everything internally is indeed preserving capitalization.
> (this possibility is the current reality.)  perhaps i misunderstand you.

Under W32, the file-system preserves the capitalization, even though
it ignores it when searching for files.  I.e. you can't have both "Foo"
and "FOO", but after creating "Foo", an application can know whether
the file was created as "Foo" or "foo" or "FOO".

If `directory-files' *can* properly return ("ChangeLog" "README" "configure"),
rather than ("changelog" "readme" "configure")
or ("CHANGELOG" "README" "CONFIGURE"), without having to read
the user's mind (and without performance cost) then I think it should.

>    I'd rather use file-name-equal.
> 
> thanks for the tip -- will check it out.  i'm working from 21.2 release
> at the moment, will move to 21.3 after "make bootstrap" (actually "mms
> bootstrap") works.

It doesn't exist yet.
It's been discussed but has always been shot down in the end until now.

>    Hold it.  What about the file name in the (autoload ...) form ?
> 
> the cvs loaddefs.el uses lowercase.

Not really, it just so happens that all the files are lowercase
except for lisp/term/AT386.el.

work/emacs-0% grep -i at386 lisp/loaddefs.el
;;;;;;  "subdirs.el" "subr.el" "tempo.el" "term/AT386.el" "term/apollo.el"
work/emacs-0% 

> the change i proposed would
> maintain that convention.  this is possible because under vms,
> `expand-file-name' upcases (and practically everything filesystem
> related passes through that function at some point -- sometimes
> multiply!).

Why do you need to upcase in expand-file-name ?
Isn't VMS happy either way ?

>    Since the `autoload' might be used on case-sensitive systems, it
>    needs to have the right capitalization (now there's an example where
>    preserving capitalization in filenames (even if you ignore it) is
>    beneficial).
> 
> well, unixoids are case-sensitive.  we sidestep the problem there by
> adhering in practice to lowercase filenames that moreover are unique
> amongst themselves w/o regard to case.  the proposed change is not for
> all systems, but only those that are vms-like.

update-file-autoloads is not used exclusievly for lisp/loaddefs.el:
it is also used by some outside elisp packages.  Admittedly, they
probably also use lowercase filenames.

>   (expand-file-name "foo.el") => "AXPA:[TTN.EMACS.EMACS212_3]FOO.EL"
> 
> because autoloading eventually triggers loading which eventually uses
> `expand-file-name', we don't need to store "FOO.EL" in loaddefs.el to
> actually access "FOO.EL" on disk; "foo.el" is sufficient (under vms).

Since VMS is case-insensitive, it would work just as well if
expand-file-name returned "AXPA:[TTN.EMACS.EMACS212_3]FOO.EL".

Of course maybe some other part of the Emacs/VMS code might rely on
upcased filenames, but I doubt VMS itself does.


	Stefan

Re: loaddefs.el filenames as seen by `update-file-autoloads'

[Miles Bader]

"Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> writes:
> Why do you need to upcase in expand-file-name ?
> Isn't VMS happy either way ?

I always assumed it was because doing so gave VMS users warm fuzzies...

[It's been probably 15 years since I used VMS (or any other DEC OS), but
lower-case filenames there feel sort of `less real,' since most apps
that canonicalize filenames also upcase them.]

-Miles
-- 
Run away!  Run away!

Re: loaddefs.el filenames as seen by `update-file-autoloads'

[Thien-Thi Nguyen]

   From: "Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu>
   Date: Mon, 12 May 2003 21:14:13 -0400

   Under W32, the file-system preserves the capitalization, even though
   it ignores it when searching for files.  I.e. you can't have both "Foo"
   and "FOO", but after creating "Foo", an application can know whether
   the file was created as "Foo" or "foo" or "FOO".

   If `directory-files' *can* properly return ("ChangeLog" "README"
   "configure"), rather than ("changelog" "readme" "configure") or
   ("CHANGELOG" "README" "CONFIGURE"), without having to read the user's
   mind (and without performance cost) then I think it should.

ok, i see what you mean.  i see that `readdir' for vms (in sysdep.c)
does no case translation, and stepping through that function a few times
in the debugger shows the filename to be upcased from the beginning.  it
looks like vms that i am using follows the last of the three models
above.

   Why do you need to upcase in expand-file-name ?
   Isn't VMS happy either way ?

i don't know if it's necessary; perhaps for consistency?  after a while,
the "vms mindset" of upcase filenames is not as disturbing as initially.
what is more disturbing is the case folding for dcl command procedure
arguments, necessitating quoting, which has its own share of freaky and
baroque idiosyncrasies, but i digress...

   update-file-autoloads is not used exclusievly for lisp/loaddefs.el:
   it is also used by some outside elisp packages.  Admittedly, they
   probably also use lowercase filenames.

yeah, i have some familiarity from use in ttn-pers-elisp, although this
is the first time i've looked at the case issue.

   >   (expand-file-name "foo.el") => "AXPA:[TTN.EMACS.EMACS212_3]FOO.EL"

   Since VMS is case-insensitive, it would work just as well if
   expand-file-name returned "AXPA:[TTN.EMACS.EMACS212_3]FOO.EL".

i presume you mean "axpa:[ttn.emacs.emacs212_3]foo.el", as currently
`expand-file-name' does indeed upcase.

thi

Re: loaddefs.el filenames as seen by `update-file-autoloads'

[Stefan Monnier]

>    From: "Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu>
>    Date: Mon, 12 May 2003 21:14:13 -0400
> 
>    Under W32, the file-system preserves the capitalization, even though
>    it ignores it when searching for files.  I.e. you can't have both "Foo"
>    and "FOO", but after creating "Foo", an application can know whether
>    the file was created as "Foo" or "foo" or "FOO".
> 
>    If `directory-files' *can* properly return ("ChangeLog" "README"
>    "configure"), rather than ("changelog" "readme" "configure") or
>    ("CHANGELOG" "README" "CONFIGURE"), without having to read the user's
>    mind (and without performance cost) then I think it should.
> 
> ok, i see what you mean.  i see that `readdir' for vms (in sysdep.c)
> does no case translation, and stepping through that function a few times
> in the debugger shows the filename to be upcased from the beginning.  it
> looks like vms that i am using follows the last of the three models
> above.

In that case, it probably makes sense to keep all file names
upcased, indeed.

But that means that there is no way we can reliably use on Unix
a loaddefs.el generated on VMS.  So I think the thing we should
do is use something like `file-name-equal' rather than string=.


	Stefan

Re: loaddefs.el filenames as seen by `update-file-autoloads'

[Thien-Thi Nguyen]

"Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> writes:

   But that means that there is no way we can reliably use on Unix
   a loaddefs.el generated on VMS.  So I think the thing we should
   do is use something like `file-name-equal' rather than string=.

good point.  hmmm, the filename "datapath" is currently:

  directory-files
  system-specific munging   ;; proposed, e.g., downcase for vms
  loaddefs.el
  string=

we can make the design more reliable by explicitly specifying the
filename to be recorded in loaddefs.el (the proper comparison operation
becomes apparent w/ a well-specified data format).  that is, follow
system-specific munging w/ "filename specification override", such as
via a local variable, or by consulting the stylized top line:

   ;;; FILENAME.el --- do something

already the file needs to be scanned for local variables, so it seems to
me that approach is best.  this would mean lisp/term/AT386.el would need
to add something like:

   ;; Local variables:
   ;; autoload-recorded-file-name: "term/AT386.el"
   ;; End:

that doesn't seem too onerous.

thi